llamafile README

The project's README from github.com/mozilla-ai/llamafile. As one markdown file: README.md.

llamafile

[line drawing of llama animal head in front of slightly open manilla folder filled with files]

License ci status Based on llama.cpp Based on whisper.cpp Discord Mozilla Builders

llamafile lets you distribute and run LLMs with a single file.

llamafile is a Mozilla Builders project (see its announcement blog post), now revamped by Mozilla.ai.

Our goal is to make open LLMs much more accessible to both developers and end users. We're doing that by combining llama.cpp with Cosmopolitan Libc into one framework that collapses all the complexity of LLMs down to a single-file executable (called a "llamafile") that runs locally on most operating systems and CPU architectures, with no installation.

llamafile also includes whisperfile, a single-file speech-to-text tool built on whisper.cpp and the same Cosmopolitan packaging. It supports transcription and translation of audio files across all the same platforms, with no installation required.

v0.10.*

llamafile versions starting from 0.10.0 use a new build system, aimed at keeping our code more easily aligned with the latest versions of llama.cpp. This means they support more recent models and functionalities, but at the same time they might be missing some of the features you were accustomed to (check out this doc for a high-level description of what has been done). If you liked the "classic experience" more, you will always be able to access the previous versions from our releases page. Our pre-built llamafiles always show which version of the server they have been bundled with (0.9.* example, 0.10.* example), so you will always know which version of the software you are downloading.

We want to hear from you! Whether you are a new user or a long-time fan, please share what you find most valuable about llamafile and what would make it more useful for you. Read more via the blog and add your voice to the discussion here.

Quick Start

Download and run your first llamafile in minutes:

# Download an example model (Qwen3.5 0.8B)
curl -LO https://huggingface.co/mozilla-ai/llamafile_0.10/resolve/main/Qwen3.5-0.8B-Q8_0.llamafile

# Make it executable (macOS/Linux/BSD)
chmod +x Qwen3.5-0.8B-Q8_0.llamafile

# Run it
./Qwen3.5-0.8B-Q8_0.llamafile

We chose this model because that's the smallest one we have built a llamafile for, so most likely to work out-of-the-box for you. If you have powerful hardware and/or GPUs, feel free to choose larger and more expressive models which should provide more accurate responses.

Windows users: Rename the file to add .exe extension before running.

Note - Only executables under 4GB can run on Windows, so any llamafile above 4GB won't work. Download the llamafile binary and run it with any external weights/models(GGUF).

Documentation

Check the full documentation at docs.mozilla.ai/llamafile, or directly jump into one of the following subsections:

Licensing

While the llamafile project is Apache 2.0-licensed, our changes to llama.cpp and whisper.cpp are licensed under MIT (just like the projects themselves) so as to remain compatible and upstreamable in the future, should that be desired.

The llamafile logo on this page was generated with the assistance of DALL·E 3.

Star History Chart

llamafile (mozilla-ai/llamafile)

Searching Database…

26.4K ·GitHub
Llamafile
这是什么?llamafile 能把开源大语言模型打包成单文件可执行程序:只需一个文件,无需安装,就能在大多数操作系统和 CPU 架构上本地运行聊天模型。它是 Mozilla Builders 的项目,现在由 Mozilla.ai 重新维护。
它是我用来开发东西的框架,还是下载就能用的现成程序?两者都是:它的底层是 llama.cpp 加上 Cosmopolitan Libc,但作为用户,你只需要下载一个预构建好的 llamafile 直接运行就行。同一个程序也能加载你已有的外部 GGUF 模型权重。
我怎么把它跑起来?下载一个预构建好的 llamafile,直接运行即可——没有安装步骤。在 Windows 上,要先给文件重命名,加上 .exe 扩展名。建议从最小的那个模型开始,它开箱即用的可能性最大。
有什么需要留意的坑吗?有:在 Windows 上,只有小于 4GB 的可执行文件能运行,所以更大的 llamafile 在 Windows 上用不了。另外,从 v0.10 开始,构建系统被重写以跟进新版 llama.cpp,因此经典版本的一些功能可能已经没有了。
它会调用云端 API 吗,还是所有东西都在我自己电脑上?一切都在你自己的硬件上本地运行——README 里描述的整个流程没有任何云服务,也不需要注册账号。
Cosmopolitan Libc 在这里到底起什么作用?正是它让同一个文件到处都能运行:llama.cpp 与它链接后,生成的可执行文件可以在大多数操作系统和 CPU 架构上原生运行——这也是为什么什么都不用安装。
如果 v0.10 去掉了我需要的功能,老版本还能下载吗?可以——以前的版本仍然可以在 releases 页面上找到。预构建的 llamafile 还会标明它内置的服务器版本(0.9.x 或 0.10.x),这样你下载前就知道拿到的是哪个版本。
为什么不直接运行 llama.cpp,而要做成单个文件?官方给出的目标是降低使用门槛:把大语言模型的所有复杂细节都收进一个可执行文件,让开源模型对开发者和普通用户来说都更容易运行和分享。
What are similar projects?project-maya, Strata, jev, Janus, ODS, rizzo-flow, gpt4all, mlx-serve
Which topics is it in?llamacpp, llama-cpp, gguf, local-ai, whisper, speech-to-text, llm-inference, local-llm, inference, cpp, c++, gpu
Creator
mozilla-ai
Size
27.5 MB
Stars
26.4K
Updated
CommitOct 08Release0.10.6Sep 15
Age
3y
Commits · 90 Days