Slotstream runs Qwen3.8-Flash-Next on your Mac by keeping most of the model
on SSD and loading the parts it needs into memory. After a one-time download,
the model works offline. No Python or cloud API account is required.
Use it to chat, ask about pictures, or work with files through an agent such
as Hermes. Developers can connect their own apps through its APIs or Swift library.
I'm building Sevra on Slotstream: private, personal AI optimized for your computer.
Sevra will choose a tested model for your hardware and keep that choice current
as models improve, with inference, memory and tools tuned together. You'll
control what it remembers and can access. The app is in development, starting
with Mac.
See Sevra and join the waitlist.
Slotstream's command-line tool, APIs, and Swift library remain independently usable.
Will it run on my Mac?
You need an Apple Silicon Mac, macOS 14 or later, and about 110 GB of free
SSD space. Choose Apple menu → About This Mac to check your chip and memory.
Smaller Macs can run the model too, but speed depends on the chip, SSD, and
available memory. An 8 GB Mac needs swap and can slow down the whole computer.
The installer has been tested on macOS 14 and 15; model runs have been tested
on macOS 26. Windows, Linux, and Intel Macs are not currently supported.
Speed
These reply speeds were measured on real Macs after the model warmed up.
tok/s means tokens per second; a token is a small piece of text, often
part of a word.
Mac
Memory
Reply speed
MacBook Pro, M5 Pro
48 GB
~12 tok/s
Mac mini, M2 (base storage)
16 GB
1.41 tok/s
MacBook Air, M5
32 GB
6.22 tok/s
MacBook Pro, M5 Max
128 GB
~21–22 tok/s
The M5 Pro is the development Mac; the other rows are community reports.
They use different releases and settings, including speculative decoding on
the M5 Max. See hardware results and test conditions for
credits, methods, and separate estimates for other Macs.
Starting a reply takes additional time. Slotstream must first process
your question and conversation history. Long prompts can take minutes;
follow-up turns can reuse unchanged history. Terminal shows progress while
you wait.
Install
Open Terminal and paste this command:
curl -fsSL https://raw.githubusercontent.com/carloslfu/slotstream/main/install.sh | sh
Run the same command to update. If slotstream isn't found afterward, open
a new terminal window.
Use it
Check your Mac, then ask for a first reply:
slotstream doctor
slotstream run --prompt "Why is the sky blue?"
doctor checks memory and disk space without loading the model. The first
run asks to download it, then prints a reply. This download can take hours,
but you only need to do it once. Interrupted downloads resume when you try
again. Follow the step-by-step setup for more help.
Install chat apps and agents separately. They provide the interface and tools;
Slotstream runs the model. Keep its server running while a connected app uses it.
For developers, the engineering guide links to the
OpenAI- and Ollama-compatible API references, Swift library, command options,
build instructions, and tests. Release notes show what changed.
How it works
Qwen3.8-Flash-Next is a mixture-of-experts model: generating each piece of
text uses only a subset of its expert networks. Slotstream keeps shared
weights in memory and reads the needed experts from SSD into a cache.
Frequently used experts stay in RAM, reducing repeated disk reads.
The whole model stays available even though it doesn't all fit in memory.
Slotstream chooses a memory target for your Mac and adjusts its cache as
other apps need room. Cache size changes speed without removing experts
from the model. The engineering explanation
covers the implementation.
Status and limits
One generation at a time: connected apps share the same running model.
Conversation length is limited: longer histories take more memory and
time. The Hermes guide includes the larger conversation window it needs.
Testing: image input and tool calling have
integration tests, but there is no broad image-accuracy benchmark or
completed comparison with other models on the same Mac.
FAQ
Does it work offline?
Yes, after downloading the model. Inference runs on your Mac. Connected
agents may still use internet services for web searches or other tools;
their settings determine what those tools send.
Why doesn't Slotstream use all of my RAM?
Auto intentionally caps the total memory target at 33 GB, or 34.6 GB
with speculative decoding enabled, even on larger Macs. This is our current
best-supported balance of speed and memory use for this model: tests on the
development Mac showed diminishing speed gains from a larger expert cache.
We'll adjust the default as real measurements across hardware and workloads
show a better tradeoff. More allocated RAM does not guarantee more speed.
If your Mac has spare memory, you can try a larger target. Stop any running
server, then preview this example without loading the model:
slotstream doctor --memory-gb 40
If the plan fits the available memory and Metal working-set limit with
headroom, start the server with the same target:
slotstream serve --memory-gb 40
Replace 40 with your chosen total-process budget in decimal GB. An explicit
target keeps the cache fixed and disables automatic resizing, so leave room
for macOS and other apps and watch memory pressure. See the
memory options for details.
Will this wear out my SSD?
Generation reads the model files without rewriting them. macOS swap adds
writes when memory runs short. Automatic memory sizing helps, but a small
Mac or an oversized manual setting can still swap heavily.
Can I run it on Linux or Windows?
Support for AMD and NVIDIA on Windows and Linux is planned for Sevra.
It isn't available in the current Slotstream engine.
Can I use a different model?
Not with Slotstream today. Its loader and memory planner are built for this
model. See related projects for runtimes
with different model and hardware support.
Why this exists
I wanted to run this model on my own Mac, but the standard loader exhausted
memory before producing a reply. Slotstream grew out of that experiment.
The published measurements
include that failed load and the experiments that followed.
The project was also discussed on Hacker News.
The questions and hardware reports from that discussion help guide the work.
Support
Report a bug if something
doesn't work, or share your Mac's results
to help others know what to expect. Reports are credited to their authors.
Code and documentation contributions are welcome; see
Contributing for the workflow.
Who made this
I'm Carlos Galarza. I work on efficient AI
and Executable Rationality, making machine cognition explicit and runnable.
I also help teams run open models on their own hardware and debug agent
workflows. For help or consulting, email me.
Star history
The badge at the top shows the latest star count; this chart is updated weekly.
Slotstream 让你在只有 16 到 64 GB 内存的 Mac 上运行一个 105 GB 的 AI 模型（Qwen3.8-Flash-Next，一个 125B 参数的专家混合模型），方法是从 SSD 流式读取模型的大部分内容，并把最繁忙的专家缓存在内存中。