The fastest local AI engine for Apple Silicon.
Drop-in OpenAI / Anthropic API · up to 3× Ollama's throughput (measured) · Runs on any M-series Mac.
rapidmlx.com ·
Docs ·
Model mirror ·
Desktop app ·
Discord
Works with your AI stack
Use Rapid-MLX as a local backend for agents, apps, or your own code. If a
client accepts an OpenAI- or Anthropic-compatible endpoint, it can usually use
Rapid-MLX without an adapter.
Five Tier-1 agents are exercised end-to-end on real weights before release.
See the tested compatibility matrix
for exact API coverage and setup status.
Install
Desktop — macOS (Apple Silicon)
The easiest way to chat locally, manage models, and use vision, files, voice,
and image generation from one app.
CLI and server — macOS (Apple Silicon)
# Homebrew — prebuilt bottle from homebrew-core
brew install rapid-mlx
# Or the guided installer — detects RAM and recommends a starter model
curl -fsSL https://rapidmlx.com/install.sh | bash
Both install the same rapid-mlx CLI. Prefer uv or pip, or want to verify
the installer before running it? See alternative install methods
and install security.
The guided installer prefers a runnable model already cached on this Mac when
it fits the RAM tier. Otherwise its quick first-chat download is
lfm2.5-1b-4bit below 16 GB and qwen3.5-4b-4bit at 16 GB or above; larger
quality picks remain available through rapid-mlx recipe and the model picker.
Quick Start (60 seconds)
1. Chat with a model right now:
rapid-mlx chat
Defaults to qwen3.5-4b-4bit. First run downloads the weights (~3 GB) with a progress bar and drops you into a REPL. Type /help for slash commands, /exit to quit.
2. Or serve it for use from other apps:
rapid-mlx serve qwen3.5-4b-4bit
Starts an OpenAI-compatible HTTP server bound to http://localhost:8000. Point any client that supports a local custom endpoint (Aider, LangChain, OpenCode, PydanticAI, your own scripts) at http://localhost:8000/v1; Claude Code / Anthropic SDK uses http://localhost:8000 (the Anthropic messages route lives at /v1/messages under the same host).
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
print(client.chat.completions.create(
model="default",
messages=[{"role": "user", "content": "Say hello"}],
).choices[0].message.content)
3. Or wire up your coding agent — one command:
rapid-mlx launch claude-code
With a server running (step 2), this patches Claude Code's local config (~/.claude/settings.json) to route at http://localhost:8000 — no manual env vars, no editing JSON by hand. You get a fully local Claude Code: $0 per token, nothing leaves your Mac. Swap in cline or continue-dev for the other IDE clients, or run rapid-mlx launch list to see what's detected on this machine.
Cursor: Cursor currently routes BYOK requests through its own servers, so its servers cannot reach a Rapid-MLX endpoint on localhost. Rapid-MLX therefore does not generate a Cursor localhost config. If you intentionally expose the server through a public HTTPS tunnel, set RAPID_MLX_API_KEY=your-secret for both rapid-mlx serve ... and rapid-mlx launch cursor --server-url https://your-public-host. This is no longer a fully local connection; never expose an unauthenticated server. Rapid-MLX rejects explicit local/private addresses but cannot verify reachability from Cursor's network, whose DNS view may differ from your Mac.
Vision / audio / video / diffusion models? Base install is text-only (~460 MB). Vision, audio (TTS, STT, voice cloning), video generation, embeddings, and DFlash speculative decoding ship as opt-in extras. → Optional extras
Not into the terminal? Rapid-MLX Desktop bundles the same engine inside a one-click Mac app.
Image generation
Generate images locally from the Desktop Images tab or the
OpenAI-compatible Images API. Install the image runtime when using the CLI:
pip install 'rapid-mlx[image]'
rapid-mlx serve flux2-klein-4b
curl http://localhost:8000/v1/images/generations \
-H 'Content-Type: application/json' \
-d '{"model":"flux2-klein-4b","prompt":"A red sailboat on an alpine lake","size":"1024x1024","seed":42}'
Edit an existing PNG or JPEG with the edit-specialized model:
In one terminal:
rapid-mlx serve qwen-image-edit
In another terminal:
curl http://localhost:8000/v1/images/edits \
-F [email protected] \
-F model=qwen-image-edit \
-F 'prompt=Change the sky to a warm sunset' \
-F response_format=b64_json
flux2-klein-4b is the recommended starting point. Download sizes are the
currently cataloged model payloads; minimum unified memory includes practical
headroom for macOS and the serving process. Omit steps to use the validated
family default shown below.
| Alias |
Best fit |
Modes |
Download |
Minimum unified memory |
Default steps |
flux2-klein-4b |
Recommended; fast everyday images |
Generate + edit |
4.3 GiB |
12 GB |
4 |
bonsai-image-4b-2bit |
Smallest download |
Generate |
3.6 GiB |
12 GB |
4 |
z-image-turbo |
Photorealistic images |
Generate |
5.5 GiB |
16 GB |
8 |
flux-schnell |
FLUX.1 compatibility |
Generate |
9.0 GiB |
16 GB |
4 |
sdxl-base |
SDXL compatibility |
Generate |
6.5 GiB |
16 GB |
30 |
flux2-klein-4b-bf16 |
Explicit full-precision Klein path |
Generate + edit |
14.9 GiB |
32 GB |
4 |
hidream-o1-dev |
Complex compositions |
Generate |
16.4 GiB |
32 GB |
28 |
sd35-large-4bit |
Stable Diffusion 3.5 |
Generate |
15.3 GiB |
32 GB |
28 |
qwen-image |
Text inside images |
Generate |
28.9 GiB |
64 GB |
20 |
qwen-image-edit |
Precise instruction edits and text changes |
Edit |
34.9 GiB |
96 GB |
20 |
At 1024×1024 with the four-step Klein default, measured warm generation was
about 9.2 seconds per image on an M3 Ultra. On a 32 GB M2 Pro, the q4 path
measured a 52.7-second median; explicitly selecting
flux2-klein-4b-bf16 reduced that median to 41.8 seconds (1.26× throughput).
The eight-step z-image-turbo baseline was about 34 seconds on the M3 Ultra
and 150 seconds on the M2 Pro.
The q4 alias remains the predictable default; on the measured 32 GB M2 Pro,
use the larger 14.9 GiB BF16 alias when latency matters. Exact results depend
on resolution, power/thermal state, and other workloads. The server completion
log reports total time and, when the backend exposes exact loop boundaries,
measured denoise s/step. For the qualified 1024-square Klein shape it also
reports estimated achieved TFLOPS; unavailable or unqualified values are never
fabricated.
→ Reproducible precision benchmark and methodology
Image work is single-flight: one generation runs at a time so two diffusion
pipelines cannot exhaust unified memory. Model weights retain their own
licenses and use restrictions; review the upstream model card before commercial
deployment.
→ Release dogfood coverage and reproducible acceptance contract
Video generation
Run text-to-video or image-to-video locally through the OpenAI-compatible
Videos API. Three backends ship — Wan 2.1 / 2.2, CogVideoX-Fun and
LTX-2.3 — across 8 registered checkpoints. wan2.2-ti2v-5b-q8 is the
recommended starting point: smallest of the Wan set, and TI2V means one
checkpoint does both text-to-video and image-to-video.
Requires Python 3.11+ (the video runtime does not support 3.10; core text and
audio still do) and ffmpeg for the final MP4 mux.
pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve wan2.2-ti2v-5b-q8
Create and download a clip:
curl http://localhost:8000/v1/videos \
-F model=wan2.2-ti2v-5b-q8 \
-F 'prompt=A fox running through fresh snow, cinematic tracking shot' \
-F seconds=1 \
-F size=832x512
# Poll until GET /v1/videos/VIDEO_ID reports "status": "completed", then:
curl http://localhost:8000/v1/videos/VIDEO_ID/content -o output.mp4
The create call returns a job immediately. Poll GET /v1/videos/VIDEO_ID
until status is completed. Add -F [email protected] for
image-to-video.
Generation is serialized — one clip at a time — because two diffusion
pipelines resident at once will exhaust unified memory. Expect minutes of
compute per second of footage, not real time.
→ Every checkpoint, RAM requirement and tuning knob
Audio: speech, transcription, voice cloning
44 audio aliases behind the OpenAI-compatible /v1/audio/* endpoints — any
OpenAI SDK works unchanged.
pip install 'rapid-mlx[audio]'
# Text to speech
rapid-mlx serve kokoro
curl http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"kokoro","input":"hello from rapid-mlx"}' --output hello.wav
# Transcription (Whisper / Parakeet / SenseVoice)
rapid-mlx serve whisper-large-v3-turbo
curl http://localhost:8000/v1/audio/transcriptions \
-F [email protected] -F model=whisper-large-v3-turbo
Beyond the basics, three things you may not expect to run locally:
- Zero-shot voice cloning from a reference clip.
indextts is the only
one that takes the clip alone; qwen3-tts-clone, f5-tts-zh and
chatterbox all require ref_text (the clip's exact transcript) paired
with ref_audio, and the request is rejected before generation if it is
missing.
- Voice design —
qwen3-tts-voicedesign has no named speakers at all.
Describe the voice you want in natural language via instructions
(timbre, gender, age, accent, emotion, prosody) and it synthesises it.
- Forced alignment —
qwen3-aligner takes audio plus the transcript you
already have and returns per-character timings. It never guesses at the
words, so it cannot mis-hear them; that is what karaoke captions and
beat-synced editing need.
Also: word-level timestamps on transcription, and local text-to-music at
/v1/audio/music.
→ All 44 aliases across 13 families
Why Rapid-MLX
|
|
| Apple-Silicon-native |
Pure MLX kernels — no llama.cpp fallback, no Metal shim. Continuous batching, prompt cache (radix + DeltaNet RNN snapshots), and a quantized live KV cache (int4/int8 on the continuous-batching cache + TurboQuant K8V4 codec) run at native MLX bandwidth on M1 → M4. |
| Drop-in OpenAI / Anthropic API |
/v1/chat/completions, /v1/responses (Codex CLI), /v1/messages (Anthropic SDK / Claude Code), /v1/embeddings, /v1/audio/*, /v1/videos — same wire as ChatGPT / Claude, no client adapter. |
| First-class ecosystem coverage |
12 agent CLIs and 3 Python frameworks are wire-verified against real weights every release (5 are Tier-1, re-verified on current binaries) — Codex CLI, Claude Code, OpenCode, Qwen Code, OpenHands, Hermes Agent, Aider, Kilo Code, DeepSeek Harness, GitHub Copilot, Factory Droid, Moonshot Kimi Code + LangChain, PydanticAI, smolagents. |
→ Full feature breakdown
Use Cases
|
|
|
| Chat in the terminal |
rapid-mlx chat qwen3.5-9b-4bit |
Streaming REPL, /help for slash commands, --think / --no-think to control CoT. |
| OpenAI server for your apps |
rapid-mlx serve qwen3.5-9b-4bit |
Point Aider, LibreChat, Open WebUI, or LangChain at http://localhost:8000/v1. |
| Agent backends |
rapid-mlx serve qwen3.6-35b-8bit &
rapid-mlx agents codex --setup && codex |
10 agents auto-configure via agents <name> --setup once the server is up (12 wire-verified total, 5 Tier-1) — see Agent support. |
| Benchmark your Mac |
rapid-mlx benchmark run qwen3.5-9b-4bit |
Reproducible model-first benchmark; saves privately, with a separate consent-gated benchmark share RUN_ID. |
→ One-shot IDE setup with rapid-mlx launch <claude-code|cline|continue-dev>
Agent Support
All 12 agents below are wire-verified against real weights every release via their own integration-test cell. Of these, five are Tier-1 — Claude Code, Codex CLI, Hermes, Aider, and DeepSeek Harness — re-verified end-to-end against the current client binary every release, with one guardian per API wire (Anthropic /v1/messages, OpenAI /v1/responses, and /v1/chat/completions covered for tool-calling depth, reach, and DeepSeek's own harness protocol). The other seven are Tier-2: wire-verified in the matrix and configured on-demand. The first nine agents each ship a rapid-mlx agents <name> --setup config template (except Claude Code, which is one env-var), and Continue.dev gets the same one-command setup via rapid-mlx agents continue --setup (ten setup-capable clients in all, though Continue.dev is not part of the wire-verified matrix below); GitHub Copilot, Factory Droid, and Moonshot Kimi Code plug in through their own documented BYOK config (auth-gated, so the matrix cell is a wire smoke).
Tier-1 is not a label — it is a job that blocks the release. tests/integrations/agent_smoke.sh drives each of the five through the same real multi-step bug-fix task against a local 35B model and asserts the repo's own test suite goes green afterwards; if any one of them fails, the version cannot tag or publish.
Tier-1 (5): Claude Code · Codex CLI · Hermes · Aider — last re-verified end-to-end 2026-07-28 on current binaries (claude 2.1.211, codex 0.145.0, hermes 0.9.0, aider 0.86.2). DeepSeek Harness — promoted 2026-08-17, verified on dsh 0.1.0-rc.7 against qwen3.6-35b-8bit.
Tier-2 (7): OpenCode · Qwen Code · OpenHands · Kilo Code · GitHub Copilot · Factory Droid · Moonshot Kimi Code.
Rapid-MLX 是一个面向 Apple Silicon Mac 的本地 LLM 推理服务器，基于 MLX 构建，提供 OpenAI 和 Anthropic 兼容的 API，专注于为 Claude Code、Codex CLI 等编程代理提供可靠的工具调用。