What is this?
Magnitude is an open source inference server that profiles your machine, recommends local models that fit it, then downloads, tunes, and runs them for your coding agent.
Magnitude is an open source inference server that profiles your machine, recommends local models that fit it, then downloads, tunes, and runs them for your coding agent.Is this a library or a standalone server I run?It is a standalone inference server/CLI that runs in the background on your machine; you connect it to an existing coding agent (harness) rather than embed it as a library.
It is a standalone inference server/CLI that runs in the background on your machine; you connect it to an existing coding agent (harness) rather than embed it as a library.How do I get it running?Run the setup command so it profiles your hardware, ranks models, and connects your harness to the one you pick; you can also just tell your agent (Pi, Claude Code, OpenCode, etc.) to handle setup for you.
Run the setup command so it profiles your hardware, ranks models, and connects your harness to the one you pick; you can also just tell your agent (Pi, Claude Code, OpenCode, etc.) to handle setup for you.Which coding agents (harnesses) does it plug into?It works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline, or you can use Magnitude's built-in harness.
It works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline, or you can use Magnitude's built-in harness.Is there a minimum hardware requirement?There's no fixed minimum spec; Magnitude profiles your machine and recommends whatever models run well on it, with more memory enabling larger models.
There's no fixed minimum spec; Magnitude profiles your machine and recommends whatever models run well on it, with more memory enabling larger models.Does it run on Windows, or just macOS and Linux?It supports macOS, Linux, and Windows (via WSL), using Metal, CUDA, or Vulkan on the GPU side, or CPU only.
It supports macOS, Linux, and Windows (via WSL), using Metal, CUDA, or Vulkan on the GPU side, or CPU only.Does any of my data leave my machine?No — prompts, files, and models stay local, and once a model is downloaded no internet connection is needed.
No — prompts, files, and models stay local, and once a model is downloaded no internet connection is needed.How is Magnitude faster than llama.cpp?It compiles and tunes its own kernels specifically for your device, including speculative decoding, so open models run up to 2x faster than llama.cpp on the same hardware.
It compiles and tunes its own kernels specifically for your device, including speculative decoding, so open models run up to 2x faster than llama.cpp on the same hardware.Does it cost anything to run?No, it's free to run with no token costs, API keys, or rate limits since everything runs on your own hardware.
No, it's free to run with no token costs, API keys, or rate limits since everything runs on your own hardware.What happens to models when I'm not using them?Models are loaded on demand when your agent needs them and automatically unloaded when idle or when memory gets tight.
Models are loaded on demand when your agent needs them and automatically unloaded when idle or when memory gets tight.What are similar projects?TensorFold, ds4, petals, deltafin, Bud-Decision-Studio, ODS, project-maya, text-generation-inference
TensorFold, ds4, petals, deltafin, Bud-Decision-Studio, ODS, project-maya, text-generation-inferenceWhich topics is it in?model-router, on-device, model-serving, amd, local-ai, local-llm, apple-silicon, offline, inference, nvidia, cuda, coding-agent
model-router, on-device, model-serving, amd, local-ai, local-llm, apple-silicon, offline, inference, nvidia, cuda, coding-agent