magnitude README

The project's README from github.com/magnitudedev/magnitude. As one markdown file: README.md.

Magnitude icon

Magnitude

The best way to run local models on your machine

Documentation Discord Follow Magnitude on Twitter GitHub Repo stars npm version

Magnitude is an open source inference server for the hardware you already have. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline, or use the built-in harness.

⭐ Help us reach more developers and grow the Magnitude community. Star this repo!

Magnitude demo: run local models on your machine

Get started

See what your Mac can run:

npm i -g @magnitudedev/cli
magnitude setup

Setup profiles your hardware, ranks models by speed, accuracy, intelligence, and memory, and connects your harness to the one you pick.

Or let your agent handle it. Send this to Pi, Claude Code, OpenCode, or whatever you use:

Set up local models for me with the Magnitude CLI. Install it with `npm i -g @magnitudedev/cli` (or my package manager), then run `magnitude docs onboarding` and follow the instructions.

Your agent will profile your hardware, walk you through the best local models for it, download the ones you pick, and switch itself over to them.

Why Magnitude?

FAQ

What is Magnitude?

An open source inference server for the hardware you already have. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into the agent you already use.

How does it know what my machine can run?

Magnitude profiles your chip, memory, and bandwidth, then estimates fit and tok/s for every model in the catalog. It ranks them by speed, accuracy, intelligence, and memory so you can pick.

What hardware do I need?

There's no fixed minimum. Magnitude profiles your machine and recommends what runs well on it. More memory lets you run larger models.

What platforms does it support?

macOS, Linux, and Windows through WSL. Metal, CUDA, and Vulkan on the GPU side, or CPU only. Magnitude profiles whatever you have and recommends models that fit.

Which harnesses work with it?

Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick. Or use Magnitude's built-in harness.

Do I need to manage it after setup?

No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the CLI anytime.

Is it private?

Yes. Prompts, files, and models stay on your machine. Once a model is downloaded, no internet connection is needed.

Learn more

License

Magnitude is licensed under the Apache License 2.0.

magnitude (magnitudedev/magnitude)

Searching Database…

这是什么?Magnitude 是一个开源推理服务器,它会检测你的机器,推荐适合该机器的本地模型,然后为你的编码代理下载、调优并运行这些模型。
这是一个库,还是需要我自己运行的独立服务器?它是一个独立的推理服务器和命令行工具,在你的机器上后台运行。你需要把它连接到现有的编码代理(harness),而不是把它当作库嵌入使用。
我该怎么把它跑起来?运行安装配置命令,它会检测你的硬件、为模型排序,并把你的编码代理(harness)连接到你选择的模型。你也可以直接告诉你的代理(Pi、Claude Code、OpenCode 等)替你完成设置。
它能接入哪些编码代理(harness)?它支持 Pi、OpenCode、Hermes、OpenClaw、Codex、Claude Code、Oh My Pi 和 Cline,你也可以使用 Magnitude 内置的 harness。
对硬件有最低要求吗?没有固定的最低配置。Magnitude 会检测你的机器,并推荐在其上运行良好的模型,内存越大,能运行的模型就越大。
它能在 Windows 上运行吗,还是只支持 macOS 和 Linux?它支持 macOS、Linux 和 Windows(通过 WSL)。GPU 方面可使用 Metal、CUDA 或 Vulkan,也可以只用 CPU。
我的数据会离开我的机器吗?不会。提示词、文件和模型都保留在本地,模型下载完成后不需要联网。
Magnitude 为什么比 llama.cpp 快?它会针对你的设备专门编译和调优自己的内核,并包含推测解码,因此在相同硬件上,开源模型最多可比 llama.cpp 快 2 倍。
运行它需要花钱吗?不需要。运行是免费的,没有 token 费用、API 密钥或速率限制,因为一切都在你自己的硬件上运行。
我不用的时候,模型会怎么样?当你的代理需要时,模型会按需加载;在空闲或内存紧张时,模型会自动卸载。
Size
138.3 MB
Stars
7.1K
Age
4mo
Commits · 90 Days