On-device AI for any device.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently.
Features
- Run locally, offline — no API keys needed or hidden fees
- Run any chat LLM — Gemma, Qwen, Mistral and more
- Fast, type-safe tool calling — automatically generates structured grammars from your function signatures, no schema writing needed
- Multimodal input — provide image and audio information to your LLM
- Text-to-speech — synthesize local WAV audio with Kokoro, Pocket TTS and Supertonic backends
- Speech-to-text — transcribe audio into text with Whisper
- Voice Activity Detection — know when to stop listening and start transcribing with Silero
- Model downloading — load models directly from Hugging Face or any URL
Under the Hood
flowchart TD
FL["Flutter"]:::lang
PY["Python"]:::lang
GO["Godot"]:::lang
K["Kotlin"]:::lang
S["Swift"]:::lang
RN["React Native"]:::lang
FRB["flutter_rust_bridge"]:::glue
P3["PyO3"]:::glue
GX["gdext"]:::glue
U["UniFFI"]:::glue
FL --> FRB
PY --> P3
GO --> GX
K --> U
S --> U
RN --> U
CORE["NobodyWho · Rust<br/>chat · templates · grammars · sampling · context shifting"]:::core
FRB --> CORE
P3 --> CORE
GX --> CORE
U --> CORE
CORE --> LCPP["llama.cpp<br/>text · vision · embeddings · reranking"]:::engine
CORE --> ORT["ONNX Runtime<br/>speech-to-text · text-to-speech · VAD"]:::engine
LCPP --> HW["Hardware<br/>Vulkan · Metal · CUDA · GPU · CPU"]:::hw
ORT --> HW
classDef lang fill:#e8eefc,stroke:#5b7bd5,color:#11204a
classDef glue fill:#f3f0fb,stroke:#8b7bd5,color:#2a1f4a
classDef core fill:#eaf6ee,stroke:#4fa46a,color:#0f3b1f
classDef engine fill:#fdf0e3,stroke:#d58f3b,color:#4a2d0b
classDef hw fill:#f0eeea,stroke:#a09a90,color:#3a352e
- GPU-accelerated inference via Vulkan or Metal — runs fast on any OS
- Compatible with thousands of pre-trained LLMs — use any LLM in the GGUF format
- Powered by the wonderful llama.cpp
You can test our inference engine on iOS, Android, Vision Pro and Apple Watch.
Desktop means Linux, macOS and Windows throughout. Three gaps worth knowing before you start:
- Godot has no iOS export. Use the Flutter, React Native or Swift binding on iOS.
- Windows ARM64 is not supported yet.
- There is no web export. It is tracked in issue #111.
⭐ Useful to you? Star the repo, it's the easiest way to say thanks.
Requirements
Inference uses the GPU where available and CPU where not. The real constraint is memory.
Desktop
- Hardware — any 64-bit Linux, macOS or Windows machine. Windows is x86_64 only, there are no
ARM64 Windows builds. macOS accelerates through Metal out of the box, Linux and Windows need a
GPU driver with Vulkan support and fall back to CPU without one.
- Memory — roughly 1.5× the model file in free RAM, or 2× on a machine that's already busy.
8 GB is a comfortable floor for models up to ~2 GB; 16 GB or more above that.
- Discrete GPUs — NobodyWho offloads as many layers as fit in free VRAM and runs the rest on
the CPU, so a model larger than your VRAM still works, just slower. If barely any of it fits, it
skips the GPU and stays on CPU. It uses one GPU, the card with the most free memory.
Mobile
- iOS — iPhone 11 or newer, 4 GB RAM or more.
- Android — Snapdragon 855 / Adreno 640 / 6 GB RAM or better.
- Rule of thumb — the device needs roughly twice the model file size in available RAM. iOS
reserves around 2 GB, Android 2 to 4 GB depending on vendor. Models under 1 GB run smoothly on
any phone.
Models
Any model in the GGUF format works. Pass a hf:owner/repo:QUANT reference, an HTTPS URL, or a local path anywhere a model path is expected; remote models are downloaded and cached on first use. Pass "auto" to fit one to available memory.
Start with Qwen3 0.6B: at ~330 MB it is
small enough for any phone and good enough to tell whether the integration works.
Quick Start
Python
pip install nobodywho
from nobodywho import Chat
chat = Chat('hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M')
response = chat.ask('What is the capital of Denmark?')
print(response.completed()) # The capital of Denmark is Copenhagen.
Python documentation · PyPI
Kotlin
// Android
implementation("ai.nobodywho:nobodywho-android:<version>")
// Desktop JVM (Linux, macOS, Windows)
implementation("ai.nobodywho:nobodywho:<version>")
Use the version from the Maven Central badge above.
import ai.nobodywho.Chat
val chat = Chat.fromPath(
modelPath = "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M"
)
val response = chat.ask("What is the capital of Denmark?").completed()
println(response) // The capital of Denmark is Copenhagen.
Kotlin documentation
Swift
Add via Swift Package Manager:
https://github.com/nobodywho-ooo/nobodywho-swift.git
import NobodyWho
let chat = try await Chat.fromPath(
modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M"
)
let response = try await chat.ask("What is the capital of Denmark?").completed()
print(response) // The capital of Denmark is Copenhagen.
Swift documentation · GitHub · starter app
React Native / Expo
# React Native
npm install react-native-nobodywho
# Expo
npx expo install react-native-nobodywho
import { Chat } from "react-native-nobodywho";
const chat = await Chat.fromPath({
modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M",
});
const msg = await chat.ask("What is the capital of Denmark?").completed();
console.log(msg); // The capital of Denmark is Copenhagen.
RN / Expo documentation · NPM · RN starter app · Expo starter app
Flutter
flutter pub add nobodywho
import 'package:nobodywho/nobodywho.dart' as nobodywho;
void main() async {
await nobodywho.NobodyWho.init();
final chat = await nobodywho.Chat.fromPath(
modelPath: 'hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M',
);
final msg = await chat.ask('What is the capital of Denmark?').completed();
print(msg); // The capital of Denmark is Copenhagen.
}
Flutter documentation · pub.dev · starter app
Godot
Install NobodyWho from inside the editor:
- In Godot 4.5+, open the AssetLib tab and search for NobodyWho.
- Download and import it, making sure Ignore asset root is ticked in the import dialogue.
- Reload the project.
You can also grab a specific version from the releases page and import the zip the same way.
Godot documentation
Local Server
NobodyWho provides an experimental local server that implements the OpenAI Chat Completions API.
Start the server :
uvx --from 'git+https://github.com/nobodywho-ooo/nobodywho.git#subdirectory=nobodywho/server' nobodywho-server --model hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M --name qwen
It listens on http://127.0.0.1:8888 and serves /v1/models and /v1/chat/completions.
See the docs for more info.
Documentation
The documentation has everything you might want to know: https://docs.nobodywho.ooo/
Working with a coding agent? Point it at llms.txt or
llms-full.txt, or install the NobodyWho skill so your agent can look up the current APIs and documentation:
npx skills add https://github.com/nobodywho-ooo/nobodywho --skill nobodywho
⭐ Star the repo if NobodyWho is useful to you. It is how people find us.
License
NobodyWho is licensed under the EUPL-1.2. You may use it in proprietary and commercial
projects, free of charge. There has been some confusion about this, so to be precise:
Linking two programs or linking an existing software with your own work does not – at least under European law – produce a derivative or extend the coverage of the linked software licence to your own work. [1]
If you distribute modified versions of the code in this repo, you must open source those changes.
Feel free to make proprietary projects using NobodyWho, but don't make a proprietary fork of NobodyWho.
NobodyWho 是一个开源推理引擎，可以在手机、桌面电脑甚至 Apple Watch 上本地离线运行大语言模型——不需要 API 密钥，也不调用云端。它底层由 llama.cpp 驱动，并通过 Vulkan 或 Metal 利用 GPU 加速。