Topics / #on-device

On-device projects on GitHub

Running speech and ML models directly on phones and local hardware.

As of , the most-starred On-device projects on GitHub are whisper.cpp (54.5K stars), colibri (42.8K stars) and BitNet (41.2K stars). GitHub.Locker tracks 44 On-device repositories with 460.5K stars combined. The fastest-growing on October 18 was FractalWorkspace, with +983 stars in a day.

44Projects
460.5KStars in total
+2.9KStars in a day
Browse in the catalog →github.com/topics/on-device ↗

Hot right now

Most stars gained in a day · Oct 18

+983

FractalWorkspace

AhmadHassan-BTed/FractalWorkspace · ★ 1.2K

Fractal is a distributed computing ecosystem for federated learning that harvests spare compute from mobile edge devices while keeping raw user data on-device. It pairs a Python-based central orchestrator with a native Android client that trains models locally and only sends back weight checkpoints.

+954

Parchment

AhmadHassan-BTed/Parchment · ★ 1.2K

Parchment is a native Android PDF reader built with Kotlin and Jetpack Compose, designed around a warm, paper-like aesthetic and a gesture-driven reading experience rather than a clinical, industrial UI.

+214

DigUp

ARahim3/DigUp · ★ 4.8K

DigUp is a free, open-source macOS menu bar app that lets you search the files on your Mac by what's actually in them — photos, PDFs, documents, recordings, videos and code — using Google DeepMind's EmbeddingGemma 2 model running entirely on-device.

+151

colibri

JustVugg/colibri · ★ 42.8K

Colibri is an open-source engine for running frontier mixture-of-experts (MoE) language models on hardware you already own — written in pure C with zero dependencies.

The all-time top

44 projects · by stars

  1. #1 · ggml-org/whisper.cpp

    whisper.cpp

    whisper.cpp is a high-performance C/C++ port of OpenAI's Whisper automatic speech recognition (ASR) model, letting you transcribe audio to text without Python or heavy ML frameworks.

    54.5K
  2. #2 · JustVugg/colibri

    colibri

    Colibri is an open-source engine for running frontier mixture-of-experts (MoE) language models on hardware you already own — written in pure C with zero dependencies.

    42.8K
  3. #3 · microsoft/BitNet

    BitNet

    bitnet.cpp is Microsoft's official inference framework for 1-bit (ternary, 1.58-bit) large language models like BitNet b1.58, letting them run fast on CPUs and GPUs.

    41.2K
  1. 4

    VoxCPM

    39.6KOpenBMB/VoxCPM · Python

    VoxCPM is a tokenizer-free Text-to-Speech system that directly generates continuous speech representations through an end-to-end diffusion autoregressive architecture, skipping discrete tokenization for more natural, expressive speech. VoxCPM2 is the latest 2B-parameter release trained on 2M+ hours of multilingual audio.

  2. 5

    meetily

    31.9KZackriya-Solutions/meetily · Rust

    A privacy-first AI meeting assistant that captures your meetings, transcribes them in real time, and generates summaries — entirely on your own machine, with no data sent to the cloud.

  3. 6

    MiniCPM-V

    26.9KOpenBMB/MiniCPM-V · Python

    A family of pocket-sized multimodal AI models from OpenBMB (MiniCPM-V and MiniCPM-o) built to run on local devices, even phones — understanding images, video and text, and in the o-series also speech.

  4. 7

    DeepSpeech

    26.8Kmozilla/DeepSpeech · C++

    Mozilla DeepSpeech is an open-source Speech-To-Text engine — a machine-learning model that turns audio into text on your own hardware.

  5. 8

    ml-stable-diffusion

    18Kapple-aiml-research/ml-stable-diffusion · Python

    This is Apple's toolkit for running Stable Diffusion image generation on Apple Silicon using Core ML, so it can run efficiently on Mac, iPad, and iPhone.

  6. 9

    KittenTTS

    16.2KKittenML/KittenTTS · Python

    Kitten TTS is an open-source, lightweight text-to-speech library built on ONNX that converts text into natural-sounding speech using tiny 15M-80M parameter models, running entirely on CPU.

  7. 10

    sherpa-onnx

    15.4Kk2-fsa/sherpa-onnx · C++

    sherpa-onnx is an offline toolkit for speech-to-text, text-to-speech, speaker diarization, speaker identification/verification, spoken language identification, audio tagging, VAD, speech enhancement, keyword spotting and source separation, built on next-gen Kaldi and ONNX Runtime, with no internet connection required.

  8. 11

    supertonic

    13.8Ksupertone-oss-archive/supertonic · Swift

    Supertonic is a lightning-fast, on-device text-to-speech system: it converts text into 44.1kHz 16-bit WAV speech locally via ONNX Runtime — no cloud, no API calls, no GPU. The open-weight model, Supertonic 3, covers 31 languages.

  9. 12

    MiniCPM

    11.6KOpenBMB/MiniCPM · Jupyter Notebook

    MiniCPM is OpenBMB's family of small, open-source LLMs built for on-device and resource-constrained use. The newest release, MiniCPM5-2B, is a dense 2B-parameter model that reaches open-source state-of-the-art for its size class while staying competitive with much larger 4B-class models.

  10. 13

    OpenJarvis

    11.6Kopen-jarvis/OpenJarvis · Python

    OpenJarvis is an open-source framework for personal AI that runs on your own devices: agents use local models by default and call the cloud only when truly necessary.

  11. 14

    moshi

    11.3Kkyutai-labs/moshi · Python

    Moshi is a speech-text foundation model and full-duplex spoken dialogue framework, meaning the model can listen and speak at the same time rather than taking turns like typical voice assistants.

  12. 15

    moonshine

    11.3Kmoonshine-ai/moonshine · C++

    Moonshine Voice is an open source AI toolkit for building real-time voice agents and interfaces, covering speech-to-text, intent recognition, and text-to-speech.

  13. 16

    RTranslator

    10.5Kniedev/RTranslator · Java

    RTranslator is an open-source Android app that translates conversations in real time. Speech recognition and translation run entirely on your phone — no cloud service behind it.

  14. 17

    pocket-tts

    9.9Kkyutai-labs/pocket-tts · Python

    Pocket TTS is a lightweight text-to-speech engine from Kyutai that runs efficiently on CPU only, using a tiny 100M-parameter model with streaming audio output and voice cloning.

  15. 18

    magnitude

    7.2Kmagnitudedev/magnitude · Rust

    Magnitude is an open source inference server that profiles your machine, recommends local models that fit it, then downloads, tunes, and runs them for your coding agent.

  16. 19

    laya-mlx

    7Kmizorewww/laya-mlx · Python

    Laya-MLX is a native MLX runtime that runs the open-weight Laya typed decision models locally on Apple Silicon, returning structured answers like a choice, a rubric score, or a probability in milliseconds instead of generating text.

  17. 20

    web-to-app

    6.8Kshiaho777/web-to-app · Kotlin

    WebToApp is an Android app that builds installable APKs from web projects entirely on your phone — no PC and no remote build server. Beyond wrapping URLs, it runs real server runtimes, ships an anti-censorship network stack, signs for Google Play and supports browser extensions.

  18. 21

    apfel

    6.5KArthur-Ficial/apfel · Swift

    Apfel is a CLI tool and local OpenAI-compatible server that exposes the built-in FoundationModels LLM shipped on Apple Silicon Macs, letting you prompt it from the shell, pipe text through it, or point OpenAI SDK clients at it — entirely on-device.

  19. 22

    neutts

    6.3Kneuphonic/neutts · Python

    NeuTTS is a collection of open-source, on-device text-to-speech language models that can clone a voice from just a few seconds of reference audio.

  20. 23

    openmed

    5.7Kmaziyarpanahi/openmed · Python

    OpenMed is a local-first healthcare AI SDK: clinical NER and HIPAA PII de-identification that runs on your own hardware, with 2,200+ medical models and Python, MLX, Android, and browser runtimes.

  21. 24

    OpenMinis

    5.3KOpenMinis/OpenMinis · Swift

    OpenMinis is a free, fully open-source AI agent app for iOS and Android that brings models like Claude, GPT and Gemini into a native mobile experience, giving them a sandboxed Linux shell, device integrations, browser automation and persistent memory on your phone.

  22. 25

    DigUp

    4.8KARahim3/DigUp · Swift

    DigUp is a free, open-source macOS menu bar app that lets you search the files on your Mac by what's actually in them — photos, PDFs, documents, recordings, videos and code — using Google DeepMind's EmbeddingGemma 2 model running entirely on-device.

  23. 26

    quill

    3.9Khumanitas-labs/quill · Swift

    quill is a minimal, fully local macOS meeting recorder and transcriber. One menu-bar click records your mic and all system audio; when you stop, it transcribes both tracks on-device into a speaker-tagged transcript.

  24. 27

    warp

    2.5Ksqliteai/warp · C

    WARP (Weight-Aware Runtime and Paging, formerly WASTE) is a dependency-free, embeddable C inference engine that runs huge mixture-of-experts models on consumer hardware by streaming each token's activated weights straight from NVMe — the full models, not distilled versions.

  25. 28

    TensorFold

    1.4Kashhart/TensorFold · Zig

    TensorFold is an LLM inference engine that serves open-weight language models through an OpenAI-compatible API, running them on Apple Silicon via MLX or on NVIDIA GPUs via CUDA.

  26. 29

    speech-swift

    1.2Ksoniqo/speech-swift · Swift

    Speech Swift is a collection of on-device AI speech models for Apple Silicon — speech recognition (ASR), text-to-speech, speech-to-speech, voice activity detection, and speaker diarization — built as Swift packages powered by MLX and CoreML.

  27. 30

    Parchment

    1.2KAhmadHassan-BTed/Parchment · Kotlin

    Parchment is a native Android PDF reader built with Kotlin and Jetpack Compose, designed around a warm, paper-like aesthetic and a gesture-driven reading experience rather than a clinical, industrial UI.

  28. 31

    FractalWorkspace

    1.2KAhmadHassan-BTed/FractalWorkspace · Python

    Fractal is a distributed computing ecosystem for federated learning that harvests spare compute from mobile edge devices while keeping raw user data on-device. It pairs a Python-based central orchestrator with a native Android client that trains models locally and only sends back weight checkpoints.

  29. 32

    MasterAgent

    1.1KOpenSparX/MasterAgent · C++

    MasterAgent (OAK) is an embeddable, local-first C++17 agent runtime: it turns input into validated tool execution, running deterministic skills before falling back to model inference. It targets on-device AI agents for use cases like automotive and embedded/Qualcomm NPU hardware, with zero cloud dependency by default.

  30. 33

    backburner

    1KStayLameBro/backburner · Python

    Backburner lets a nearby iPhone act as a coprocessor for your Mac so a locally-run 27B LLM (Qwen3.8-27B) reads prompts faster and holds far more context. The phone connects over a 10 Gb/s USB-C cable and offloads part of the model's layers plus storage/attention for old context.

  31. 34

    Spark-X2.5

    814XHToken/Spark-X2.5

    Spark-X2.5 is a series of two open-source language models (4B and 1.7B parameters) built for everyday conversation, writing, translation, coding, reasoning and agentic tool-use tasks, aiming for strong performance in a compact size.

  32. 35

    souchastnik

    726MShverdiakov/souchastnik · Kotlin

    An Android app (Souchastnik) that tells you what criminal charge you might face in Russia for something you wrote online, and flags mentions of 'foreign agents' or banned social networks.

  33. 36

    lipflow

    633amywork777/lipflow · Python

    Lipflow is 'Wispr Flow for your lips' — hold a key, silently mouth what you want to say with no sound and no microphone, and the text appears at your cursor in whatever app you're using. It reads your lips via webcam and runs entirely locally on your Mac.

  34. 37

    jev-chat-jarvis-mac

    499jev-chat/jev-chat-jarvis-mac · Python

    A floating overlay for WeChat on macOS: the moment a message arrives, it shows the sender's likely intent, a 0-9 risk score with a suggested action, and ready-to-use reply candidates.

  35. 38

    SCM

    488allenv0/SCM · TypeScript

    SCM (Screen Memories) is a macOS desktop app that runs deep AI search locally over every photo and every video frame in a chosen folder, letting you search by meaning, on-screen text, or spoken dialogue, all without the cloud.

  36. 39

    pi-voice

    302earendil-works/pi-voice · TypeScript

    A speech-to-text extension for the Pi coding agent: you hit a shortcut, speak, and the transcript is inserted at your editor cursor. Everything is transcribed locally on your machine.

  37. 40

    ESP32s3-LLM-Cluster

    243Low-Zi-Hong/ESP32s3-LLM-Cluster · C++

    This project is a distributed inference engine that runs a 0.5B-parameter LLM, quantized to 1.58-bit (BitNet) ternary weights, split across a cluster of 7 ESP32-S3 microcontrollers connected via SPI.

  38. 41

    murmur-youtube

    219per-simmons/murmur-youtube · Swift

    Murmur is a push-to-talk dictation app: hold a hotkey, talk, release, and the cleaned-up transcript is typed into whatever text field currently has focus. It runs natively and fully on-device, with separate builds for macOS (Swift) and Windows (C#/Avalonia).

  39. 42

    InstinctFlash

    200General-Instinct/InstinctFlash · Python

    InstinctFlash is a high-performance serving runtime for robotics foundation models - it loads checkpoints like GR00T, Cosmos3, pi0.5, LingBot-VA/VLA and DreamZero, applies acceleration (FP8, fused Triton kernels, caching) and serves them over Python or WebSocket, reporting up to 33.78x speedup on Jetson Thor.

  40. 43

    High-Angle_Robust_Fast_FaceAlignment

    197PINTO0309/High-Angle_Robust_Fast_FaceAlignment · Python

    HRFFA is a training, distillation and ONNX deployment pipeline for whole-head face alignment that stays robust at extreme head poses (looking up/down, full 360° roll, profile views) while being light enough for CPU inference.

  41. 44

    cardputer-ai

    119therezor/cardputer-ai · C++

    Cardputer AI is a tiny offline chatbot that runs entirely on the M5Stack Cardputer / Cardputer ADV handheld, powered by an 8-million-parameter language model baked directly into the firmware.

Often tagged together