Topics / #llama-cpp

Llama-cpp projects on GitHub

Runtime for quantized local inference of large language models on CPUs and laptops.

As of , the most-starred Llama-cpp projects on GitHub are gpt4all (77.4K stars), minimind (63.9K stars) and textgen (47.7K stars). GitHub.Locker tracks 37 Llama-cpp repositories with 543.4K stars combined. The fastest-growing on October 16 was Strata, with +718 stars in a day.

37Projects
543.4KStars in total
+1.7KStars in a day
Browse in the catalog →github.com/topics/llama-cpp ↗

Hot right now

Most stars gained in a day · Oct 16

+718

Strata

Niko1221/Strata · ★ 26.4K

Strata runs Qwen3.8-Flash-Next, a 125-billion-parameter AI model, on a normal gaming PC with one NVIDIA GPU (12-24GB VRAM) and 64GB of RAM, with a one-click installer for Windows and Linux.

+215

DigUp

ARahim3/DigUp · ★ 4.3K

DigUp is a free, open-source macOS menu bar app that lets you search the files on your Mac by what's actually in them — photos, PDFs, documents, recordings, videos and code — using Google DeepMind's EmbeddingGemma 2 model running entirely on-device.

+179

PS5LM

cobanov/PS5LM · ★ 337

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

+150

StartLux-Decision

StartLuxLabs/StartLux-Decision · ★ 552

StartLux-Decision is a family of typed decision models (five dense sizes from 0.8B to 27B plus a 35B-A3B mixture-of-experts) that answer a question with a probability for each option instead of generating text.

The all-time top

37 projects · by stars

  1. #1 · nomic-ai/gpt4all

    gpt4all

    GPT4All is a desktop application that runs large language models privately on everyday laptops and desktops, with no API calls or GPU required.

    77.4K
  2. #2 · jingyaogong/minimind

    minimind

    MiniMind is an open-source project that lets you train a tiny large language model (around 64M parameters) completely from scratch, in about 2 hours on a single RTX 3090 for roughly the cost of 3 RMB in GPU rental.

    63.9K
  3. #3 · oobabooga/textgen

    textgen

    TextGen is an open-source desktop app for running local LLMs with text, vision, tool-calling, and an OpenAI/Anthropic-compatible API, built to be 100% private with no telemetry.

    47.7K
  1. 4

    VoxCPM

    39.4KOpenBMB/VoxCPM · Python

    VoxCPM is a tokenizer-free Text-to-Speech system that directly generates continuous speech representations through an end-to-end diffusion autoregressive architecture, skipping discrete tokenization for more natural, expressive speech. VoxCPM2 is the latest 2B-parameter release trained on 2M+ hours of multilingual audio.

  2. 5

    llmfit

    38KAlexsJones/llmfit · Rust

    llmfit inspects your CPU, RAM, GPU(s), VRAM, and accelerator setup, then tells you which open-source LLMs and quantizations will actually run well on your machine.

  3. 6

    qmd

    30.5Ktobi/qmd · TypeScript

    QMD is an on-device search engine for your personal knowledge: it indexes markdown notes, meeting transcripts, documentation and knowledge bases, and searches them with keywords or natural language.

  4. 7

    MiniCPM-V

    26.8KOpenBMB/MiniCPM-V · Python

    A family of pocket-sized multimodal AI models from OpenBMB (MiniCPM-V and MiniCPM-o) built to run on local devices, even phones — understanding images, video and text, and in the o-series also speech.

  5. 8

    Strata

    26.4KNiko1221/Strata · C++

    Strata runs Qwen3.8-Flash-Next, a 125-billion-parameter AI model, on a normal gaming PC with one NVIDIA GPU (12-24GB VRAM) and 64GB of RAM, with a one-click installer for Windows and Linux.

  6. 9

    llamafile

    26.4Kmozilla-ai/llamafile · C++

    llamafile turns open LLMs into single-file executables: one file that runs a chat model locally on most operating systems and CPU architectures, with no installation. It is a Mozilla Builders project, now revamped by Mozilla.ai.

  7. 10

    guidance

    21.8Kguidance-ai/guidance · Jupyter Notebook

    Guidance is a Python library — an efficient programming paradigm for steering large language models: you control how output is structured while reducing latency and cost compared with conventional prompting or fine-tuning.

  8. 11

    FunASR

    20.7Kmodelscope/FunASR · Python

    FunASR is an open-source industrial speech recognition toolkit covering ASR, VAD, punctuation, speaker diarization, and emotion/audio-event models, deployable offline, in streaming mode, on edge devices, or behind an OpenAI-compatible API.

  9. 12

    DB-GPT

    20.2Keosphoros-ai/DB-GPT · Python

    An open-source agentic AI data assistant: connect databases, files or knowledge bases, ask questions in plain language, and the AI writes SQL and code, runs the analysis, and produces charts, dashboards and reports.

  10. 13

    Llama-Chinese

    14.8KLlamaChinese/Llama-Chinese · Python

    Llama-Chinese is a Chinese-language community project built around Meta's Llama models (Llama2, Llama3, Llama4). It provides Chinese pretraining and fine-tuning, a custom Atom model series optimized for Chinese, and resources for training, deploying and using Llama in Chinese for free commercial use.

  11. 14

    FluidVoice

    12.2Kaltic-dev/FluidVoice · Swift

    FluidVoice is an open-source voice-to-text dictation app for macOS that transcribes speech on-device and can enhance the text with AI, acting as a local alternative to Wispr Flow.

  12. 15

    G0DM0D3

    11.8Kelder-plinius/G0DM0D3 · TypeScript

    G0DM0D3 is an open-source, privacy-transparent multi-model chat interface built to push past standard AI guardrails for red teaming, cognition research, and jailbreak experimentation.

  13. 16

    llama-gpt

    10.9Kgetumbrel/llama-gpt · TypeScript

    LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot powered by Llama 2 (and now Code Llama), so you get a familiar chat UI without sending any data to a third-party cloud.

  14. 17

    Operit

    8.7KAAswordman/Operit · Kotlin

    Operit is an open-source AI agent platform for Android that connects cloud or local AI models to your phone's system, terminal, browser, files, and project workspaces so the AI can actually perform tasks like file management, coding, and device automation.

  15. 18

    stable-diffusion.cpp

    7.7Kleejet/stable-diffusion.cpp · C++

    A pure C/C++ inference engine for diffusion models like Stable Diffusion, Flux, Wan, Qwen Image, and Z-Image, built the same way as llama.cpp so you can generate images and videos without heavy Python dependencies.

  16. 19

    ODS

    7.6KOsmantic/ODS · Python

    ODS (Osmantic Deployment System) turns your PC, Mac, or Linux box into a private AI server by installing and wiring together local LLM inference, a chat UI, agents, voice, RAG, image generation, and privacy tools in one go.

  17. 20

    openplayground

    6.3Knat/openplayground · TypeScript

    openplayground is a self-hosted LLM playground you run on your laptop: a web UI for trying out, tuning, and comparing models from many different providers.

  18. 21

    SemIf-OpenJev

    4.8KTheoLeeCJ/SemIf-OpenJev · Python

    SemIf (formerly OpenJev) is an independent research project that runs runtime-defined semantic decisions with open models on one consumer GPU, mirroring the interface of TypeSafe's closed Jev service but not its undisclosed model.

  19. 22

    DigUp

    4.3KARahim3/DigUp · Swift

    DigUp is a free, open-source macOS menu bar app that lets you search the files on your Mac by what's actually in them — photos, PDFs, documents, recordings, videos and code — using Google DeepMind's EmbeddingGemma 2 model running entirely on-device.

  20. 23

    atomic-agent

    3.4KAtomicBot-ai/atomic-agent · TypeScript

    Atomic Agent is a local-first AI agent that runs the full control loop and all its state on your own machine, using open-weight models through llama.cpp or cloud models. It can browse the web, edit files, run approved shell commands, inspect documents, and remember context across sessions.

  21. 24

    mlx-serve

    2Kddalcu/mlx-serve · Zig

    mlx-serve is a native local inference server for Apple Silicon Macs that runs MLX and GGUF language models and exposes OpenAI-, Anthropic-, and Ollama-compatible APIs, plus generates images, video, music, speech, and 3D models, all without Python or a cloud connection.

  22. 25

    nobodywho

    1.6Knobodywho-ooo/nobodywho · Rust

    NobodyWho is an open-source inference engine that runs LLMs locally and offline on phones, desktops and even Apple Watch — no API keys, no cloud calls. Under the hood it is powered by llama.cpp and accelerates through the GPU via Vulkan or Metal.

  23. 26

    jev

    1.3Kfeder-cr/jev · C++

    jevos-1b is a free, open-source yes/no decision engine that runs locally on your laptop's CPU, returning the probability that the answer is 'yes' for a given text plus question in about 50-220 ms, as a self-hosted alternative to the cloud-based Jev service.

  24. 27

    decider

    1.2KMapika/decider · Python

    Decider is a family of language models fine-tuned from Qwen3.5 that don't generate text; instead they read a state plus a set of typed questions and return calibrated probability distributions for every question in one forward pass.

  25. 28

    backburner

    1KStayLameBro/backburner · Python

    Backburner lets a nearby iPhone act as a coprocessor for your Mac so a locally-run 27B LLM (Qwen3.8-27B) reads prompts faster and holds far more context. The phone connects over a 10 Gb/s USB-C cable and offloads part of the model's layers plus storage/attention for old context.

  26. 29

    mac-performance-monitor

    937Zesty0wl/mac-performance-monitor · Swift

    A native macOS performance monitor and recorder that tracks CPU, memory, GPU, network, disk, battery, and sensor readings, letting you compare them against the processes behind them. It works as a menu bar readout, a quiet background recorder, or a window you open when needed.

  27. 30

    rizzo-flow

    903Rizzo-AI-Academy/rizzo-flow · Python

    Rizzo Flow is an open-source, local-first implementation of TypeSafe's Jev 'System One' model: it takes unstructured state as input and returns typed, probabilistic decisions (a yes/no, a choice, a score, a number) without generating a single text token, running entirely on your own hardware via llama.cpp.

  28. 31

    souchastnik

    725MShverdiakov/souchastnik · Kotlin

    An Android app (Souchastnik) that tells you what criminal charge you might face in Russia for something you wrote online, and flags mentions of 'foreign agents' or banned social networks.

  29. 32

    StartLux-Decision

    552StartLuxLabs/StartLux-Decision · Python

    StartLux-Decision is a family of typed decision models (five dense sizes from 0.8B to 27B plus a 35B-A3B mixture-of-experts) that answer a question with a probability for each option instead of generating text.

  30. 33

    qwen38-uncensored

    365Wassimyounes01/qwen38-uncensored · JavaScript

    A Node.js install harness that sets up Qwen 3.8 27B as a fully uncensored local model, using weight-level abliterated GGUF weights plus a reinforcement SYSTEM prompt, runnable through Ollama or Apple's MLX.

  31. 34

    qwen38-mtp

    363sudoingX/qwen38-mtp · Python

    This repo documents a single llama.cpp server flag that unlocks multi-token-prediction (MTP) speculative decoding for Qwen3.8-27B, giving +33% to +145% faster decode on consumer GPUs. It bundles the exact launch command, a crowdsourced benchmark table across dozens of GPUs, and a probe.py script for measuring the gain yourself.

  32. 35

    PS5LM

    337cobanov/PS5LM · C++

    llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

  33. 36

    CUDA-for-AMD-Windows

    321Speedstu/CUDA-for-AMD-Windows · Python

    A Windows compatibility stack that runs CUDA-targeted applications on AMD GPUs via ZLUDA plus AMD ROCm/HIP, with tooling to verify the work truly executes on the GPU instead of silently falling back to CPU or producing wrong results.

  34. 37

    gemma4.c

    173qmog-ai/gemma4.c · C

    gemma4.c is a CPU inference runtime for Google's Gemma 4 E2B model written in about 700 lines of pure C, built to show how LLM inference actually works under the hood.

Often tagged together