Topics / #cpu-inference

CPU inference projects on GitHub

Running model inference on CPUs with optimized kernels for edge deployment.

As of October 20, 2026, the most-starred CPU inference projects on GitHub are ktransformers (19.6K stars), KittenTTS (16.2K stars) and ChatGLM3 (13.6K stars). GitHub.Locker tracks 12 CPU inference repositories with 74.9K stars combined. The fastest-growing on October 19 was earshot, with +681 stars in a day.

12Projects

74.9KStars in total

+771Stars in a day

Browse in the catalog →github.com/topics/cpu-inference ↗

Hot right now

Most stars gained in a day · Oct 19

+681

earshot

Leonxlnx/earshot · ★ 1.5K

Earshot gives AI coding agents the ability to hear and reason about sound: it can describe what a file sounds like, judge how well audio matches a description, find the best moments in a recording, tell real recordings from code-synthesized ones, and critique a video's sound design against professional launch films.

+25

Open-Source-Face-Recognition-SDK

Faceplugin-ltd/Open-Source-Face-Recognition-SDK · ★ 2.7K

An open-source face recognition SDK from Faceplugin that does face detection, landmark extraction, embedding generation and face similarity comparison, all processed on-premise.

+23

KittenTTS

KittenML/KittenTTS · ★ 16.2K

Kitten TTS is an open-source, lightweight text-to-speech library built on ONNX that converts text into natural-sounding speech using tiny 15M-80M parameter models, running entirely on CPU.

+16

kimi-k3-in-c

FareedKhan-dev/kimi-k3-in-c · ★ 9.1K

An inference engine for the 2.78-trillion-parameter Kimi K3 model, written in portable C99. It runs on a plain CPU in as little as 8 GB of RAM — no GPU, no BLAS, no framework — streaming the 1.56 TB checkpoint straight off disk.

The all-time top

12 projects · by stars

#1 · kvcache-ai/ktransformers
ktransformers
KTransformers is a research framework for efficient inference and fine-tuning of large language models through CPU-GPU heterogeneous computing, letting huge MoE models run on hardware that doesn't have enough VRAM to hold them entirely on GPU.
19.6K
#2 · KittenML/KittenTTS
KittenTTS
Kitten TTS is an open-source, lightweight text-to-speech library built on ONNX that converts text into natural-sounding speech using tiny 15M-80M parameter models, running entirely on CPU.
16.2K
#3 · zai-org/ChatGLM3
ChatGLM3
ChatGLM3 is a series of open bilingual (Chinese-English) chat large language models released jointly by Zhipu AI and Tsinghua University's KEG Lab, including a 6B-parameter chat model, its base model, and 32K/128K long-context variants.
13.6K

4
kimi-k3-in-c
9.1KFareedKhan-dev/kimi-k3-in-c · C
An inference engine for the 2.78-trillion-parameter Kimi K3 model, written in portable C99. It runs on a plain CPU in as little as 8 GB of RAM — no GPU, no BLAS, no framework — streaming the 1.56 TB checkpoint straight off disk.
5
esp32-ai
4.6KslvDev/esp32-ai · Python
A 28.9 million parameter language model that runs text generation entirely on an ESP32-S3 microcontroller, with the generated text shown on a small screen wired to the chip.
6
MOSS-TTS-Nano
4.5KOpenMOSS/MOSS-TTS-Nano · Python
MOSS-TTS-Nano is an open-source, 100M-parameter multilingual text-to-speech model from MOSI.AI and OpenMOSS that supports voice cloning, 48kHz stereo output, and real-time generation directly on CPU.
7
Open-Source-Face-Recognition-SDK
2.7KFaceplugin-ltd/Open-Source-Face-Recognition-SDK · Python
An open-source face recognition SDK from Faceplugin that does face detection, landmark extraction, embedding generation and face similarity comparison, all processed on-premise.
8
earshot
1.5KLeonxlnx/earshot · Python
Earshot gives AI coding agents the ability to hear and reason about sound: it can describe what a file sounds like, judge how well audio matches a description, find the best moments in a recording, tell real recordings from code-synthesized ones, and critique a video's sound design against professional launch films.
9
jev
1.4Kfeder-cr/jev · C++
jevos-1b is a free, open-source yes/no decision engine that runs locally on your laptop's CPU, returning the probability that the answer is 'yes' for a given text plus question in about 50-220 ms, as a self-hosted alternative to the cloud-based Jev service.
10
decider
1.2KMapika/decider · Python
Decider is a family of language models fine-tuned from Qwen3.5 that don't generate text; instead they read a state plus a set of typed questions and return calibrated probability distributions for every question in one forward pass.
11
janas
449prabanta-dev/janas · C
Janas-LLM is a C-based inference engine that runs large mixture-of-experts LLMs (even ones bigger than your RAM) on ordinary laptops, by streaming the experts each token needs from an NVMe SSD and caching the hot ones in memory.
12
ParkerLabel
109Parker-Lyu/ParkerLabel · Python
ParkerLabel is a lightweight, AI-assisted interactive annotation tool for object detection and instance segmentation that uses MobileSAM to generate masks from click prompts.