Topics / #knowledge-distillation

Knowledge-distillation projects on GitHub

Training small student models to mimic large teacher models.

As of October 18, 2026, the most-starred Knowledge-distillation projects on GitHub are DeepSeek-R1 (91.9K stars), minimind (63.9K stars) and pytorch-image-models (37.2K stars). GitHub.Locker tracks 12 Knowledge-distillation repositories with 305.5K stars combined. The fastest-growing on October 16 was minimind, with +42 stars in a day.

12Projects

305.5KStars in total

+146Stars in a day

Browse in the catalog →github.com/topics/knowledge-distillation ↗

Hot right now

Most stars gained in a day · Oct 16

+42

minimind

jingyaogong/minimind · ★ 63.9K

MiniMind is an open-source project that lets you train a tiny large language model (around 64M parameters) completely from scratch, in about 2 hours on a single RTX 3090 for roughly the cost of 3 RMB in GPU rental.

+32

distilly

titanwings/distilly · ★ 25.7K

Distilly turns a person's source material, working habits, judgment, and voice into a versioned Person Profile for AI agents — recalled during a run or installed as a long-lived Skill. Formerly known as Colleague Skill.

+31

cangjie-skill

kangarooking/cangjie-skill · ★ 11.4K

A Python CLI pipeline that distills high-value content — books, long videos, podcasts, interviews, courses — into executable, callable AI agent skills, so the methodology gets invoked by an agent instead of gathering dust in notes and watch-later lists.

+13

MiniCPM

OpenBMB/MiniCPM · ★ 11.6K

MiniCPM is OpenBMB's family of small, open-source LLMs built for on-device and resource-constrained use. The newest release, MiniCPM5-2B, is a dense 2B-parameter model that reaches open-source state-of-the-art for its size class while staying competitive with much larger 4B-class models.

The all-time top

12 projects · by stars

#1 · deepseek-ai/DeepSeek-R1
DeepSeek-R1
DeepSeek-R1 is an open-source family of reasoning-focused large language models from DeepSeek, trained with large-scale reinforcement learning to handle math, code, and general reasoning at a level comparable to OpenAI's o1.
91.9K
#2 · jingyaogong/minimind
minimind
MiniMind is an open-source project that lets you train a tiny large language model (around 64M parameters) completely from scratch, in about 2 hours on a single RTX 3090 for roughly the cost of 3 RMB in GPU rental.
63.9K
#3 · huggingface/pytorch-image-models
pytorch-image-models
PyTorch Image Models (timm) — the largest collection of PyTorch image encoders and backbones, shipped with pretrained weights plus train, eval, inference and export scripts.
37.2K

4
distilly
25.7Ktitanwings/distilly · Python
Distilly turns a person's source material, working habits, judgment, and voice into a versioned Person Profile for AI agents — recalled during a run or installed as a long-lived Skill. Formerly known as Colleague Skill.
5
llm-action
25.2Kliguodongiot/llm-action · HTML
It is a Chinese-language hub of in-depth tutorials on large language model engineering: training, fine-tuning, distributed training, inference, compression, evaluation, data engineering, LLMOps and AI infrastructure, organized as a big index linking the author's article series, many with companion code.
6
leedl-tutorial
16.8Kdatawhalechina/leedl-tutorial · Jupyter Notebook
A free Chinese-language deep learning tutorial (nicknamed the 'Apple Book') based on Prof. Li Hongyi's (Hung-yi Lee) 'Machine Learning' course, covering most major deep learning topics in detail.
7
Swin-Transformer
16.1Kmicrosoft/Swin-Transformer · Python
The official Microsoft implementation of the Swin Transformer paper and its follow-ups: a hierarchical vision Transformer that computes attention with shifted windows and serves as a general-purpose backbone for computer vision tasks.
8
MiniCPM
11.6KOpenBMB/MiniCPM · Jupyter Notebook
MiniCPM is OpenBMB's family of small, open-source LLMs built for on-device and resource-constrained use. The newest release, MiniCPM5-2B, is a dense 2B-parameter model that reaches open-source state-of-the-art for its size class while staying competitive with much larger 4B-class models.
9
cangjie-skill
11.4Kkangarooking/cangjie-skill · Python
A Python CLI pipeline that distills high-value content — books, long videos, podcasts, interviews, courses — into executable, callable AI agent skills, so the methodology gets invoked by an agent instead of gathering dust in notes and watch-later lists.
10
Model-Optimizer
5.3KNVIDIA/Model-Optimizer · Python
NVIDIA Model Optimizer (ModelOpt) is a Python library that bundles state-of-the-art model-compression techniques - quantization, pruning, neural architecture search, distillation, speculative decoding and sparsity - to shrink and speed up deep learning models before deployment.
11
infinite-livestream
240reactor-team/infinite-livestream · Python
An end-to-end, chat-driven, never-ending AI video broadcast: viewers type !prompt in Twitch or YouTube chat, an LLM expands the idea into a styled scene, the FastH3 model renders it as a 768p clip with synced audio, and it goes out continuously over RTMP.
12
DMI
65ProjectDMX/DMI · Python
DMI is a decoupled, asynchronous observability stack that gives real-time access to a running LLM's internal state — residual streams, attention patterns, MLP outputs, KV-cache slices, logits — during inference or training, with minimal overhead.