Quantization projects on GitHub

1. 1
   #1 · ggml-org/llama.cpp

   llama.cpp

   llama.cpp is a plain C/C++ implementation for running LLM (and VLM) inference with minimal setup and state-of-the-art performance, either locally or in the cloud.

   131.4K
2. 2
   #2 · deepseek-ai/DeepSeek-V3

   DeepSeek-V3

   DeepSeek-V3 is an open-weight Mixture-of-Experts (MoE) language model with 671B total parameters but only 37B activated per token, built for efficient yet high-performing inference and matching leading closed-source models on many benchmarks.

   104.5K
3. 3
   #3 · vllm-project/vllm

   vllm

   vLLM is a library for fast LLM inference and serving, built around PagedAttention for efficient memory management of attention keys and values.

   94K

4. 4

   llm-course

   83.7Kmlabonne/llm-course

   A free, open-source course for getting into Large Language Models (LLMs), combining theory roadmaps with hands-on Colab notebooks covering everything from math fundamentals to fine-tuning and deployment.

   • roadmap
   • llms
   • course
   • jupyter notebook
5. 5

   unsloth

   78.2Kunslothai/unsloth · Python

   Unsloth is a local-first app for running and training LLMs, diffusion, TTS and embedding models on your own machine, including models like Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4 and FLUX.

   • dpo
   • qlora
   • intel
   • kaggle
6. 6

   LlamaFactory

   75.5Khiyouga/LlamaFactory · Python

   LlamaFactory unifies efficient fine-tuning of 100+ large language models and vision-language models in one Python toolkit, published as an ACL 2024 paper.

   • llama3
   • peft
   • qlora
   • instruction-tuning
7. 7

   whisper.cpp

   54.4Kggml-org/whisper.cpp · C++

   whisper.cpp is a high-performance C/C++ port of OpenAI's Whisper automatic speech recognition (ASR) model, letting you transcribe audio to text without Python or heavy ML frameworks.

   • on-device
   • whisper
   • vulkan
   • speech-to-text
8. 8

   grok-1

   52.2Kxai-org/grok-1 · Python

   Grok-1 is the 314B-parameter Mixture-of-Experts language model that xAI opened up; this repository holds the JAX example code for loading and running it.

   • tokenizer
   • xai
   • jax
   • torrent
9. 9

   BitNet

   41.2Kmicrosoft/BitNet · C++

   bitnet.cpp is Microsoft's official inference framework for 1-bit (ternary, 1.58-bit) large language models like BitNet b1.58, letting them run fast on CPUs and GPUs.

   • cpu
   • x86
   • arm
   • on-device
10. 10

    faiss

    41.2Kfacebookresearch/faiss · C++

    Faiss is a library for efficient similarity search and clustering of dense vectors, developed primarily at Meta's Fundamental AI Research group. It can search collections of vectors of any size, even ones that do not fit in RAM.

    • nearest-neighbor-search
    • similarity-search
    • numpy
    • vector-search
11. 11

    ChatGLM-6B

    40.9Kzai-org/ChatGLM-6B · Python

    ChatGLM-6B is an open bilingual (Chinese-English) dialogue language model with 6.2 billion parameters, based on the General Language Model (GLM) architecture.

    • chatglm
    • open-weights
    • glm
    • bilingual
12. 12

    FastChat

    39.5Klm-sys/FastChat · Python

    FastChat is an open platform for training, serving, and evaluating large language model based chatbots. It is the release repo for Vicuna and powers Chatbot Arena (lmarena.ai), which has served over 10 million chat requests for 70+ LLMs.

    • gradio
    • vllm
    • fine-tuning
    • openai-compatible
13. 13

    airllm

    39.1Klyogavin/airllm · Jupyter Notebook

    An open-source Python library that runs huge LLMs on tiny GPUs: a 70B model fits on a single 4GB card without quantization, distillation or pruning — even the 2.8T-parameter Kimi K3 runs in under 4GB.

    • consumer-gpu
    • chinese-nlp
    • qlora
    • mlx
14. 14

    llmfit

    38KAlexsJones/llmfit · Rust

    llmfit inspects your CPU, RAM, GPU(s), VRAM, and accelerator setup, then tells you which open-source LLMs and quantizations will actually run well on your machine.

    • localai
    • llama-cpp
    • mlx
    • gguf
15. 15

    LLM101n

    37.5Kkarpathy/LLM101n

    LLM101n is a planned course by Andrej Karpathy that walks through building a 'Storyteller' AI language model end-to-end, from basics all the way to a ChatGPT-like web app, implemented from scratch in Python, C and CUDA.

    • attention
    • tokenization
    • rlhf
    • jax
16. 16

    sglang

    37.2Ksgl-project/sglang · Python

    SGLang is a high-performance serving framework for large language models and multimodal models, built to deliver low-latency, high-throughput inference from a single GPU up to large distributed clusters.

    • blackwell
    • qwen-image
    • tpu
    • wan
17. 17

    heretic

    35.6Kp-e-w/heretic · Python

    Heretic is a command-line tool that automatically removes censorship (safety alignment) from transformer-based language models using directional ablation combined with an Optuna-powered parameter optimizer, without any training.

    • uncensored
    • optimization
    • evaluation
    • transformer
18. 18

    qdrant

    35.1Kqdrant/qdrant · Rust

    Qdrant is a high-performance vector similarity search engine and vector database written in Rust, built to store, search, and manage 'points' (vectors plus a JSON payload) for AI applications like semantic search, recommendations, and RAG.

    • image-search
    • neural-search
    • recommender-system
    • vector-search-engine
19. 19

    insightface

    30Kdeepinsight/insightface · Python

    An open-source 2D and 3D deep face analysis toolbox that efficiently implements state-of-the-art face recognition, detection and alignment, optimized for both training and deployment.

    • paddlepaddle
    • mxnet
    • onnxruntime
    • face-recognition
20. 20

    cs249r_book

    29Kharvard-edge/cs249r_book · Python

    An open textbook and full curriculum on Machine Learning Systems — AI engineering — from Harvard's CS249r course: four volumes covering foundations, scaling, agentic AI, and physical AI, plus labs, projects, and instructor materials.

    • teaching
    • physical-ai
    • computer-architecture
    • simulator
21. 21

    Qwen3

    27.7KQwenLM/Qwen3 · Python

    Qwen3 is a series of open-weight large language models built by the Qwen team at Alibaba Cloud, released as both dense and Mixture-of-Experts (MoE) models from 0.6B up to 235B-A22B parameters, with switchable 'thinking' and 'non-thinking' reasoning modes.

    • alibaba
    • qwen3
    • llamacpp
    • foundation-models
22. 22

    MiniCPM-V

    26.8KOpenBMB/MiniCPM-V · Python

    A family of pocket-sized multimodal AI models from OpenBMB (MiniCPM-V and MiniCPM-o) built to run on local devices, even phones — understanding images, video and text, and in the o-series also speech.

    • harmonyos
    • video-understanding
    • llama-cpp
    • on-device
23. 23

    Strata

    26.4KNiko1221/Strata · C++

    Strata runs Qwen3.8-Flash-Next, a 125-billion-parameter AI model, on a normal gaming PC with one NVIDIA GPU (12-24GB VRAM) and 64GB of RAM, with a one-click installer for Windows and Linux.

    • llama-cpp
    • speculative-decoding
    • inference-engine
    • mixture-of-experts
24. 24

    faster-whisper

    25.9KSYSTRAN/faster-whisper · Python

    A Python reimplementation of OpenAI's Whisper model on CTranslate2, a fast Transformer inference engine — up to 4x faster than the original at the same accuracy, using less memory, with optional 8-bit quantization on CPU and GPU.

    • speech
    • asr
    • transcription
    • audio-processing
25. 25

    llm-action

    25.2Kliguodongiot/llm-action · HTML

    It is a Chinese-language hub of in-depth tutorials on large language model engineering: training, fine-tuning, distributed training, inference, compression, evaluation, data engineering, LLMOps and AI infrastructure, organized as a big index linking the author's article series, many with companion code.

    • peft
    • ai-infrastructure
    • knowledge-distillation
    • llm-training
26. 26

    LLaVA

    25.1Khaotian-liu/LLaVA · Python

    LLaVA is an open-source large multimodal chatbot: a vision-language model you can show images to and talk with, built by visual instruction tuning toward GPT-4V-level capabilities (NeurIPS 2023 Oral).

    • llama2
    • instruction-tuning
    • gpt-4
    • foundation-models
27. 27

    DeepSeek-Coder

    24.3Kdeepseek-ai/DeepSeek-Coder · Python

    DeepSeek Coder is a family of open-source code language models trained from scratch on 2 trillion tokens (87% code, 13% natural language in English and Chinese), built for code generation, completion, and infilling.

    • finetuning
    • code-generation
    • vllm
    • transformer
28. 28

    ncnn

    24KTencent/ncnn · C++

    ncnn is a high-performance neural network inference framework built by Tencent and optimized for mobile, embedded, and desktop deployment. It is used in Tencent apps like QQ, Qzone, and WeChat to run deep learning models on-device.

    • darknet
    • riscv
    • ncnn
    • mxnet
29. 29

    ds4

    23.9Kantirez/ds4 · C

    DwarfStar (ds4) is a small, self-contained local inference engine specialized for running DeepSeek V4 Flash and PRO, plus GLM 5.2/5.3, on consumer hardware like high-memory Macs, NVIDIA GPUs, and AMD Strix Halo systems.

    • kv-cache
    • llm-serving
    • amd-gpu
    • glm
30. 30

    Qwen

    21.9KQwenLM/Qwen · Python

    The official repo of Qwen (通义千问, Tongyi Qianwen) — Alibaba Cloud's family of chat and pretrained large language models, released in sizes from 1.8B up to 72B parameters on Hugging Face and ModelScope.

    • finetuning
    • flash-attention
    • long-context
    • alibaba
31. 31

    peft

    21.8Khuggingface/peft · Python

    Hugging Face's library of Parameter-Efficient Fine-Tuning methods: you adapt a large pretrained model by training only a small number of extra parameters instead of all of them, which slashes compute and storage costs.

    • peft
    • llm-training
    • rlhf
    • diffusion
32. 32

    candle

    21.2Khuggingface/candle · Rust

    Candle is a minimalist machine learning framework for Rust, focused on performance (including GPU support) and ease of use. It ships with ready examples for LLMs, speech, vision and text-to-image models.

    • yolo
    • stable-diffusion
    • gguf
    • whisper
33. 33

    gpt-oss

    20.5Kopenai/gpt-oss · Python

    The gpt-oss series: two open-weight language models from OpenAI — gpt-oss-120b and gpt-oss-20b — built for powerful reasoning, agentic tasks, and versatile developer use cases. The repo itself adds reference implementations, tools, and example clients around the models.

    • chain-of-thought
    • lmstudio
    • gpt-oss
    • function-calling
34. 34

    llama2.c

    20.2Kkarpathy/llama2.c · C

    It's a minimal project to train a small Llama 2 model in PyTorch and then run inference on it using a single pure C file, with no dependencies.

    • llama2
    • from-scratch
    • c
    • inference
35. 35

    Qwen3-VL

    20.1KQwenLM/Qwen3-VL · Jupyter Notebook

    Qwen3-VL is a family of multimodal vision-language models built by the Qwen team at Alibaba Cloud, combining strong text understanding with deep visual perception, reasoning, and agent capabilities.

    • video-understanding
    • alibaba
    • edge-ai
    • vision-language-model
36. 36

    ktransformers

    19.6Kkvcache-ai/ktransformers · Python

    KTransformers is a research framework for efficient inference and fine-tuning of large language models through CPU-GPU heterogeneous computing, letting huge MoE models run on hardware that doesn't have enough VRAM to hold them entirely on GPU.

    • cpu-inference
    • intel
    • sglang
    • amd-gpu
37. 37

    trl

    19.5Khuggingface/trl · Python

    TRL (Transformers Reinforcement Learning) is a Hugging Face library for post-training foundation models — fine-tuning them with techniques like SFT, GRPO and DPO on top of the Transformers stack.

    • dpo
    • grpo
    • rlhf
    • lora
38. 38

    Chinese-LLaMA-Alpaca

    18.9Kymcui/Chinese-LLaMA-Alpaca · Python

    Open-source Chinese LLaMA and Alpaca large language models. The base LLaMA was extended with a Chinese vocabulary and further pre-trained on Chinese data, and Alpaca versions were instruction-tuned on Chinese instructions; training and deployment scripts are open too.

    • llama3
    • llama2
    • instruction-tuning
    • pretraining
39. 39

    alpaca-lora

    18.9Ktloen/alpaca-lora · Jupyter Notebook

    Alpaca-LoRA reproduces the Stanford Alpaca results by instruct-tuning LLaMA with low-rank adaptation (LoRA). It ships the training code, inference scripts, and the resulting LoRA weights, producing an Instruct model of similar quality to text-davinci-003.

    • finetuning
    • llama
    • gradio
    • lora
40. 40

    Qbot

    18.7KUFund-Me/Qbot · Jupyter Notebook

    Qbot is a free, AI-oriented quantitative investment research platform that covers the whole loop: data acquisition, strategy development, backtesting, simulated trading, and live trading.

    • quantitative-trading
    • bitcoin
    • trading-bot
    • quant
41. 41

    Megatron-LM

    18.2KNVIDIA/Megatron-LM · Python

    NVIDIA's open-source toolkit for training large transformer (LLM) models at scale. It comes as Megatron-LM, a reference setup with ready training scripts, and Megatron Core, a composable library of GPU-optimized building blocks for training models across thousands of GPUs.

    • parallel
    • model-training
    • moe
    • transformers
42. 42

    ml-stable-diffusion

    18Kapple-aiml-research/ml-stable-diffusion · Python

    This is Apple's toolkit for running Stable Diffusion image generation on Apple Silicon using Core ML, so it can run efficiently on Mac, iPad, and iPhone.

    • coreml
    • on-device
    • stable-diffusion
    • text-to-image
43. 43

    turbovec

    17.4KRyanCodrai/turbovec · Rust

    A vector index for nearest-neighbor search over embeddings, written in Rust with Python bindings. It is built on Google Research's TurboQuant quantization and aimed at RAG where memory, latency and privacy matter.

    • neon
    • faiss
    • embedding
    • simd
44. 44

    transformers.js

    16.3Khuggingface/transformers.js · JavaScript

    Transformers.js lets you run Hugging Face's state-of-the-art machine learning models directly in a web browser, with no server required, using an API designed to mirror the Python transformers library.

    • jax
    • image-classification
    • object-detection
    • onnx
45. 45

    MNN

    16.2Kalibaba/MNN · C++

    MNN is a lightweight, high-performance deep learning inference (and training) engine built by Alibaba, used to run models like LLMs, diffusion models, and CNNs efficiently on mobile, embedded, and edge devices.

    • opencl
    • ml
    • arm
    • edge-ai
46. 46

    zvec

    16.1Kalibaba/zvec · C++

    Alibaba's open-source, in-process vector database — a lightweight, lightning-fast similarity-search engine you embed directly into your application, battle-tested inside Alibaba Group.

    • hnsw
    • faiss
    • similarity-search
    • llm-memory
47. 47

    ms-swift

    15.9Kmodelscope/ms-swift · Python

    ms-swift is the ModelScope community's fine-tuning and deployment framework for LLMs and multimodal models, covering training, inference, evaluation, quantization and deployment in one pipeline.

    • peft
    • embedding
    • sft
    • grpo
48. 48

    ChatGLM2-6B

    15.5Kzai-org/ChatGLM2-6B · Python

    ChatGLM2-6B is an open bilingual (Chinese-English) conversational language model with 6 billion parameters, the second generation of ChatGLM-6B from Zhipu AI (Z.ai).

    • chatglm
    • glm
    • bilingual
    • gradio
49. 49

    ggml

    15.5Kggml-org/ggml · C++

    ggml is a plain C/C++ tensor library built for running machine learning models with minimal setup, no external dependencies, and support for CPU, GPU, NPU, and browser backends. It's the core engine that powers projects like llama.cpp.

    • tensor
    • risc-v
    • simd
    • cpu
50. 50

    llm_interview_note

    15.3Kwdndev/llm_interview_note · HTML

    This is a curated Chinese-language knowledge base of large language model (LLM) concepts and interview questions, aimed at algorithm/application engineers preparing for LLM-related job interviews.

    • interview-prep
    • rlhf
    • function-calling
    • coding-interview
51. 51

    TensorRT-LLM

    14.8KNVIDIA/TensorRT-LLM · Python

    TensorRT LLM is NVIDIA's Python framework for defining large language models (and now visual generation models) and compiling them into highly optimized inference engines that run on NVIDIA GPUs, with specialized kernels and Python/C++ runtimes.

    • blackwell
    • llm-serving
    • speculative-decoding
    • moe
52. 52

    Llama-Chinese

    14.8KLlamaChinese/Llama-Chinese · Python

    Llama-Chinese is a Chinese-language community project built around Meta's Llama models (Llama2, Llama3, Llama4). It provides Chinese pretraining and fine-tuning, a custom Atom model series optimized for Chinese, and resources for training, deploying and using Llama in Chinese for free commercial use.

    • rl
    • chinese-nlp
    • pretraining
    • llama-cpp
53. 53

    FreeToken

    14.5KFlashML-org/FreeToken · Python

    FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine built to run frontier-scale open-weight models, including 290B+ parameter models, on consumer hardware like gaming PCs.

    • edge
    • minimax
    • glm
    • edge-ai
54. 54

    needle

    14.1Kcactus-compute/needle · Python

    Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction that ships as a single 14MB binary and runs a full session in about 28MB of RAM. This repo is its Python package for inference, LoRA fine-tuning and export.

    • jax
    • function-calling
    • on-device-ai
    • tool-calling
55. 55

    LoRA

    13.9Kmicrosoft/LoRA · Python

    LoRA (loralib) is Microsoft's reference implementation of the paper 'LoRA: Low-Rank Adaptation of Large Language Models'. It lets you fine-tune large models like GPT-2, RoBERTa and DeBERTa by training small low-rank matrices instead of all the original weights.

    • gpt-2
    • gpt-3
    • language-model
    • paper
56. 56

    litgpt

    13.7KLightning-AI/litgpt · Python

    LitGPT is a collection of 20+ high-performance large language models (Llama, Gemma, Phi, Qwen, Mistral, Falcon, TinyLlama and more) implemented from scratch, with ready-made recipes to pretrain, finetune, evaluate and deploy them at scale.

    • tpu
    • qlora
    • llama
    • llms
57. 57

    color-thief

    13.7Klokesh/color-thief · TypeScript

    Color Thief is a JavaScript library that extracts the dominant color or a full color palette from an image, working the same way in both the browser and Node.js.

    • color-picker
    • photos
    • color
    • analysis
58. 58

    ChatGLM3

    13.6Kzai-org/ChatGLM3 · Python

    ChatGLM3 is a series of open bilingual (Chinese-English) chat large language models released jointly by Zhipu AI and Tsinghua University's KEG Lab, including a 6B-parameter chat model, its base model, and 32K/128K long-context variants.

    • cpu-inference
    • streamlit
    • function-calling
    • bilingual
59. 59

    AI-Research-SKILLs

    13.6KOrchestra-Research/AI-Research-SKILLs · TeX

    It's an open-source library of 98 AI research and engineering skills that turn a Claude Code, Codex, or Gemini CLI agent into a full AI research agent, covering everything from idea generation to paper writing.

    • gpt-5
    • paper-writing
    • grpo
    • ai-research
60. 60

    TensorRT

    13.4KNVIDIA/TensorRT · C++

    TensorRT is NVIDIA's SDK for high-performance deep learning inference on NVIDIA GPUs. This repository holds the open source components: plugins, the ONNX parser, and sample applications that demonstrate TensorRT usage.

    • tensorrt
    • model-serving
    • onnx
    • llm-inference
61. 61

    DiffSynth-Studio

    13.3Kmodelscope/DiffSynth-Studio · Python

    DiffSynth-Studio is an open-source Diffusion model engine built and maintained by the ModelScope Community, covering image, video, and audio generation plus image quality metrics, with support for both inference and training.

    • music-generation
    • model-training
    • diffusion
    • comfyui
62. 62

    CogVideo

    13.1Kzai-org/CogVideo · Python

    An open-source family of text-to-video and image-to-video models: CogVideoX (2024) and its CogVideoX1.5 upgrade, following CogVideo (ICLR 2023), the first open-source large Transformer-based text-to-video model.

    • image-to-video
    • text-to-video
    • diffusion-models
    • lora
63. 63

    FlashMLA

    13.1Kdeepseek-ai/FlashMLA · C++

    FlashMLA is DeepSeek's library of highly optimized CUDA attention kernels (Multi-head Latent Attention) that power the DeepSeek-V3 and DeepSeek-V3.2 models, covering both dense and sparse attention for prefill and decoding.

    • attention-mechanism
    • llm-inference
    • high-performance
    • gpu-acceleration
64. 64

    PaddleNLP

    13KPaddlePaddle/PaddleNLP · Python

    PaddleNLP is a large language model (LLM) and small language model (SLM) development toolkit built on the PaddlePaddle deep learning framework, offering efficient training, lossless compression, and high-performance inference for large models.

    • document-intelligence
    • neural-search
    • embedding
    • bert
65. 65

    the-incredible-pytorch

    12.7Kritchieng/the-incredible-pytorch

    A curated list of tutorials, papers, projects, libraries, videos, books and communities related to PyTorch, covering everything from basic deep learning to LLMs, agentic AI, and reinforcement learning.

    • deep-learning-tutorial
    • deep-neural-networks
    • reinforcement-learning
    • fine-tuning
66. 66

    HunyuanVideo

    12.6KTencent-Hunyuan/HunyuanVideo · Python

    HunyuanVideo is Tencent's open-source video foundation model, a 13B+ parameter diffusion transformer that generates videos from text (and images) at quality the authors claim matches or beats closed-source competitors like Runway Gen-3 and Luma 1.6.

    • comfyui
    • pretrained-models
    • diffusion-models
    • video-generation
67. 67

    axolotl

    12.6Kaxolotl-ai-cloud/axolotl · Python

    Axolotl is a free, open-source framework that streamlines post-training and fine-tuning of large language models — you describe your run in one YAML config and it drives the whole pipeline.

    • distributed-training
    • moe
    • lora
    • training
68. 68

    MOSS

    12.3KOpenMOSS/MOSS · Python

    MOSS is an open-source, tool-augmented bilingual (Chinese-English) conversational language model built by Fudan University's OpenMOSS team, released with model weights, training data, and deployment/finetuning code.

    • tool-use
    • text-generation
    • streamlit
    • gradio
69. 69

    ludwig

    11.8Kludwig-ai/ludwig · Python

    Ludwig is a declarative deep learning framework that lets you train, fine-tune, and deploy AI models — from LLM fine-tuning to tabular classification — using a YAML config file and zero boilerplate Python.

    • machinelearning
    • llama2
    • mistral
    • llm-training
70. 70

    MiniCPM

    11.6KOpenBMB/MiniCPM · Jupyter Notebook

    MiniCPM is OpenBMB's family of small, open-source LLMs built for on-device and resource-constrained use. The newest release, MiniCPM5-2B, is a dense 2B-parameter model that reaches open-source state-of-the-art for its size class while staying competitive with much larger 4B-class models.

    • tool-use
    • knowledge-distillation
    • long-context
    • llamacpp
71. 71

    LTX-Video

    11.1KLightricks/LTX-Video · Python

    LTX-Video is the official repo for Lightricks' DiT-based video generation model, which generates synchronized audio+video, supports text-to-video and image-to-video, keyframe conditioning, video extension, and video-to-video transformations.

    • foundation-models
    • image-to-video
    • comfyui
    • text-to-video
72. 72

    qlora

    11Kartidoro/qlora · Jupyter Notebook

    QLoRA is a research method and code release for efficiently finetuning large quantized language models. It backpropagates gradients through a frozen 4-bit quantized model into small LoRA adapters, cutting memory use enough to finetune a 65B model on a single 48GB GPU while matching full 16-bit finetuning quality.

    • finetuning
    • peft
    • qlora
    • llama
73. 73

    openvino

    11Kopenvinotoolkit/openvino · C++

    OpenVINO is Intel's open-source toolkit for optimizing and deploying AI inference — it takes trained deep learning models for vision, speech, NLP, and generative AI and runs them faster, from edge devices to cloud servers.

    • openvino
    • recommendation-system
    • good-first-issue
    • yolo
74. 74

    text-generation-inference

    10.9Khuggingface/text-generation-inference · Python

    Text Generation Inference (TGI) is a toolkit for deploying and serving large language models — a Rust, Python and gRPC server that powers Hugging Chat, the Inference API and Inference Endpoints at Hugging Face.

    • rocm
    • speculative-decoding
    • model-serving
    • openai-api
75. 75

    nlp_course

    10.7Kyandexdataschool/nlp_course · Jupyter Notebook

    This is the Yandex School of Data Analysis (YSDA) Natural Language Processing course: a full set of lecture slides, seminar notebooks, and homework assignments organized by week, covering NLP from word embeddings through modern LLM topics.

    • interpretability
    • university
    • rlhf
    • attention-mechanism
76. 76

    WanGP

    10.6Kdeepbeepmeep/WanGP · Python

    WanGP is a one-stop open source app for running top generative AI models for video, image, and audio/TTS generation, built specifically to work on modest consumer GPUs (the 'GPU Poor').

    • finetuning
    • wan
    • background-removal
    • flux
77. 77

    demucs

    10.4Kfacebookresearch/demucs · Python

    Demucs is a state-of-the-art music source separation model from Meta's research lab: give it a song and it splits it into stems — drums, bass, vocals and the rest of the accompaniment.

    • karaoke
    • audio-processing
    • music
    • neural-network
78. 78

    InternVL

    10.2KOpenGVLab/InternVL · Python

    A family of open-source multimodal chat models that understand images, video and text, positioned as an open alternative to GPT-4o. InternVL2.5-78B was the first open-source model to top 70% on the MMMU benchmark.

    • gpt-4o
    • semantic-segmentation
    • open-weights
    • vlm
79. 79

    LTX-2

    9.8KLightricks/LTX-2 · Python

    The official Python package for LTX-2, Lightricks' audio-video foundation model: one DiT-based model that generates video with synchronized audio from text or an image, in several quality/speed modes.

    • video-upscaling
    • flash-attention
    • foundation-models
    • diffusion
80. 80

    Soup

    9.2KMakazhanAlpamys/Soup · Python

    Soup is a command-line tool for fine-tuning and post-training LLMs from a single YAML config file, automating GPU detection, batching and quantization so you just write a config and run one command.

    • consumer-gpu
    • dpo
    • peft
    • qlora
81. 81

    LongCat-Video

    9.2Kmeituan-longcat/LongCat-Video · Python

    LongCat-Video is a 13.6B-parameter foundational video generation model that unifies Text-to-Video, Image-to-Video, and Video-Continuation generation in a single architecture, aimed at efficient long-video generation and framed as a first step toward world models.

    • talking-head
    • digital-human
    • lipsync
    • ai-video-generation
82. 82

    kimi-k3-in-c

    9.1KFareedKhan-dev/kimi-k3-in-c · C

    An inference engine for the 2.78-trillion-parameter Kimi K3 model, written in portable C99. It runs on a plain CPU in as little as 8 GB of RAM — no GPU, no BLAS, no framework — streaming the 1.56 TB checkpoint straight off disk.

    • cpu-inference
    • from-scratch
    • simd
    • inference-engine
83. 83

    DeepGEMM

    9Kdeepseek-ai/DeepGEMM · Cuda

    DeepGEMM is a unified CUDA kernel library that provides high-performance tensor core GEMM operations (FP8, FP4, BF16), fused MoE kernels, and attention-indexer kernels used in large language models such as DeepSeek.

    • tensor
    • cuda-kernels
    • jit
    • gpu-programming
84. 84

    Kimi-K3

    8.9KMoonshotAI/Kimi-K3

    Kimi K3 is MoonshotAI's open-weight, native multimodal agentic model — a 2.8T-parameter Mixture-of-Experts system built for frontier-level long-horizon coding, agentic knowledge work, and reasoning.

    • long-horizon-agents
    • deep-research
    • open-weights
    • model
85. 85

    OBLITERATUS

    8.8Kelder-plinius/OBLITERATUS · Python

    OBLITERATUS is an open-source toolkit for 'abliteration' — it probes a language model's hidden states to locate the internal directions responsible for refusing prompts, then surgically removes them so the model stops refusing while keeping its core language abilities.

    • uncensored
    • interpretability
    • red-teaming
    • ai-safety
86. 86

    tilelang

    8.6Ktile-ai/tilelang · Python

    TileLang (tile-lang) is a domain-specific language for writing high-performance GPU/CPU/NPU kernels like GEMM, Dequant GEMM, FlashAttention, and LinearAttention, using Pythonic syntax backed by a TVM-based compiler.

    • blackwell
    • cuda-kernels
    • dsl
    • gpu-programming
87. 87

    stable-diffusion.cpp

    7.7Kleejet/stable-diffusion.cpp · C++

    A pure C/C++ inference engine for diffusion models like Stable Diffusion, Flux, Wan, Qwen Image, and Z-Image, built the same way as llama.cpp so you can generate images and videos without heavy Python dependencies.

    • esrgan
    • text2image
    • qwen-image
    • wan
88. 88

    maths-cs-ai-compendium

    7.7KHenryNdubuaku/maths-cs-ai-compendium · TypeScript

    An open, unconventional online textbook covering maths, computing and AI from the ground up — intuition first, real-world context, for practitioners who want deep understanding rather than exam survival.

    • graph-neural-networks
    • machine-learning-algorithms
    • linear-algebra
    • from-scratch
89. 89

    ANE

    7.3Kmaderix/ANE · Objective-C

    ANE is a research project that trains neural networks directly on Apple's Neural Engine using reverse-engineered private APIs, instead of CoreML's inference-only path.

    • model-training
    • edge-ai
    • objective-c
    • optimization
90. 90

    vllm-omni

    7.2Kvllm-project/vllm-omni · Python

    vLLM-Omni is an inference and serving framework that extends vLLM beyond text-only LLMs to omni-modality models, handling text, image, audio, video, and action outputs.

    • world-model
    • world-models
    • diffusion
    • model-serving
91. 91

    turboquant_plus

    7KTheTom/turboquant_plus · Python

    TurboQuant+ is the research home for TurboQuant (ICLR 2026), a KV cache compression method for transformer LLMs. It shrinks the KV cache 3.8-6.4x using PolarQuant plus Walsh-Hadamard rotation, and this repo holds the Python reference implementation, validation papers and benchmark data.

    • long-context
    • llamacpp
    • mlx
    • amd
92. 92

    turbo-fieldfare

    6.9Kdrumih/turbo-fieldfare · Swift

    TurboFieldfare is a custom Swift and Metal runtime that runs the instruction-tuned Gemma 4 26B-A4B model on Apple Silicon Macs using only about 2 GB of RAM, even on 8 GB machines, by streaming only the experts needed for each token from SSD instead of loading the full 14.3 GB model.

    • gpgpu
    • gemma
    • on-device-ai
    • edge-ai
93. 93

    SenseNova-U1

    6.9KOpenSenseNova/SenseNova-U1 · Python

    Open-source multimodal models that unify image understanding, reasoning and image generation in one architecture. The current release, SenseNova-U1.5-8B-MoT, creates and edits images up to native 4K.

    • open-weights
    • vision-language-model
    • comfyui
    • gguf
94. 94

    lectures

    6.9Kgpu-mode/lectures · Jupyter Notebook

    This is the supplementary material repository for the GPU MODE lectures YouTube channel — a growing collection of notebooks, slides and code from each numbered lecture on GPU and CUDA programming.

    • triton
    • flash-attention
    • sglang
    • gpu-programming
95. 95

    whichllm

    6.8KAndyyyy64/whichllm · Python

    A CLI tool that finds the best local LLM for your machine: it auto-detects your GPU/CPU/RAM and ranks HuggingFace models that fit, scored by real benchmarks rather than parameter count.

    • gguf
    • hardware
    • local-llm
    • apple-silicon
96. 96

    neutts

    6.3Kneuphonic/neutts · Python

    NeuTTS is a collection of open-source, on-device text-to-speech language models that can clone a voice from just a few seconds of reference audio.

    • on-device
    • voice-assistant
    • voice-cloning
    • speech-synthesis
97. 97

    dflash

    6.2Kz-lab/dflash · Python

    DFlash is a lightweight block diffusion model built for speculative decoding, drafting tokens in parallel blocks so a larger target LLM can verify them faster than generating one token at a time.

    • gpt-oss
    • gemma
    • minimax
    • sglang
98. 98

    Model-Optimizer

    5.3KNVIDIA/Model-Optimizer · Python

    NVIDIA Model Optimizer (ModelOpt) is a Python library that bundles state-of-the-art model-compression techniques - quantization, pruning, neural architecture search, distillation, speculative decoding and sparsity - to shrink and speed up deep learning models before deployment.

    • knowledge-distillation
    • sglang
    • speculative-decoding
    • edge-ai
99. 99

    Edge0

    5.3KEdge0-AI/Edge0 · Python

    Edge0 is an open-source streaming MoE (Mixture-of-Experts) inference framework that lets you run large MoE models, like a 35B-parameter model, using only about 2.5-2.9 GB of active RAM by streaming expert weights from SSD and predicting routing one step ahead.

    • on-device-ai
    • edge-ai
    • language-model
    • moe
100. 100

     parameter-golf

     5.2Kopenai/parameter-golf · Python

     This is OpenAI's 'Model Craft Challenge: Parameter Golf' - a competition to train the best language model that fits in a 16MB artifact and trains in under 10 minutes on 8xH100 GPUs, scored by bits-per-byte compression on the FineWeb validation set.

     • triton
     • tokenizer
     • compression
     • attention-mechanism

Showing the 100 most-starred of 183. See every one in the catalog.

• python2,383
• gpu789
• llm1,483
• inference327
• machine-learning916
• deep-learning737
• linux4,184
• huggingface362
• open-source5,666
• ai1,955
• cuda396
• apache-2.01,434
• llm-inference119
• pytorch419
• large-language-models213
• nvidia328
• macos3,544
• fine-tuning185
• benchmark431
• research764
• local-llm218
• transformer212
• mit3,312
• windows3,403