Global

8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. 8-Flash-Next (125B) on 6GB, and DeepSeek-V3 (671B) on 12GB. 8-Flash-Next (125B) under 6GB.

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

A high-throughput and memory-efficient inference and serving engine for LLMs

pth） - 支持跨不同 GPU 数量恢复（自动调整 step） - 支持 wandb 训练记录连续性（自动恢复同一个 run）

SGLang is a high-performance serving framework for large language models and multimodal models.

Implementing LLM architectures from scratch in PyTorch

Local UI to run and train LLMs and diffusion models. 8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

🐧 PenguinHarness: Local-first multi-agent app development platform - create, fine-tune, deploy AI apps automatically!

FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.

> 💡 This command can run on a GPU with at least 80GB VRAM.

24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

NOTE] > The total size of DeepSeek-V3 models on Hugging Face is 685B, which includes 671B of the Main Model weights and 14B of the Multi-Token Prediction (MTP) Module weights.

\janas-chat. 8 times as fast with the dense model (Qwen3-4B 346 tok/s against 193), whose whole blocks go to the GPU, through Intel's own Windows driver there.

DeepSeek-R1-Distill models are fine-tuned based on open-source models, using samples generated by DeepSeek-R1. We slightly change their configs and tokenizers. Please use our setting to run these models.

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

8 is the large language model series developed by Qwen team, Alibaba Group.

The Open Source Strix Halo AI engine. 56tg tok/s single user with DFlash2

GPU-optimized library for training transformer models at scale

🍔 In addition, ms-swift integrates the latest training technologies, including Megatron parallelism techniques such as TP, PP, CP, EP to accelerate training, as well as numerous GRPO algorithm family reinforcement learning algorithms including: GRPO, DAPO, GSPO, SAPO, CISPO…

Hy-MT2 is a family of “fast-thinking” multilingual translation models designed for complex real-world scenarios. 8B, 7B, and 30B-A3B (MoE), all of which support translation among 33 languages and effectively follow translation instructions in multiple languages. 5x.

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

Run MoE models beyond VRAM on AMD, NVIDIA & Intel GPUs. Vulkan-powered, portable Windows app with automatic setup. | 让游戏显卡跑起超显存大模型，支持 AMD、NVIDIA、Intel，免安装、自动配置。只需13MB，即下即用！

Kimi K3 is an open-weight, native multimodal agentic model and our most capable model to date. 8T-parameter model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window.

Welcome to the gpt-oss series, OpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases.

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Axolotl is a free and open-source tool designed to streamline post-training and fine-tuning for the latest large language models (LLMs).