Global

8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

A high-throughput and memory-efficient inference and serving engine for LLMs

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. 8-Flash-Next (125B) on 6GB, and DeepSeek-V3 (671B) on 12GB. 8-Flash-Next (125B) under 6GB.

5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

SGLang is a high-performance serving framework for large language models and multimodal models.

Local UI to run and train LLMs and diffusion models. 8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

pth） - 支持跨不同 GPU 数量恢复（自动调整 step） - 支持 wandb 训练记录连续性（自动恢复同一个 run）

FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

> 💡 This command can run on a GPU with at least 80GB VRAM.

Implementing LLM architectures from scratch in PyTorch

DeepSeek-R1-Distill models are fine-tuned based on open-source models, using samples generated by DeepSeek-R1. We slightly change their configs and tokenizers. Please use our setting to run these models.

NOTE] > The total size of DeepSeek-V3 models on Hugging Face is 685B, which includes 671B of the Main Model weights and 14B of the Multi-Token Prediction (MTP) Module weights.

Hy-MT2 is a family of “fast-thinking” multilingual translation models designed for complex real-world scenarios. 8B, 7B, and 30B-A3B (MoE), all of which support translation among 33 languages and effectively follow translation instructions in multiple languages. 5x.

DeepEP (DeepEveryParallel) is a high-performance communication library for modern machine learning training and inference.

🐧 PenguinHarness: Local-first multi-agent app development platform - create, fine-tune, deploy AI apps automatically!

24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

The Open Source Strix Halo AI engine. 56tg tok/s single user with DFlash2

GPU-optimized library for training transformer models at scale

🍔 In addition, ms-swift integrates the latest training technologies, including Megatron parallelism techniques such as TP, PP, CP, EP to accelerate training, as well as numerous GRPO algorithm family reinforcement learning algorithms including: GRPO, DAPO, GSPO, SAPO, CISPO…

Axolotl is a free and open-source tool designed to streamline post-training and fine-tuning for the latest large language models (LLMs).

DeepGEMM-Ascend: clean and efficient matrix multiplication kernel library for Huawei Ascend NPUs

Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

Kimi K3 is an open-weight, native multimodal agentic model and our most capable model to date. 8T-parameter model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window.

分享AI Infra知识&代码练习：PyTorch、vLLM/SGLang、slime/vime框架入门⚡️、性能加速🚀、大模型基础🧠、AI软硬件🔧等

Welcome to the gpt-oss series, OpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases.