Global

预览地址为 ，简体中文位于根目录，英文与繁體中文分别位于 en/ 和 zh-tw/。生成文件位于 build/，详细说明见网站构建与发布。

A high-performance distributed file system designed to address the challenges of AI training and inference workloads.

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

sh as its gate, so latest is always the current stack. /models), and serves on port 18020. Pick a mode — one GPU serves one at a time:

Use as the client base URL and the model ID from /v1/models. 2 or newer on a Mac (pip installs it). See the runbook for installation and a first request.

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

A comprehensive, open collection of Agent Skills focused on context engineering and harness engineering principles for building production-grade AI agent systems.

⭐ If LMCache helps you serve LLMs faster and cheaper, give us a star — it helps more teams discover the project.

TypeLLM brings type-safe generation to existing autoregressive LLMs without changing their architecture or weights. Inspired by TypeSafe AI's Jev, it lets models retain their native thinking and free-form generation while producing schema-guaranteed outputs through JSON Schema.

Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

This is the main change in how the project should be understood compared with the public 0.4.40 documentation.

A high-performance serving framework for robotics models.

Two production inference engines that share no cache format, no framework, no vendor and no quantization, serving one request together.

Given a context and a list of pre-written options, the model prefills the context once, expands that KV cache across the option batch, and scores every option in a single padded forward pass. No decoding.