Kimi-K3 README

The project's README from github.com/MoonshotAI/Kimi-K3. As one markdown file: README.md.

Kimi K3

Chat Homepage
Hugging Face Twitter Follow Discord ModelScope
License

📰  Tech Blog |     📄  Full Report

1. Model Introduction

Kimi K3 is an open-weight, native multimodal agentic model and our most capable model to date. It is a 2.8T-parameter model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

Key Features

2. Model Summary

Architecture Mixture-of-Experts (MoE)
Total Parameters 2.8T
Activated Parameters 104B
Number of Layers 93
Number of Dense Layers 1
Attention-Layer Composition 69 KDA + 24 Gated MLA
Attention Hidden Dimension 7168
Number of Attention Heads 96
Latent MoE Dimension 3584
MoE Hidden Dimension (per Expert) 3072
Number of Experts 896
Selected Experts per Token 16
Number of Shared Experts 2
Vocabulary Size 160K
Context Length 1048576
Attention Mechanism KDA & Gated MLA
Activation Function SiTU-GLU
Vision Encoder MoonViT-V2
Parameters of Vision Encoder 401M
Quantization MXFP4 weights / MXFP8 activations
(quantization-aware training)
Modality Text, Image

3. Evaluation Results

Benchmark Kimi K3
(max)
Claude Fable 5
(max, w/ fallback)
GPT-5.6 Sol
(max)
Claude Opus 4.8
(max)
GPT-5.5
(xhigh)
GLM-5.2
(max)
Reasoning & Knowledge
GPQA Diamond 93.5 92.6 94.1 91.0 93.5 91.2
CritPt 23.4 28.6 32.3 20.9 27.1 20.9
AA-LCR 74.7 70.0 73.7 67.7 74.3 71.3
HLE-Full 43.5 / 56.0 53.3 / 63.0 44.5 / 58.0 49.8 / 57.9 41.4 / 52.2 —
Coding
DeepSWE 67.5 70.0 73.0 59.0 67.0 46.2
ProgramBench 77.8 76.8 77.6 71.9 70.8 63.7
Terminal-Bench 2.1 88.3 88.0 88.8 84.6 83.4 82.7
FrontierSWE 81.2 86.6 71.3 66.7 64.9 67.3
SWE-Marathon 42.0 35.0 39.0 40.0 14.0 13.0
PostTrainBench 36.6 41.4 34.6 34.1 28.4 34.3
MLS-Bench-Lite 48.3 49.9 46.2 42.8 35.5 40.4
SciCode 58.7 60.2 56.1 53.5 56.1 50.5
Kimi Code Bench 2.0 72.9 76.9 64.8 71.7 69.0 64.2
Agentic
BrowseComp 91.2 88.0 90.4 84.3 84.4 —
DeepSearchQA (F1) 95.0 94.2 — 93.1 — —
ResearchRubrics 76.2 — 73.8 73.5 64.0 71.1
GDPval-AA v2 (Elo) 1686 1747 1736 1593 1491 1510
Toolathlon-Verified 76.5 77.9 74.9 76.2 73.5 59.9
MCPMark-Verified 94.5 87.4 92.9 76.4 92.9 —
MCP-Atlas

Kimi-K3 (MoonshotAI/Kimi-K3)

Searching Database…

8.9K ·GitHub
这是什么?Kimi K3 是 MoonshotAI 的开放权重、原生多模态智能体模型——一个 2.8T 参数的混合专家(Mixture-of-Experts)系统,为前沿水平的长程编程、智能体知识工作和推理而构建。
这是可以直接使用的托管聊天机器人,还是必须自己运行的东西?它发布的是原始模型权重,不是托管产品——你需要在 Kimi K3 License 条款下自行下载并部署这些权重。
运行完整模型到底需要什么硬件?得益于稀疏 MoE 设计,每个 token 只激活 2.8T 总参数中的 104B;权重采用 MXFP4/MXFP8 量化感知训练来压缩体积,但要部署这么大的模型,你仍然需要多 GPU 基础设施。
这个模型到底有多稀疏——它会一次用上所有专家吗?不会,它通过 Stable LatentMoE 框架每个 token 只激活 896 个专家中的 16 个,这正是它尽管参数量高达 2.8T、有效计算量却仍可管理的原因。
Kimi Delta Attention(KDA)是什么?KDA 是 K3 采用的全新注意力机制,用在其 93 层中的 69 层,其余层使用 Gated MLA,并配合注意力残差(Attention Residuals,AttnRes)来提升扩展效率。
K3 比它的前代 Kimi K2 强多少?MoonshotAI 报告称,得益于新的 Stable LatentMoE 稀疏设计,整体扩展效率比 Kimi K2 提升了约 2.5 倍。
它能理解图像和视频,还是只能处理文本?K3 是原生多模态的:它在同一个模型内处理文本、图像和视频,使用 401M 参数的 MoonViT-V2 视觉编码器,并拥有 100 万 token 的上下文窗口。
在编程基准测试上它与 Claude 和 GPT 相比如何?在 Terminal-Bench 2.1 上它拿到 88.3 分,领先于 Claude Opus 4.8(84.6)和 GPT-5.5(83.4);在所对比的前沿模型中,它还以 42.0 分领跑 SWE-Marathon。
Creator
MoonshotAI
Size
1.4 MB
Stars
8.9K
Updated
CommitAug 06
Age
2mo
Commits · 90 Days