6.7K ·GitHub
What is this?
An open-source book, 'Understanding AI Infra: Quantitative Analysis and System Design' by Bojie Li, that derives LLM inference and training system designs from hardware constraints and model architecture using quantitative, order-of-magnitude estimates.
An open-source book, 'Understanding AI Infra: Quantitative Analysis and System Design' by Bojie Li, that derives LLM inference and training system designs from hardware constraints and model architecture using quantitative, order-of-magnitude estimates.Is this a book I read, or a software project I run?Mostly a book: twelve chapters with figures, published as PDF, EPUB and an online reading site. The repo also carries Python calculation tools and per-chapter experiments, which you can run but do not have to.
Mostly a book: twelve chapters with figures, published as PDF, EPUB and an online reading site. The repo also carries Python calculation tools and per-chapter experiments, which you can run but do not have to.How do I read it: online, PDF or EPUB?All three, in Simplified Chinese (the original), English and Traditional Chinese. The online reading site adds chapter navigation, full-text search and dark mode, and every update to main auto-builds all formats and publishes them to Releases.
All three, in Simplified Chinese (the original), English and Traditional Chinese. The online reading site adds chapter navigation, full-text search and dark mode, and every update to main auto-builds all formats and publishes them to Releases.Is the book finished, or still being written?It is still a first draft under continuous revision. The English and Traditional Chinese editions are community translations that may lag, so numbers, formulas and citations should follow the Simplified Chinese original.
It is still a first draft under continuous revision. The English and Traditional Chinese editions are community translations that may lag, so numbers, formulas and citations should follow the Simplified Chinese original.Who is it written for, and what background do I need?Engineers, researchers and students who have called a model API or run models on their own machines and want to understand why models are slow and how to cut costs. Some Python, linear algebra and computer-systems basics are assumed.
Engineers, researchers and students who have called a model API or run models on their own machines and want to understand why models are slow and how to cut costs. Some Python, linear algebra and computer-systems basics are assumed.Can I reuse the text, figures or code in my own work?Yes - the original text, figures and companion code are Apache License 2.0, copyright 2026 Bojie Li. Third-party code, fonts, templates and reference materials bundled in the repo keep their own licenses.
Yes - the original text, figures and companion code are Apache License 2.0, copyright 2026 Bojie Li. Third-party code, fonts, templates and reference materials bundled in the repo keep their own licenses.Why did the author write this book?After finishing 'Understanding AI Agent' (45k+ stars on GitHub), he kept feeling that building model apps also requires understanding the infrastructure they run on, just as most engineers study OS and computer architecture. No existing book derived AI Infra design quantitatively from hardware constraints and model architecture, so he wrote one.
After finishing 'Understanding AI Agent' (45k+ stars on GitHub), he kept feeling that building model apps also requires understanding the infrastructure they run on, just as most engineers study OS and computer architecture. No existing book derived AI Infra design quantitatively from hardware constraints and model architecture, so he wrote one.Is there one idea that ties the whole book together?Data movement. Whether the author was accelerating Bing ranking on FPGAs, generating Ascend AKG operators, or interconnecting 10,000 cards over UB, the same questions kept recurring: what data moves, how much, how many times, through where, and who must wait for it.
Data movement. Whether the author was accelerating Bing ranking on FPGAs, generating Ascend AKG operators, or interconnecting 10,000 cards over UB, the same questions kept recurring: what data moves, how much, how many times, through where, and who must wait for it.What can the companion calculation tools do?They recompute the numbers in the book and let you swap in different models and inputs to estimate resource needs. Static calculations need only the Python 3.10+ standard library - no GPU and no model weights required.
They recompute the numbers in the book and let you swap in different models and inputs to estimate resource needs. Static calculations need only the Python 3.10+ standard library - no GPU and no model weights required.How should I approach the worked examples in the book?Try to estimate each example yourself before reading the derivation and experimental results - then repeat the estimate with your own model and hardware to see whether the conclusions still hold.
Try to estimate each example yourself before reading the derivation and experimental results - then repeat the estimate with your own model and hardware to see whether the conclusions still hold.What are similar projects?pd-bridge, vllm, ai-system-design, LMCache, llm-action, sglang, ktransformers, openTPU
pd-bridge, vllm, ai-system-design, LMCache, llm-action, sglang, ktransformers, openTPUWhich topics is it in?accelerator, ai-infrastructure, kv-cache, llm-training, llm-serving, epub, mixture-of-experts, system-design, ebook, training, llm-inference, hardware
accelerator, ai-infrastructure, kv-cache, llm-training, llm-serving, epub, mixture-of-experts, system-design, ebook, training, llm-inference, hardware