Machine Learning Systems
Principles and Practices of Engineering Artificially Intelligent Systems
English •
中文 •
日本語 •
한국어
Mission
The world is rushing to build AI systems. It is not engineering them.
That gap is what we mean by AI engineering.
AI engineering is the discipline of building efficient, reliable, safe, and robust intelligent systems that operate in the real world, not just models in isolation. Our mission is to establish AI engineering as a foundational discipline alongside software engineering and computer engineering, by teaching how to design, build, and evaluate end-to-end intelligent systems.
Our goal: Help 100,000 learners master ML Systems this year, and reach 1 million by 2030.
Why One Repository
I designed this as a single integrated curriculum, not a collection of independent projects. The textbook teaches the theory. TinyTorch makes you build the internals. The hardware kits force you to confront real constraints. The simulator lets you reason about infrastructure you can't afford to rent. Each piece exists because I found that students who only read don't internalize, and students who only code don't generalize.
The repository is the curriculum.
A growing community of contributors helps improve every part of it: fixing errors, sharpening explanations, testing on new hardware. Their work makes this better for everyone, and I'm grateful for every pull request.
The Curriculum
Every component connects. The textbook gives you the mental models. The labs let you reason through trade-offs interactively, powered by MLSys·im — a modeling engine for infrastructure you can't physically access, and a standalone tool in its own right. TinyTorch makes you build the machinery yourself. The hardware kits put you face-to-face with real deployment constraints. StaffML tests whether you actually understand it. Socratiq adds AI-guided reading, contextual quizzes, and spaced repetition inside the learning experience. And the instructor hub, slides, and newsletter give educators everything they need to bring this into a classroom.
For Students
|
Component |
Role in the Curriculum |
Link |
| 📖 |
Textbook Series |
Four-volume MIT Press textbook series on Machine Learning Systems Architecture:
• Vol I: Introduction to Machine Learning Systems (Released)
The foundations, abstractions, lifecycle, data, training, inference, evaluation, deployment, and responsible systems.
• Vol II: Scaling Machine Learning Systems (Preview)
Distributed computation, accelerators, memory, communication, parallelism, serving, reliability, and efficiency.
• Vol III: Agentic Machine Learning Systems (In Development)
Reasoning and acting loops, memory, tools, planning, search, orchestration, evaluation, security, and multi-agent systems.
• Vol IV: Physical AI Systems (In Development)
Sensing, perception, world models, control, robotics, embodiment, real-time constraints, safety, and hardware.
⚠️ Volumes III and IV are in development and change quickly as I iterate. Please do not cite or teach from them yet. Feedback is welcome through the Book feedback issue forms.
|
Vol I · Vol II (Preview) · Vol III (In Dev) · Vol IV (In Dev) |
| 🔬 |
Labs |
Interactive Marimo notebooks where you explore trade-offs from the textbook: change a parameter, see what breaks, build intuition. Powered by MLSys·im under the hood. |
Launch labs · Repo guide |
| 🔥 |
Tiny🔥Torch |
Build your own ML framework from scratch across 20 progressive modules. You don't understand a system until you've built one. |
Get started |
| 🛠️ |
Hardware Kits |
Deploy ML to Arduino, Seeed, Grove, and Raspberry Pi devices. Real memory limits, real power budgets, real latency. |
Browse labs |
| 🔮 |
MLSys·im |
Calculate memory bottlenecks, network saturation, and scheduling limits at infrastructure scales you can't physically access. |
Use simulator · Repo guide |
| 💼 |
StaffML |
Physics-grounded interview questions for ML systems roles. Vault, practice drills, mock interviews, and progress tracking. |
Practice · Repo guide |
For Educators
|
Component |
What It Provides |
Link |
| 🎓 |
Instructor Hub |
The AI Engineering Blueprint: two 16-week syllabi, pedagogy guide, assessment rubrics, and a TA handbook. |
View hub · Repo guide |
| 🎬 |
Lecture Slides |
Beamer slide decks for every chapter, with four theme variants. Drop into your course and teach. |
Browse decks · Repo guide |
| 📬 |
Newsletter |
Updates on the curriculum, new chapters, and what the community is building. |
Subscribe |
Choose Your Path
The pieces are designed to work together, but you do not need to adopt everything at once.
| If you are... |
Start here |
Then go deeper |
| A student or self-learner |
Read Volume I and try Lab 00 |
Build TinyTorch, use MLSys·im, and practice with StaffML |
| An instructor |
Open The AI Engineering Blueprint |
Use the course map, slides, rubrics, and TA guide |
| A contributor |
Pick the component you use most |
Improve chapters, labs, tests, examples, hardware notes, simulator models, or assessment content |
The learning loop is: Read → Explore → Build → Model → Deploy → Practice → Teach.
Adjacent and Experimental Work
Some projects are intentionally earlier-stage than the main curriculum:
- Socratiq explores AI-guided reading, contextual quizzes, and spaced repetition for static learning sites.
- MLPerf EDU is an under-construction pedagogical benchmark suite aligned with MLCommons MLPerf.
What You Will Learn
This textbook teaches you to think at the intersection of machine learning and systems engineering. Each chapter bridges algorithmic concepts with the infrastructure that makes them work in practice.
| You know... |
|
You will learn... |
| How to train a model |
→ |
How training scales across GPU clusters |
| That quantization shrinks models |
→ |
How INT8 math maps to silicon |
| What a transformer is |
→ |
Why KV-cache dominates memory at inference |
| Models run on GPUs |
→ |
How schedulers balance latency vs throughput |
| Edge devices have limits |
→ |
How to co-design models and hardware |
Book Structure
The textbook series follows a comprehensive systems progression across four core volumes:
|
Volume |
Unit of Work |
Core Systems Question |
Consequence of Failure |
| 📗 |
Volume I
Foundations |
The Model |
How do we make intelligence execute efficiently on a single node? |
Bad prediction or degraded runtime efficiency. |
| 📘 |
Volume II
Scaling (Preview) |
The Fleet |
How do we scale intelligence across distributed clusters and datacenters? |
Multi-million dollar cluster stall or service outage. |
| 🟣 |
Volume III
Agentic (In Dev) |
The Trajectory |
How do we govern intelligence acting autonomously over long horizons? |
Compounding trajectory drift and unauthorized side-effects. |
| 🌲 |
Volume IV
Physical AI (In Dev) |
The Physical Plant |
How do we let intelligence act safely on matter and physical systems? |
Irreversible physical damage in the real world. |
Note: Volumes III and IV are in development. They change quickly as I iterate, so expect chapters to be added, reorganized, and rewritten. You are welcome to follow along as they take shape.
How the Volumes Connect
Each volume in the series expands the systems boundary, answering the question that the previous level leaves open:
- From Model to Fleet (Vol I → Vol II): When one machine is not enough. Volume I masters single-node execution, memory walls, and kernel efficiency. Volume II scales those foundations across thousands of accelerators, collective communication fabrics, fault-tolerant networks, and datacenter orchestration.
- From Request to Trajectory (Vol II → Vol III): When a stateless response is not enough. Volume II scales stateless request-response inference. Volume III introduces stateful, multi-step autonomous loops—managing context memory hierarchies, tool execution protocols (MCP), isolation sandboxes, and non-deterministic recovery.
- From Cyberspace to Matter (Vol III → Vol IV): When software acts on the physical world. Volume III governs digital tools and software environments. Volume IV crosses the causal boundary into physical plants—governing real-time sensor-actuator loops where computational latency becomes uncontrolled distance, reflected inertia dictates motion, and failure has irreversible physical consequences.
FAQ
Why do you do all this?
I consider myself a student of my own learning.
Whenever I want to understand an emerging or complex systems discipline, I learn it by trying to teach and build it from first principles. Writing forces me to slow down: to identify invariants, formalize abstractions, derive quantitative limits, and separate durable engineering foundations from passing industry hype.
This repository—the books, software frameworks, and hardware kits—is my open research and teaching instrument. It is how I clarify my own thinking for research, benchmarking programs (like MLPerf), and classroom instruction. If sharing this open process helps other engineers and students along the way, you are more than welcome to follow along.
Who is this for, and what should I know first?
This is for anyone who wants to engineer intelligent systems, not only train models: students, working engineers moving into ML infrastructure, and educators building a course. We assume you can program in Python and have met basic machine learning ideas, but the book builds the systems concepts from the ground up. You do not need a background in computer architecture, distributed systems, or datacenter operations. Volume I starts at the foundations, and the rest of the curriculum (TinyTorch, labs, hardware kits, and the simulator) lets you learn by building rather than only by reading.
Do I need Volume I before Volume II? What is the difference?
The two volumes differ in scope, not depth. Both are equally rigorous. Volume I is the single-machine world: how an ML system works on one node with a handful of accelerators, from data and a single neuro
一套关于机器学习系统（也就是 AI 工程）的开源教科书和完整课程体系，来自哈佛 CS249r 课程：共四卷，涵盖基础、规模化、智能体 AI 和具身 AI，还配有实验、项目和教师教学材料。