19.5K ·GitHub
What is this?
An open book on machine learning engineering: methodologies, tools and step-by-step instructions for training, fine-tuning and running inference on large language models and multi-modal (VLM) models, written by the engineer who led the BLOOM-176B training.
An open book on machine learning engineering: methodologies, tools and step-by-step instructions for training, fine-tuning and running inference on large language models and multi-modal (VLM) models, written by the engineer who led the BLOOM-176B training.Is it a book I read, or software I install?It is a book rather than a package: you read it free on GitHub or download PDF and EPUB versions, and the text is filled with scripts and copy-n-paste commands you run on your own machines.
It is a book rather than a package: you read it free on GitHub or download PDF and EPUB versions, and the text is filled with scripts and copy-n-paste commands you run on your own machines.How do I get started with it?Just read it on GitHub, or grab the PDF/EPUB from the HuggingFace hub. There is also a companion course, 'Lessons Learned from Training LLMs', that goes over the terse insights first and lets you dive deeper where needed.
Just read it on GitHub, or grab the PDF/EPUB from the HuggingFace hub. There is also a companion course, 'Lessons Learned from Training LLMs', that goes over the terse insights first and lets you dive deeper where needed.Do I need my own GPU cluster to get anything out of it?Reading is free, but the hands-on parts — tools like torch-distributed-gpu-test.py and the network benchmarks — are meant to run on the GPU machines you train on, so a laptop alone won't exercise most of it.
Reading is free, but the hands-on parts — tools like torch-distributed-gpu-test.py and the network benchmarks — are meant to run on the GPU machines you train on, so a laptop alone won't exercise most of it.Is this a finished, polished textbook?No — the author calls it an ongoing brain dump that keeps growing, with ebooks rebuilt only every few weeks and updates announced on his Twitter channel, so some sections are rougher than a curated textbook.
No — the author calls it an ongoing brain dump that keeps growing, with ebooks rebuilt only every few weeks and updates announced on his Twitter channel, so some sections are rougher than a curated textbook.Is this for beginners or for people already running big trainings?For working LLM/VLM training engineers and operators rather than ML newcomers — it assumes you are dealing with real GPU jobs and shares hard-won recipes from training BLOOM-176B and IDEFICS-80B, not introductory theory.
For working LLM/VLM training engineers and operators rather than ML newcomers — it assumes you are dealing with real GPU jobs and shares hard-won recipes from training BLOOM-176B and IDEFICS-80B, not introductory theory.Can I reuse or redistribute the content?Yes, under CC BY-SA 4.0: share and adapt it as long as you attribute the author and share alike — typical for an open book, but different from the permissive code licenses you may be used to.
Yes, under CC BY-SA 4.0: share and adapt it as long as you attribute the author and share alike — typical for an open book, but different from the permissive code licenses you may be used to.What does the book actually cover?Seven parts: insights (cloud provider choice, when to upgrade GPUs), hardware (compute, storage, network), orchestration including SLURM, training, inference, debugging and testing, plus a resources chapter.
Seven parts: insights (cloud provider choice, when to upgrade GPUs), hardware (compute, storage, network), orchestration including SLURM, training, inference, debugging and testing, plus a resources chapter.Does it come with ready-to-run tools?Yes: allreducebench.py for benchmarking network throughput more easily than nccl-tests, torch-distributed-gpu-test.py for checking inter-node connectivity, and mamf-finder.py for measuring the TFLOPS you can actually get, plus PyTorch debugging and SLURM cheat sheets.
Yes: allreducebench.py for benchmarking network throughput more easily than nccl-tests, torch-distributed-gpu-test.py for checking inter-node connectivity, and mamf-finder.py for measuring the TFLOPS you can actually get, plus PyTorch debugging and SLURM cheat sheets.Can I use it to teach my AI coding agent?Yes — the repo ships a SKILL.md meant to teach AI agents to train and operate large-scale ML models better, alongside companion skills: The Art of Debugging and Stas' Python Cookbook.
Yes — the repo ships a SKILL.md meant to teach AI agents to train and operate large-scale ML models better, alongside companion skills: The Art of Debugging and Stas' Python Cookbook.Will it help me decide what hardware to rent or buy?It maintains key comparison tables of accelerator TFLOPS, memory size and speed, and inter- and intra-node network speeds, and works through a real H200 to B200 benchmark to decide when a GPU upgrade is worth its cost.
It maintains key comparison tables of accelerator TFLOPS, memory size and speed, and inter- and intra-node network speeds, and works through a real H200 to B200 benchmark to decide when a GPU upgrade is worth its cost.What are similar projects?verl, ColossalAI, transformers, halo, skypilot, airllm, EvoVLM, ms-swift
verl, ColossalAI, transformers, halo, skypilot, airllm, EvoVLM, ms-swiftWhich topics is it in?hpc, courses, llm-training, vlm, distributed-training, skill-md, mlops, network, scalability, cheatsheet, storage, training
hpc, courses, llm-training, vlm, distributed-training, skill-md, mlops, network, scalability, cheatsheet, storage, training