What is this?
Happy-LLM is Datawhale's free open-source LLM course: a systematic tutorial that goes from NLP basics and the Transformer architecture to building and training your first large language model, then applying it with RAG and agents.
Happy-LLM is Datawhale's free open-source LLM course: a systematic tutorial that goes from NLP basics and the Transformer architecture to building and training your first large language model, then applying it with RAG and agents.Is this software I install, or a course I read?A course you read: eight chapters of tutorial text with companion Python/Jupyter code to reproduce, plus a free PDF edition (lightly watermarked to deter resellers) and PPT slides available in Releases.
A course you read: eight chapters of tutorial text with companion Python/Jupyter code to reproduce, plus a free PDF edition (lightly watermarked to deter resellers) and PPT slides available in Releases.How do I set up to run the chapter code?Dependencies are split per chapter, so the repo recommends a separate Python environment per chapter to avoid version conflicts; start with the Learning and Environment Preparation chapter, which also gives hardware advice.
Dependencies are split per chapter, so the repo recommends a separate Python environment per chapter to avoid version conflicts; start with the Learning and Environment Preparation chapter, which also gives hardware advice.Which frameworks does the hands-on part use?Chapter 5 works in plain PyTorch, then chapter 6 switches to the mainstream Transformers framework for pretraining, supervised fine-tuning and efficient LoRA/QLoRA (PEFT) fine-tuning; chapter 7 adds evaluation, RAG and agents.
Chapter 5 works in plain PyTorch, then chapter 6 switches to the mainstream Transformers framework for pretraining, supervised fine-tuning and efficient LoRA/QLoRA (PEFT) fine-tuning; chapter 7 adds evaluation, RAG and agents.Does the material keep up with how fast LLMs move?All eight chapters are complete, and chapter 8 already covers recent Agentic-RL methods (GRPO, OPD, Search-R1, ReTool); for newer algorithms the authors point readers to their faster-updating agentic-rl-lab repo.
All eight chapters are complete, and chapter 8 already covers recent Agentic-RL methods (GRPO, OPD, Search-R1, ReTool); for newer algorithms the authors point readers to their faster-updating agentic-rl-lab repo.Do I need prior ML experience to follow it?It targets students, researchers and LLM enthusiasts; you should have some programming experience, ideally Python, and deep-learning knowledge plus NLP terminology will make it much smoother.
It targets students, researchers and LLM enthusiasts; you should have some programming experience, ideally Python, and deep-learning knowledge plus NLP terminology will make it much smoother.Can I reuse the material in my own course or product?Not commercially: the work is licensed CC BY-NC-SA 4.0, so sharing and adapting with attribution under the same license is fine, but commercial use is excluded. GitHub lists it as NOASSERTION.
Not commercially: the work is licensed CC BY-NC-SA 4.0, so sharing and adapting with attribution under the same license is fine, but commercial use is excluded. GitHub lists it as NOASSERTION.Will I need a beefy GPU to follow along?Specific requirements are not listed here, but the Learning and Environment Preparation chapter provides per-chapter dependencies and hardware suggestions for reproducing the code.
Specific requirements are not listed here, but the Learning and Environment Preparation chapter provides per-chapter dependencies and hardware suggestions for reproducing the code.Why did Datawhale create this course?After their self-llm guide on using open-source LLMs, many readers wanted to go deeper, so Datawhale built Happy-LLM to explain the principles and training process behind large language models rather than just how to use them.
After their self-llm guide on using open-source LLMs, many readers wanted to go deeper, so Datawhale built Happy-LLM to explain the principles and training process behind large language models rather than just how to use them.Can I try or download the model built in the book?Yes. The chapter 5 checkpoints — Happy-LLM-Chapter5-Base-215M and Happy-LLM-Chapter5-SFT-215M — are downloadable from ModelScope, and there is an online ModelScope demo space to try them in the browser.
Yes. The chapter 5 checkpoints — Happy-LLM-Chapter5-Base-215M and Happy-LLM-Chapter5-SFT-215M — are downloadable from ModelScope, and there is an online ModelScope demo space to try them in the browser.Is the tutorial available in English?Yes. The README links both a Chinese and an English edition, so you can follow the course without reading Chinese.
Yes. The README links both a Chinese and an English edition, so you can follow the course without reading Chinese.What are similar projects?halo, train-llm-from-scratch, ms-swift, autumn-recruitment, nlp_course, minimind, Soup, jeeves
halo, train-llm-from-scratch, ms-swift, autumn-recruitment, nlp_course, minimind, Soup, jeevesWhich topics is it in?peft, qlora, sft, grpo, attention-mechanism, lora, transformers, reinforcement-learning, jupyter notebook, fine-tuning, transformer, nlp
peft, qlora, sft, grpo, attention-mechanism, lora, transformers, reinforcement-learning, jupyter notebook, fine-tuning, transformer, nlp