30u30 README

项目的 README,来自 github.com/Ojhaharsh/30u30。合并为一个 markdown 文件:README.md。

30u30: Master Ilya's 30 Foundational AI Papers in 30 Days

License: CC BY-NC-ND 4.0 Python 3.8+ GitHub stars Website

"If you really learn all of these, you'll know 90% of what matters today" - Ilya Sutskever

Complete implementations • Interactive notebooks • Beginner-friendly • 100% Free

30u30 is an open-source study guide that takes you through the 30 foundational AI papers recommended by Ilya Sutskever — one paper per day. Each paper comes with a full implementation from scratch, detailed notes, interactive exercises with solutions, and visualizations.
Think of it as Rustlings, but for deep learning fundamentals. You read the paper, understand the math, and build it yourself.


🚀 All units are complete! Bonus papers are coming soon!!!

✅ Unit 1: The Foundations (Days 1-7) - COMPLETE!

Day 1: "The Unreasonable Effectiveness of Recurrent Neural Networks"

Day 2: "Understanding LSTM Networks"

Day 3: "RNN Regularization"

Day 4: "Minimizing Description Length"

Day 5: "MDL Principle Tutorial"

Day 6: "The First Law of Complexodynamics"

Day 7: "The Coffee Automaton"

✅ Unit 2: Deep Learning Explosion (Vision & Architectures) - COMPLETE!

Day 8: "ImageNet Classification with Deep CNNs (AlexNet)"

Day 9: "Deep Residual Learning for Image Recognition (ResNet)"

Day 10: "Identity Mappings in Deep Residual Networks (ResNet v2)"

Day 11: "Multi-Scale Context Aggregation by Dilated Convolutions"

Day 12: "Dropout: A Simple Way to Prevent Overfitting"

✅ Unit 3: The Transformer Era (Days 13-16) - COMPLETE!

Day 13: "Attention Is All You Need"

Day 14: "The Annotated Transformer"

Day 15: "Neural Machine Translation by Jointly Learning to Align and Translate"

Day 16: "Order Matters: Sequence to Sequence for Sets"

✅ Unit 4: Specialized Architectures (Days 17-22) - COMPLETE!

Day 17: "Neural Turing Machines"

Day 18: "Pointer Networks"

Day 19: "Relational Reasoning"

Day 20: "Relational Recurrent Neural Networks"

Day 21: "Neural Message Passing for Quantum Chemistry"

Day 22: "Deep Speech 2: End-to-End Speech Recognition"

✅ Unit 5: Generative Models & Scaling (Days 23-28) - COMPLETE!

Day 23: "Variational Lossy Autoencoder"

Day 24: "GPipe: Efficient Training of Giant Neural Networks"

Day 25: "Scaling Laws for Neural Language Models"

Day 26: "Kolmogorov Complexity and Algorithmic Randomness"

Day 27: "Machine Super Intelligence (Shane Legg)"

Day 28: "CS231n: CNNs for Visual Recognition"

✅ Unit 6: Deep Reinforcement Learning & Alignment (Days 29-30) - COMPLETE!

Day 29: "Proximal Policy Optimization (PPO)"

Day 30: "Deep Reinforcement Learning from Human Feedback (RLHF)"


🎯 Mission

This is the most comprehensive, beginner-friendly, open-source journey through the papers that defined modern AI. No paywalls. No gatekeeping. Just pure knowledge.

Whether you're pivoting to AI, a student, or a curious mind - this is your roadmap.

📚 What You'll Find Here

Each paper gets the full treatment:

🗺️ The Journey

Unit 1: The Foundations (Days 1-7) - ✅ COMPLETE

Day Paper Status Core Concept
1 The Unreasonable Effectiveness of RNNs 🚀 LIVE Why predicting = intelligence
2 Understanding LSTM Networks 🚀 LIVE The mechanics of memory
3 RNN Regularization 🚀 LIVE Making RNNs generalize
4 Minimizing Description Length 🚀 LIVE Compression = Intelligence
5 MDL Principle Tutorial 🚀 LIVE Math of compression
6 The First Law of Complexodynamics 🚀 LIVE Physics of complexity
7 The Coffee Automaton 🚀 LIVE Why intelligence exists

Unit 2: Deep Learning Explosion (Days 8-12) - ✅ COMPLETE

Vision, depth, and the techniques that changed everything

Day Paper Status Core Concept
8 ImageNet Classification (AlexNet) 🚀 LIVE Deep learning revolution
9 Deep Residual Learning (ResNet) 🚀 LIVE Skip connections
10 Identity Mappings in ResNets 🚀 LIVE Pre-activation design
11 Multi-Scale Context (Dilated Conv) 🚀 LIVE Dilated convolutions
12 Dropout (Srivastava et al.) 🚀 LIVE Preventing overfitting

Unit 3: The Transformer Era (Days 13-16) - ✅ COMPLETE

The architecture that ate the world

Day Paper Status Core Concept
13 Attention Is All You Need 🚀 LIVE Self-attention, Transformer
14 The Annotated Transformer 🚀 LIVE Code-level Transformer
15 Bahdanau Attention (NMT) 🚀 LIVE Original attention mechanism
16 Order Matters (Pointer Networks) 🚀 LIVE Set-to-sequence problems

✅ Unit 4: Specialized Architectures (Days 17-22) - COMPLETE

Memory, graphs, and reasoning

Day Paper Status Core Concept
17 Neural Turing Machines 🚀 LIVE Differentiable external memory
18 Pointer Networks 🚀 LIVE Selecting input via attention
19 Relational Reasoning 🚀 LIVE Pairwise object relations; g_theta & f_phi modules
20 Relational RNNs 🚀 LIVE Self-attention inside recurrence
21 Neural Message Passing 🚀 LIVE MPNN framework for graph neural networks
22 Deep Speech 2 🚀 LIVE End-to-end speech recognition with CTC

✅ Unit 5: Generative Models & Scaling (Days 23-28) - COMPLETE

From theory to massive models

Day Paper Status Core Concept
23 Variational Lossy Autoencoder 🚀 LIVE Curing posterior collapse with IAF
24 GPipe: Efficient Training of Giant Neural Networks 🚀 LIVE Pipeline parallelism
25 Scaling Laws for Neural Language Models 🚀 LIVE The physics of AI scaling
26 Kolmogorov Complexity 🚀 LIVE Math of compression & randomness
27 Machine Super Intelligence 🚀 LIVE Safety & intelligence definitions
28 CS231n: CNNs for Visual Recognition 🚀 LIVE CNN layers from scratch, VGGNet analysis

✅ Unit 6: Deep Reinforcement Learning & Alignment (Days 29-30) - COMPLETE

From Policy Gradients to RLHF

Day Paper Status Core Concept
29 Proximal Policy Optimization (PPO) 🚀 LIVE The algorithm behind ChatGPT (RLHF)
30 Deep Reinforcement Learning from Human Feedback 🚀 LIVE The birth of "Human Feedback" (RLHF)

💻⏭️ Bonus Papers: The Language Model Revolution (Coming Soon)

The modern era of LLMs

Paper Core Concept
BERT: Pre-training of Deep Bidirectional Transformers Coming Soon
GPT-2: Language Models are Unsupervised Multitask Learners Coming Soon
GPT-3: Language Models are Few-Shot Learners Coming Soon
Chinchilla: Training Compute-Optimal Large Language Models Coming Soon

Complete paper list with links →

⚡ Quick Start

# Clone the repo
git clone https://github.com/yourusername/30u30.git
cd 30u30

# Start with Day 1
cd papers/01_Unreasonable_Effectiveness

Prerequisites

For beginners: Basic Python knowledge. We'll teach you the rest.

For practitioners: Jump to any paper that interests you.

🎓 How to Use This Repo

🎯 The 30-Day Challenge:

🔀 Choose Your Path:

🌟 Why This Project?

There are many paper summaries online. But this is different:

  1. You build everything - No "import magic_ai_library"
  2. Multiple learning paths - Theory-first, code-first, or interactive
  3. Production-quality - Code that actually works and teaches
  4. Beginner-friendly - Real-world analogies + rigorous math
  5. Community-driven - Your feedback shapes future days

Goal: The best free resource for learning AI fundamentals.

If this helps you, ⭐ star the repo and share it with others!


🤝 Contributing

We'd love your help making this better!

Every contribution helps thousands of learners.

📜 License

CC BY-NC-ND 4.0 — Free to read, learn, and share with attribution. Not for commercial use.

🙏 Acknowledgments


💬 Stay Connected


Ready to start?
→ Day 1: Character-Level RNN
→ Day 2: Understanding LSTMs
→ Day 3: RNN Regularization
→ Day 4: Minimizing Description Length
→ Day 5: MDL Principle Tutorial
→ Day 6: The First Law of Complexodynamics
→ Day 7: The Coffee Automaton
→ Day 8: ImageNet Classification (AlexNet)
→ Day 9: Deep Residual Learning (ResNet)
→ Day 10: Identity Mappings (ResNet v2)
→ Day 11: Dilated Convolutions
→ Day 12: Dropout
→ Day 13: Attention Is All You Need
→ Day 14: The Annotated Transformer
→ Day 15: Bahdanau Attention (NMT)
→ Day 16: Order Matters (Pointer Networks)
→ Day 17: Neural Turing Machines
→ Day 18: Pointer Networks
→ Day 19: Relational Reasoning
→ Day 20: Relational RNNs
→ Day 21: Neural Message Passing
→ Day 22: Deep Speech 2
→ Day 23: Variational Lossy Autoencoder
→ Day 24: GPipe (Giant Neural Networks)
→ Day 25: Scaling Laws for Neural Language Models
→ Day 26: Kolmogorov Complexity
→ Day 27: Machine Super Intelligence
→ Day 28: CS231n — CNNs for Visual Recognition
→ Day 29: Proximal Policy Optimization (PPO)
→ Day 30: RLHF 🆕 ← Start here!

Let's build something amazing together! 🚀

30u30 (Ojhaharsh/30u30)

正在搜索数据库…

1.8K ·GitHub
这是什么?30u30 是一份免费的开源学习指南,讲解 Ilya Sutskever 推荐的 30 篇 AI 奠基性论文,每天学习一篇,每篇都配有从零开始的完整实现、笔记和练习。
这只是一份阅读清单,还是我真的要写代码?这是一门动手编程的课程,而不只是链接列表:每篇论文都附带从零开始的实现、5 道带答案的练习、可视化内容,以及可以边学边做的 Jupyter 笔记本。
我该怎么开始学这门课程?从 Day 1 开始,按顺序学完 30 个每日文件夹即可;每个文件夹都有 README、笔记本、练习,以及一个可以立即运行的精简快速入门训练脚本。
这30天的课程真的全部完成了吗?是的,目前涵盖全部 30 天的 6 个单元都已完成;作者目前只计划在未来新增一些额外的加餐论文。
这门课程是给谁准备的?它面向转行进入 AI 领域的初学者、学生,以及希望亲手实现经典论文、而不只是阅读介绍的好奇自学者。
每天的文件夹是怎么组织的?每天都包含一份深入的 README、“ELI5”笔记、一份清晰且带注释的实现、可视化内容、5 道带答案的练习,以及一个可交互的 Jupyter 笔记本。
这门课程涵盖哪些主题?课程分为六个单元:RNN/LSTM 与复杂度基础,AlexNet、ResNet 等 CNN 架构,Transformer 时代,专用架构(神经图灵机、图网络、Deep Speech 2),生成模型与缩放定律,最后以 PPO 和 RLHF 收尾。
为什么要拿它和 Rustlings 相比?就像 Rustlings 帮助学习 Rust 一样,30u30 要求你先读懂理论,然后通过有指导的练习亲手实现每个概念,而不是被动地阅读论文摘要。
使用这门课程需要花钱吗?不需要,该项目宣传自己为“100% 免费”,30 天的所有内容都没有付费墙或门槛限制。
作者
Ojhaharsh
大小
3 MB
星标
1.8K
更新
提交Feb 19
时长
9个月
提交 · 90 天