The project's README from github.com/Ojhaharsh/30u30. As one markdown file: README.md.
"If you really learn all of these, you'll know 90% of what matters today" - Ilya Sutskever
Complete implementations • Interactive notebooks • Beginner-friendly • 100% Free
30u30 is an open-source study guide that takes you through the 30 foundational AI papers recommended by Ilya Sutskever — one paper per day.
Each paper comes with a full implementation from scratch, detailed notes, interactive exercises with solutions, and visualizations.
Think of it as Rustlings, but for deep learning fundamentals. You read the paper, understand the math, and build it yourself.
Day 1: "The Unreasonable Effectiveness of Recurrent Neural Networks"
Day 2: "Understanding LSTM Networks"
Day 3: "RNN Regularization"
Day 4: "Minimizing Description Length"
Day 5: "MDL Principle Tutorial"
Day 6: "The First Law of Complexodynamics"
Day 7: "The Coffee Automaton"
Day 8: "ImageNet Classification with Deep CNNs (AlexNet)"
Day 9: "Deep Residual Learning for Image Recognition (ResNet)"
Day 10: "Identity Mappings in Deep Residual Networks (ResNet v2)"
Day 11: "Multi-Scale Context Aggregation by Dilated Convolutions"
Day 12: "Dropout: A Simple Way to Prevent Overfitting"
Day 13: "Attention Is All You Need"
Day 14: "The Annotated Transformer"
Day 15: "Neural Machine Translation by Jointly Learning to Align and Translate"
Day 16: "Order Matters: Sequence to Sequence for Sets"
Day 17: "Neural Turing Machines"
Day 18: "Pointer Networks"
Day 19: "Relational Reasoning"
Day 20: "Relational Recurrent Neural Networks"
Day 21: "Neural Message Passing for Quantum Chemistry"
Day 22: "Deep Speech 2: End-to-End Speech Recognition"
Day 23: "Variational Lossy Autoencoder"
Day 24: "GPipe: Efficient Training of Giant Neural Networks"
Day 25: "Scaling Laws for Neural Language Models"
Day 26: "Kolmogorov Complexity and Algorithmic Randomness"
Day 27: "Machine Super Intelligence (Shane Legg)"
Day 28: "CS231n: CNNs for Visual Recognition"
Day 29: "Proximal Policy Optimization (PPO)"
Day 30: "Deep Reinforcement Learning from Human Feedback (RLHF)"
This is the most comprehensive, beginner-friendly, open-source journey through the papers that defined modern AI. No paywalls. No gatekeeping. Just pure knowledge.
Whether you're pivoting to AI, a student, or a curious mind - this is your roadmap.
Each paper gets the full treatment:
| Day | Paper | Status | Core Concept |
|---|---|---|---|
| 1 | The Unreasonable Effectiveness of RNNs | 🚀 LIVE | Why predicting = intelligence |
| 2 | Understanding LSTM Networks | 🚀 LIVE | The mechanics of memory |
| 3 | RNN Regularization | 🚀 LIVE | Making RNNs generalize |
| 4 | Minimizing Description Length | 🚀 LIVE | Compression = Intelligence |
| 5 | MDL Principle Tutorial | 🚀 LIVE | Math of compression |
| 6 | The First Law of Complexodynamics | 🚀 LIVE | Physics of complexity |
| 7 | The Coffee Automaton | 🚀 LIVE | Why intelligence exists |
Vision, depth, and the techniques that changed everything
| Day | Paper | Status | Core Concept |
|---|---|---|---|
| 8 | ImageNet Classification (AlexNet) | 🚀 LIVE | Deep learning revolution |
| 9 | Deep Residual Learning (ResNet) | 🚀 LIVE | Skip connections |
| 10 | Identity Mappings in ResNets | 🚀 LIVE | Pre-activation design |
| 11 | Multi-Scale Context (Dilated Conv) | 🚀 LIVE | Dilated convolutions |
| 12 | Dropout (Srivastava et al.) | 🚀 LIVE | Preventing overfitting |
The architecture that ate the world
| Day | Paper | Status | Core Concept |
|---|---|---|---|
| 13 | Attention Is All You Need | 🚀 LIVE | Self-attention, Transformer |
| 14 | The Annotated Transformer | 🚀 LIVE | Code-level Transformer |
| 15 | Bahdanau Attention (NMT) | 🚀 LIVE | Original attention mechanism |
| 16 | Order Matters (Pointer Networks) | 🚀 LIVE | Set-to-sequence problems |
Memory, graphs, and reasoning
| Day | Paper | Status | Core Concept |
|---|---|---|---|
| 17 | Neural Turing Machines | 🚀 LIVE | Differentiable external memory |
| 18 | Pointer Networks | 🚀 LIVE | Selecting input via attention |
| 19 | Relational Reasoning | 🚀 LIVE | Pairwise object relations; g_theta & f_phi modules |
| 20 | Relational RNNs | 🚀 LIVE | Self-attention inside recurrence |
| 21 | Neural Message Passing | 🚀 LIVE | MPNN framework for graph neural networks |
| 22 | Deep Speech 2 | 🚀 LIVE | End-to-end speech recognition with CTC |
From theory to massive models
| Day | Paper | Status | Core Concept |
|---|---|---|---|
| 23 | Variational Lossy Autoencoder | 🚀 LIVE | Curing posterior collapse with IAF |
| 24 | GPipe: Efficient Training of Giant Neural Networks | 🚀 LIVE | Pipeline parallelism |
| 25 | Scaling Laws for Neural Language Models | 🚀 LIVE | The physics of AI scaling |
| 26 | Kolmogorov Complexity | 🚀 LIVE | Math of compression & randomness |
| 27 | Machine Super Intelligence | 🚀 LIVE | Safety & intelligence definitions |
| 28 | CS231n: CNNs for Visual Recognition | 🚀 LIVE | CNN layers from scratch, VGGNet analysis |
From Policy Gradients to RLHF
| Day | Paper | Status | Core Concept |
|---|---|---|---|
| 29 | Proximal Policy Optimization (PPO) | 🚀 LIVE | The algorithm behind ChatGPT (RLHF) |
| 30 | Deep Reinforcement Learning from Human Feedback | 🚀 LIVE | The birth of "Human Feedback" (RLHF) |
The modern era of LLMs
| Paper | Core Concept |
|---|---|
| BERT: Pre-training of Deep Bidirectional Transformers | Coming Soon |
| GPT-2: Language Models are Unsupervised Multitask Learners | Coming Soon |
| GPT-3: Language Models are Few-Shot Learners | Coming Soon |
| Chinchilla: Training Compute-Optimal Large Language Models | Coming Soon |
Complete paper list with links →
# Clone the repo
git clone https://github.com/yourusername/30u30.git
cd 30u30
# Start with Day 1
cd papers/01_Unreasonable_Effectiveness
For beginners: Basic Python knowledge. We'll teach you the rest.
For practitioners: Jump to any paper that interests you.
🎯 The 30-Day Challenge:
🔀 Choose Your Path:
There are many paper summaries online. But this is different:
Goal: The best free resource for learning AI fundamentals.
If this helps you, ⭐ star the repo and share it with others!
We'd love your help making this better!
Every contribution helps thousands of learners.
CC BY-NC-ND 4.0 — Free to read, learn, and share with attribution. Not for commercial use.
Ready to start?
→ Day 1: Character-Level RNN
→ Day 2: Understanding LSTMs
→ Day 3: RNN Regularization
→ Day 4: Minimizing Description Length
→ Day 5: MDL Principle Tutorial
→ Day 6: The First Law of Complexodynamics
→ Day 7: The Coffee Automaton
→ Day 8: ImageNet Classification (AlexNet)
→ Day 9: Deep Residual Learning (ResNet)
→ Day 10: Identity Mappings (ResNet v2)
→ Day 11: Dilated Convolutions
→ Day 12: Dropout
→ Day 13: Attention Is All You Need
→ Day 14: The Annotated Transformer
→ Day 15: Bahdanau Attention (NMT)
→ Day 16: Order Matters (Pointer Networks)
→ Day 17: Neural Turing Machines
→ Day 18: Pointer Networks
→ Day 19: Relational Reasoning
→ Day 20: Relational RNNs
→ Day 21: Neural Message Passing
→ Day 22: Deep Speech 2
→ Day 23: Variational Lossy Autoencoder
→ Day 24: GPipe (Giant Neural Networks)
→ Day 25: Scaling Laws for Neural Language Models
→ Day 26: Kolmogorov Complexity
→ Day 27: Machine Super Intelligence
→ Day 28: CS231n — CNNs for Visual Recognition
→ Day 29: Proximal Policy Optimization (PPO)
→ Day 30: RLHF 🆕 ← Start here!
Let's build something amazing together! 🚀
30u30 is an open-source study guide that takes you through the 30 foundational AI papers recommended by Ilya Sutskever — one paper per day. Each paper comes with a full implementation from scratch, detailed notes, interactive exercises with solutions, and visualizations.