What is this?
A collection of 60+ PyTorch implementations of deep learning papers, each annotated with side-by-side explanatory notes so you can read the code and the math/explanation together.
A collection of 60+ PyTorch implementations of deep learning papers, each annotated with side-by-side explanatory notes so you can read the code and the math/explanation together.Is this a library I import, or more like a reading resource?It's primarily a literate-programming reading resource: annotated source code meant to be read and studied rather than a packaged library for production use, though you can still run or copy the implementations.
It's primarily a literate-programming reading resource: annotated source code meant to be read and studied rather than a packaged library for production use, though you can still run or copy the implementations.How do I actually read or run these implementations?Browse the annotated notes on the labml.ai website for side-by-side code and explanations, or clone the repo and run the individual Python/PyTorch scripts directly.
Browse the annotated notes on the labml.ai website for side-by-side code and explanations, or clone the repo and run the individual Python/PyTorch scripts directly.What do I need to run the code?Python and PyTorch, since every implementation is written as a simple PyTorch script.
Python and PyTorch, since every implementation is written as a simple PyTorch script.Are these production-ready implementations?No — they're deliberately kept simple and explanatory ('simple PyTorch implementations') to aid understanding, not optimized or hardened for production use.
No — they're deliberately kept simple and explanatory ('simple PyTorch implementations') to aid understanding, not optimized or hardened for production use.What topics are covered besides transformers?GANs (CycleGAN, StyleGAN2, Wasserstein GAN), diffusion models (DDPM, DDIM, Stable Diffusion), reinforcement learning (PPO, DQN), graph neural networks, capsule networks, optimizers (Adam, AdaBelief, Sophia), normalization layers, and more.
GANs (CycleGAN, StyleGAN2, Wasserstein GAN), diffusion models (DDPM, DDIM, Stable Diffusion), reinforcement learning (PPO, DQN), graph neural networks, capsule networks, optimizers (Adam, AdaBelief, Sophia), normalization layers, and more.Is the project actively maintained?Yes, the maintainers say they add new implementations almost weekly.
Yes, the maintainers say they add new implementations almost weekly.Does it cover LoRA or large language model techniques?Yes, it includes a Low-Rank Adaptation (LoRA) implementation and GPT-NeoX examples covering generation, fine-tuning, and LLM.int8() quantization.
Yes, it includes a Low-Rank Adaptation (LoRA) implementation and GPT-NeoX examples covering generation, fine-tuning, and LLM.int8() quantization.What is special about the way the code is presented?Each implementation uses literate programming style, pairing code line-by-line with explanatory notes rendered as a side-by-side document on the labml.ai website.
Each implementation uses literate programming style, pairing code line-by-line with explanatory notes rendered as a side-by-side document on the labml.ai website.What are similar projects?DeepLearning, pytorch-tutorial, ganhacks, LLM101n, numpy-ml, pytorch-book, nlp-tutorial, train-llm-from-scratch
DeepLearning, pytorch-tutorial, ganhacks, LLM101n, numpy-ml, pytorch-book, nlp-tutorial, train-llm-from-scratchWhich topics is it in?graph-neural-networks, deep-learning-tutorial, attention, generative-adversarial-network, gan, diffusion, neural-networks, optimization, lora, transformers, reinforcement-learning, transformer
graph-neural-networks, deep-learning-tutorial, attention, generative-adversarial-network, gan, diffusion, neural-networks, optimization, lora, transformers, reinforcement-learning, transformer