Topics / #from-scratch

From-scratch projects on GitHub

Building complex systems from first principles with minimal dependencies.

As of , the most-starred From-scratch projects on GitHub are LLMs-from-scratch (106.9K stars), ai-engineering-from-scratch (71.3K stars) and serenity (33.9K stars). GitHub.Locker tracks 21 From-scratch repositories with 373.2K stars combined. The fastest-growing on October 19 was ai-engineering-from-scratch, with +420 stars in a day.

21Projects
373.2KStars in total
+685Stars in a day
Browse in the catalog →github.com/topics/from-scratch ↗

Hot right now

Most stars gained in a day · Oct 19

+420

ai-engineering-from-scratch

rohitg00/ai-engineering-from-scratch · ★ 71.3K

A free, open-source AI engineering curriculum with 523 lessons across 20 phases (about 342 hours), teaching you to build AI systems from raw math up to autonomous agent swarms, in Python, TypeScript, Rust, and Julia.

+95

LLMs-from-scratch

rasbt/LLMs-from-scratch · ★ 106.9K

This is the official code repository for the book 'Build a Large Language Model (From Scratch)', which teaches you to implement a GPT-like LLM in PyTorch step by step, from tokenization through pretraining and finetuning.

+55

llm-architecture-from-scratch

YeonwooSung/llm-architecture-from-scratch · ★ 1.1K

A collection of PyTorch implementations of modern LLM architectures (GPT-2, Llama 3/3.2, Llama 4, OLMo 2, Phi-4, DeepSeek v3, Gemma 3, xLSTM, Mistral 3.1) built from scratch to understand how they work internally.

+30

how-to-train-your-gpt

raiyanyahya/how-to-train-your-gpt · ★ 4.1K

A 12-chapter, 7,500+ line interactive textbook that teaches you how to build, train and run a modern GPT-style language model completely from scratch, with every line of code commented and explained in plain English.

The all-time top

21 projects · by stars

  1. #1 · rasbt/LLMs-from-scratch

    LLMs-from-scratch

    This is the official code repository for the book 'Build a Large Language Model (From Scratch)', which teaches you to implement a GPT-like LLM in PyTorch step by step, from tokenization through pretraining and finetuning.

    106.9K
  2. #2 · rohitg00/ai-engineering-from-scratch

    ai-engineering-from-scratch

    A free, open-source AI engineering curriculum with 523 lessons across 20 phases (about 342 hours), teaching you to build AI systems from raw math up to autonomous agent swarms, in Python, TypeScript, Rust, and Julia.

    71.3K
  3. #3 · SerenityOS/serenity

    serenity

    SerenityOS is a graphical Unix-like operating system for 64-bit x86, Arm, and RISC-V computers, built entirely from scratch.

    33.9K
  1. 4

    ML-From-Scratch

    33Keriklindernoren/ML-From-Scratch · Python

    A Python library of fundamental machine learning models and algorithms implemented from scratch in NumPy, aiming to cover everything from linear regression to deep learning.

  2. 5

    llama2.c

    20.2Kkarpathy/llama2.c · C

    It's a minimal project to train a small Llama 2 model in PyTorch and then run inference on it using a single pure C file, with no dependencies.

  3. 6

    numpy-ml

    16.3Kddbourgin/numpy-ml · Python

    A collection of machine learning algorithms — Gaussian mixture models, HMMs, neural networks, tree ensembles, reinforcement learning agents and more — implemented from scratch in pure NumPy.

  4. 7

    llama3-from-scratch

    15.2Knaklecha/llama3-from-scratch · Jupyter Notebook

    A Jupyter notebook that implements Llama3 entirely from scratch, building up every matrix multiplication by hand instead of calling pre-built transformer modules.

  5. 8

    raspberry-pi-os

    14Ks-matyukevich/raspberry-pi-os · C

    A step-by-step guide that teaches you how to create a simple operating system kernel from scratch, called RPi OS. Its code is largely based on the Linux kernel, and everything targets the Raspberry Pi 3.

  6. 9

    c4

    10.8Krswier/c4 · C

    C4 is a tiny, self-hosting compiler and interpreter for a small subset of the C language, written in just four functions as an exercise in extreme minimalism.

  7. 10

    db_tutorial

    10.5Kcstack/db_tutorial · C

    This is a hands-on tutorial that walks through writing a SQLite clone from scratch in C, teaching how a real database works internally by building one chapter by chapter.

  8. 11

    Machine-Learning

    10.4KJack-Cherish/Machine-Learning · Python

    A Chinese-language tutorial project that implements classic machine learning algorithms in Python3 - kNN, Decision Tree, Naive Bayes, Logistic Regression, SVM, Regression, Regression Tree and AdaBoost - following the exercises in the book 'Machine Learning in Action', with companion blog articles explaining each chapter.

  9. 12

    kimi-k3-in-c

    9.1KFareedKhan-dev/kimi-k3-in-c · C

    An inference engine for the 2.78-trillion-parameter Kimi K3 model, written in portable C99. It runs on a plain CPU in as little as 8 GB of RAM — no GPU, no BLAS, no framework — streaming the 1.56 TB checkpoint straight off disk.

  10. 13

    maths-cs-ai-compendium

    7.7KHenryNdubuaku/maths-cs-ai-compendium · TypeScript

    An open, unconventional online textbook covering maths, computing and AI from the ground up — intuition first, real-world context, for practitioners who want deep understanding rather than exam survival.

  11. 14

    ai-agents-from-scratch

    4.9Kpguso/ai-agents-from-scratch · JavaScript

    A hands-on tutorial repo that teaches you to build AI agents from first principles using local LLMs via node-llama-cpp, with no frameworks involved, so you understand function calling, memory, and ReAct before using production frameworks.

  12. 15

    how-to-train-your-gpt

    4.1Kraiyanyahya/how-to-train-your-gpt · Jupyter Notebook

    A 12-chapter, 7,500+ line interactive textbook that teaches you how to build, train and run a modern GPT-style language model completely from scratch, with every line of code commented and explained in plain English.

  13. 16

    zero-to-sglang

    1.5Kdatawhalechina/zero-to-sglang · Python

    The official SGLang x Datawhale course on LLM inference: it teaches how inference engines work, has you build a mini-sglang from scratch, and then digs into the real SGLang source code.

  14. 17

    mini-AGI

    1.3Kvolotat/mini-AGI · Python

    mini-AGI is a continual learning, byte-level language model that assembles its own architecture, trains from scratch on a single 8GB VRAM GPU, and keeps learning from everything it reads, without a separate fine-tuning phase.

  15. 18

    llm-architecture-from-scratch

    1.1KYeonwooSung/llm-architecture-from-scratch · Python

    A collection of PyTorch implementations of modern LLM architectures (GPT-2, Llama 3/3.2, Llama 4, OLMo 2, Phi-4, DeepSeek v3, Gemma 3, xLSTM, Mistral 3.1) built from scratch to understand how they work internally.

  16. 19

    TrenTorch

    559TrenTorch/TrenTorch · Python

    TrenTorch is a from-scratch ML framework and learning curriculum built on top of Harvard's TinyTorch (CS249r), taking the same teaching bricks further with cleaner internals and more modules covering deep learning, CNNs, transformers and inference.

  17. 20

    OpenArch

    426anuj0456/OpenArch · Python

    A set of hand-written PyTorch implementations of modern open-source LLM architectures (Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi and more), written from scratch for readability and learning.

  18. 21

    lytlnyblOS

    116red-mutt/lytlnyblOS · C

    lytlnyblOS ('little nibble OS') is a small 32-bit operating system built from scratch to teach beginners OS development practically. You write every line of it yourself while following the companion book.

Often tagged together