Global

I implemented a transformer model from scratch using PyTorch, based on the paper Attention is All You Need. You can use my scripts to train your own billion or million parameter LLM using a single GPU.

This repository contains the code for developing, pretraining, and finetuning a GPT-like LLM and is the official code repository for the book Build a Large Language Model (From Scratch).

A high-throughput and memory-efficient inference and serving engine for LLMs

Implementing LLM architectures from scratch in PyTorch

Build a modern LLM from scratch. Every line commented. Explained like we are five.

> 第八章聚焦 GRPO、OPD、Search-R1 与 ReTool。更多 Agentic RL 算法、训练代码与实验实践，可以前往作者持续维护的另一个仓库 agentic-rl-lab；该仓库更新频率更高，会持续跟进新的算法与环境。

> Breakthrough inference speed for MiniMax H3 Reference-to-Video (Ref2VA / R2VA): Generate a full 5-second, 24 fps video conditioned on reference character images in 72s (warm) / 95s (first run) on a single consumer NVIDIA GeForce RTX 4090 (24GB).

A course on neural networks that starts all the way at the basics. The course is a series of YouTube videos where we code and train neural networks together. The Jupyter notebooks we build in the videos are then captured here inside the lectures directory.

分享AI Infra知识&代码练习：PyTorch、vLLM/SGLang、slime/vime框架入门⚡️、性能加速🚀、大模型基础🧠、AI软硬件🔧等

OpenWAM is an open research stack for systematically developing World-Action Models (WAMs). It turns tightly coupled design choices into modular components and controlled experiments. It consists of:

邱锡鹏《神经网络与深度学习》第二版与通识版：电子书、章节目录、学习资源与勘误。

RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.

DeepGEMM is a unified, high-performance tensor core kernel library that brings together the key computation primitives of modern large language models — GEMMs (FP8, FP4, BF16), fused MoE with overlapped communication (Mega MoE), MQA scoring for the lightning indexer…

📍 Use GLM-5.3 & GLM-5.3-Flash API services on Z.ai API Platform.

Material for gpu-mode lectures

Download everything (about 82 GB) into ckpts/ from Hugging Face using

A PyTorch re-implementation of GPT, both training and inference. minGPT tries to be small, clean, interpretable and educational, as most of the currently available GPT model implementations can a bit sprawling. py).

This is the official repository for Attention Residuals (AttnRes), a drop-in replacement for standard residual connections in Transformers that enables each layer to selectively aggregate earlier representations via learned, input-dependent attention over depth.

deep learning for image processing including classification and object-detection etc.

3 快速搭建你的第一个系统，并进行迭代(Build your first system quickly, then iterate)

Each implementation: - ✅ Uses only NumPy (no deep learning frameworks) for educational clarity - ✅ Includes synthetic/bootstrapped data for immediate execution - ✅ Provides extensive visualizations and explanations - ✅ Demonstrates core concepts from each paper - ✅ Runs in…

An interactive coding platform, but for tensors. Self-hosted. Jupyter-based.

0, running FP8 on the tensor cores. The intermediates match too, not just the final image: all 75 block boundaries, byte for byte.

We've been very happy to see FlashAttention being widely adopted in such a short time after its release. This page contains a partial list of places where FlashAttention is being used.

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

2-Exp models. This repository contains the following implementations:

Every clip was produced by the launchers in this repo, on the data bundled in examples/ — no external dataset, no cherry-picking across seeds. mp4.

The Engram module augments the backbone by retrieving static $N$-gram memory and fusing it with dynamic hidden states. The architecture is shown below (drawio provided):

8-Flash-Next is the foundation model developed by Qwen Team, Alibaba Group.

Note: This series will continue to grow as I write more blogs and create more videos on new topics. Keep learning.