Global

SGLang is a high-performance serving framework for large language models and multimodal models.

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,

mini-AGI - is a continual learning byte-level language model that assembles its own architecture, trains from scratch on a single 8 GB VRAM GPU, and keeps learning from everything it reads.

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

深度学习入门教程, 优秀文章, Deep Learning Tutorial

Ensure PyTorch and CUDA are installed. The repo needs CUDA extensions to be built. If not present, run the following commands:

NOTE: this course does not yet exist. It is current being developed by Eureka Labs. Until it is ready I am archiving this repo !!!

Am I missing your favorite model? Is there something that could be cleaner / less confusing? Did I mess something up?

This is an advanced course on natural language processing. Automatically processing natural language inputs and producing language outputs is a key component of Artificial General Intelligence.

also, im going to load tensors directly from the model file that meta provided for llama3, you need to download the weights before running this file. here is the offical link to download the weights:

nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using Pytorch. (except comments or blank lines)

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

xFormers is: - Customizable building blocks: Independent/customizable building blocks that can be used without boilerplate code. The components are domain-agnostic and xFormers is used by researchers in vision, NLP and more.

This is the official repository for Attention Residuals (AttnRes), a drop-in replacement for standard residual connections in Transformers that enables each layer to selectively aggregate earlier representations via learned, input-dependent attention over depth.

Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

A Jev-like model takes a piece of text and a list of N text options. It returns one probability for each option. It does this in one pass instead of writing an answer word by word.

Self-supervised video pretraining has so far relied on architectural machinery to avoid representation collapse: an EMA target encoder, a stop-gradient, a capacity-limited predictor (V-JEPA), or pixel-space reconstruction with a dedicated decoder (VideoMAE).