Ppo projects on GitHub

Proximal policy optimization algorithm for reinforcement learning.

As of October 19, 2026, the most-starred Ppo projects on GitHub are minimind (64K stars), easy-rl (14.8K stars) and train-llm-from-scratch (12.2K stars). GitHub.Locker tracks 8 Ppo repositories with 130.6K stars combined. The fastest-growing on October 18 was train-llm-from-scratch, with +86 stars in a day.

8Projects

130.6KStars in total

+234Stars in a day

Browse in the catalog →github.com/topics/ppo ↗

Hot right now

Most stars gained in a day · Oct 18

+86

train-llm-from-scratch

FareedKhan-dev/train-llm-from-scratch · ★ 12.2K

A from-scratch tutorial and codebase for training your own LLM end to end: tokenizing raw text, pretraining a GPT-style Transformer, and then post-training it with SFT, a reward model, DPO/ORPO/KTO, PPO and GRPO into an aligned, reasoning-style assistant.

+65

minimind

jingyaogong/minimind · ★ 64K

MiniMind is an open-source project that lets you train a tiny large language model (around 64M parameters) completely from scratch, in about 2 hours on a single RTX 3090 for roughly the cost of 3 RMB in GPU rental.

+57

microduck

pollen-robotics/microduck · ★ 9.6K

Microduck is a tiny biped duck robot (about 25cm, 800g), and this repo is the robot's brain: Rust daemons that run a 50Hz control loop driving fifteen servos using reinforcement-learning policies exported to ONNX.

+12

hands-on-modern-rl

walkinglabs/hands-on-modern-rl · ★ 4.7K

Hands-On Modern RL is an open-source course and book that walks learners from basic reinforcement learning concepts through LLM alignment (RLHF, DPO, GRPO, RLVR) and into advanced agentic systems, combining theory with runnable code.

The all-time top

8 projects · by stars

#1 · jingyaogong/minimind
minimind
MiniMind is an open-source project that lets you train a tiny large language model (around 64M parameters) completely from scratch, in about 2 hours on a single RTX 3090 for roughly the cost of 3 RMB in GPU rental.
64K
#2 · datawhalechina/easy-rl
easy-rl
This is Easy-RL (nicknamed the 'Mushroom Book', 蘑菇书), a free Chinese-language tutorial on deep reinforcement learning adapted from Hung-yi Lee's lecture videos, plus supplementary material from Zhou Bolei's 'Reinforcement Learning Outline' and other courses.
14.8K
#3 · FareedKhan-dev/train-llm-from-scratch
train-llm-from-scratch
A from-scratch tutorial and codebase for training your own LLM end to end: tokenizing raw text, pretraining a GPT-style Transformer, and then post-training it with SFT, a reward model, DPO/ORPO/KTO, PPO and GRPO into an aligned, reasoning-style assistant.
12.2K

4
tianshou
11Kthu-ml/tianshou · Python
Tianshou is an elegant deep reinforcement learning library built on pure PyTorch and Gymnasium, providing modular implementations of dozens of RL algorithms like DQN, PPO, SAC, and TD3.
5
cleanrl
10.6Kvwxyzjn/cleanrl · Python
CleanRL is a collection of high-quality single-file implementations of deep reinforcement learning algorithms (PPO, DQN, C51, SAC, DDPG, TD3, PPG and more), written to be read and tweaked rather than imported.
6
microduck
9.6Kpollen-robotics/microduck · Rust
Microduck is a tiny biped duck robot (about 25cm, 800g), and this repo is the robot's brain: Rust daemons that run a 50Hz control loop driving fifteen servos using reinforcement-learning policies exported to ONNX.
7
hands-on-modern-rl
4.7Kwalkinglabs/hands-on-modern-rl · Python
Hands-On Modern RL is an open-source course and book that walks learners from basic reinforcement learning concepts through LLM alignment (RLHF, DPO, GRPO, RLVR) and into advanced agentic systems, combining theory with runnable code.
- rl
- dpo
- sft
- grpo
8
GR00T-WholeBodyControl
3.8KNVlabs/GR00T-WholeBodyControl · Python
NVIDIA GR00T Whole-Body Control — a codebase hosting pretrained checkpoints plus training, evaluation, and deployment code for advanced humanoid robot controllers: the Decoupled WBC used in GR00T N1.5/N1.6, the GEAR-SONIC series, and MotionBricks.