What is this?
ART (Agent Reinforcement Trainer) is an open-source RL framework from OpenPipe that lets LLM agents learn from experience: you train multi-step agents for real-world tasks with GRPO.
ART (Agent Reinforcement Trainer) is an open-source RL framework from OpenPipe that lets LLM agents learn from experience: you train multi-step agents for real-world tasks with GRPO.Is ART a library I code against, or a service that trains models for me?Both halves: a Python client you embed in your app plus a separate training server. The client is OpenAI-compatible, so your code just exchanges messages while the server handles vLLM inference and GRPO training behind the scenes.
Both halves: a Python client you embed in your app plus a separate training server. The client is OpenAI-compatible, so your code just exchanges messages while the server handles vLLM inference and GRPO training behind the scenes.How do I get started with it?Add the pip package to any Python project, then run one of the example training notebooks — email search, 2048, Tic Tac Toe — before wiring up your own agent and reward function.
Add the pip package to any Python project, then run one of the example training notebooks — email search, 2048, Tic Tac Toe — before wiring up your own agent and reward function.Which models can I train, and are there any it won't take?Most vLLM/HuggingFace-transformers-compatible causal models should work, including everything Unsloth supports — but Gemma 3 is not supported for now. Uncooperative models can be reported on GitHub or Discord.
Most vLLM/HuggingFace-transformers-compatible causal models should work, including everything Unsloth supports — but Gemma 3 is not supported for now. Uncooperative models can be reported on GitHub or Discord.Do I need my own GPU to train an agent?No. Run the client on your laptop and let the server kick off an ephemeral GPU-enabled environment (or use a local GPU); the managed W&B serverless option removes the infra entirely.
No. Run the client on your laptop and let the server kick off an ephemeral GPU-enabled environment (or use a local GPU); the managed W&B serverless option removes the infra entirely.What is W&B Training (Serverless RL)?A managed service layered on ART — the first public service for training models with RL. It runs the training and inference infrastructure for you, with claimed 40% lower cost, 28% faster training, scale to 2000+ concurrent requests, and every checkpoint instantly available via W&B Inference.
A managed service layered on ART — the first public service for training models with RL. It runs the training and inference infrastructure for you, with claimed 40% lower cost, 28% faster training, scale to 2000+ concurrent requests, and every checkpoint instantly available via W&B Inference.What can I actually train an agent to do with it?The example notebooks cover email research (ART•E), games like 2048, Tic Tac Toe, Codenames and Temporal Clue, tool use on MCP servers (MCP•RL), and LangGraph agents. There are also SFT-only jobs, like distilling text-to-SQL from Qwen 3 235B down to Qwen 3.6 27B.
The example notebooks cover email research (ART•E), games like 2048, Tic Tac Toe, Codenames and Temporal Clue, tool use on MCP servers (MCP•RL), and LangGraph agents. There are also SFT-only jobs, like distilling text-to-SQL from Qwen 3 235B down to Qwen 3.6 27B.Does it work with LangGraph or MCP servers?Yes. ART integrates with LangGraph so you can train existing LangGraph agents with RL, and MCP•RL automatically trains models to use the tools of any MCP server.
Yes. ART integrates with LangGraph so you can train existing LangGraph agents with RL, and MCP•RL automatically trains models to use the tools of any MCP server.Can I use it for plain supervised fine-tuning, not just RL?Yes — alongside RL it supports SFT: distillation from a larger model, fine-tuning from a dataset, and an SFT warmup stage before switching to RL, as shown in the dedicated notebooks.
Yes — alongside RL it supports SFT: distillation from a larger model, fine-tuning from a dataset, and an SFT warmup stage before switching to RL, as shown in the dedicated notebooks.What are similar projects?unsloth, halo, verl, ms-swift, verl, OpenClaw-RL, DeepAnalyze, NeoHorse
unsloth, halo, verl, ms-swift, verl, OpenClaw-RL, DeepAnalyze, NeoHorseWhich topics is it in?rl, grpo, qwen3, langgraph, llms, lora, vllm, reinforcement-learning, qwen, fine-tuning, openai-compatible, agentic-ai
rl, grpo, qwen3, langgraph, llms, lora, vllm, reinforcement-learning, qwen, fine-tuning, openai-compatible, agentic-ai