ART Agent Reinforcement Trainer
Train multi-step agents from real task experience with GRPO.
- Source repo
- OpenPipe/ART
- Stars
- ★ 11k
- Last updated
- today
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 34/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API (Partial support)
- Cost
- Free tier plus a paid hosted plan
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- A team with an existing Python agent application wants to add reward-driven GRPO training without implementing the complete training loop itself.
- Not a fit if
- Teams without GPU hardware or any infra budget
- Services that must keep serving while training runs
- Teams unwilling to use W&B's managed service or API key
- Source review
- 34/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
ART is an open-source reinforcement-learning framework for improving the reliability of multi-step agents through task experience. Its architecture separates a Python client from an independent training server: the client integrates with an existing application through an OpenAI-compatible interface, while the server handles inference, GRPO training, checkpoints, and LoRA loading. During an agent workflow, ART records system, user, and assistant messages in a Trajectory, to which the application assigns a reward. The server trains on grouped trajectories, saves the resulting LoRA to a local directory, and loads it into vLLM for the next inference round. Teams can run the server on their own GPU or use W&B Training, which requires a W&B API key and manages the training and inference infrastructure.
An application sends completion requests through the ART client and can execute multiple rollouts in parallel. The server runs the model's latest LoRA in vLLM, while the client records each rollout's system, user, and assistant messages in a Trajectory; application code then assigns that Trajectory a reward. After all rollouts finish, grouped trajectories are sent to the server, inference is paused, and GRPO training begins from the latest checkpoint or an empty LoRA on the first iteration. The server writes the updated LoRA to a local directory, loads it into vLLM, and resumes inference. The supplied examples also cover dataset-based SFT, SFT followed by RL, LangGraph agents, MCP tool use, email retrieval, and game-playing tasks.
- A team with an existing Python agent application wants to add reward-driven GRPO training without implementing the complete training loop itself.
- Researchers need to collect parallel multi-step rollouts and train Qwen or another compatible causal language model from full conversation trajectories.
- An infrastructure team wants to control the training server, checkpoints, and vLLM inference on its own GPU hardware.
- A team that does not want to operate GPU infrastructure can use W&B Training to manage training and inference and expose checkpoints through W&B Inference.
- LangGraph developers want to apply reinforcement learning to multi-step reasoning and tool-use behavior.
- Model teams want to experiment with text-to-SQL distillation, document summarization using SFT plus RL, or Qwen fine-tuning from a dataset file.
How do you install or deploy this agent?
On any client machine capable of running Python, add ART to an existing project with:
pip install openpipe-artThe source does not state a minimum Python version, supported operating systems, or a complete local-server installation command. A self-hosted training server requires a GPU-equipped machine. The W&B Training path additionally requires a valid W&B API key and network access.
How do you use this agent?
The source provides the following W&B Training registration example. It selects Qwen/Qwen3.6-27B, creates a ServerlessBackend, and registers it with an art.TrainableModel:
from art.serverless.backend import ServerlessBackend
model = art.TrainableModel(
project="voice-agent",
name="agent-001",
run_name="agent-001",
base_model="Qwen/Qwen3.6-27B"
)
backend = ServerlessBackend(
api_key="your_wandb_api_key"
)
model.register(backend)Replace the placeholder with a W&B API key. This excerpt assumes that art has been imported elsewhere; the source does not show that import or a complete first rollout, Trajectory reward assignment, or training-start invocation. Runnable introductions are instead provided as Colab notebooks for 2048, ART·E, LangGraph, MCP·RL, Tic Tac Toe, Codenames, and SFT workflows.
What are this agent's strengths and limitations?
- The client-server split lets an ordinary Python client drive remote GPU training, while also permitting a locally hosted GPU server.
- It defines a concrete iterative workflow spanning trajectory capture, rewards, GRPO, LoRA checkpoints, and vLLM inference.
- The client is designed for integration into existing Python applications and exposes an OpenAI-compatible interface.
- Both managed W&B Training and self-hosted deployment paths are documented; the managed path handles training and inference infrastructure.
- Examples span email retrieval, LangGraph, MCP, games, text-to-SQL, summarization, and combined SFT/RL workflows.
- Inference is blocked while each training phase runs, which may affect availability or throughput for continuously served applications.
- Self-hosting requires GPU hardware and responsibility for the runtime environment, while the source does not provide a complete deployment command.
- The W&B Training route depends on a W&B API key and managed service; pricing and permission details are not supplied.
- Gemma 3 is explicitly described as apparently unsupported, and support for other models is expressed as an expectation rather than an exhaustive compatibility guarantee.
- Model portability is bounded by compatibility with vLLM, HuggingFace Transformers, or the models supported by Unsloth.
How does this agent compare with similar options?
Compared with running on a local GPU, W&B Training manages the training and inference infrastructure and makes each checkpoint available through W&B Inference; local operation preserves a self-hosted server boundary but carries the GPU and operational burden. The source claims that the managed option lowers cost by 40%, trains 28% faster, and scales beyond 2,000 concurrent requests through a shared production inference cluster, but it provides neither a pricing table nor independent validation. ART also credits Unsloth, vLLM, TRL, and torchtune as foundational projects, without presenting a feature-by-feature comparison against them.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| ART Agent Reinforcement Trainer This agent | 34 · Major gaps | Library / SDKFreemium | ★ 11k | today | Python | — |
| AReaL Async RL Platform | Insufficient evidence | CLIFree | ★ 5.9k | 1d ago | Python | — |
| Agent Lightning | 40 · Major gaps | CLIFree + model costs | ★ 19k | 1d ago | Python | OpenAI API |
| Relax Omni-Modal RL Engine | 64 · Some gaps | CLIFree | ★ 670 | 1d ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: the repository does not provide a permission model, user confirmation mechanism, data flow transparency, sensitive data handling, dependency security audit, external effects control, rollback mechanism, or source attribution. All trust-related criteria are unmet, hence score 0.
Evidence shows: README and test files are consistent in describing the training loop and step skipping behavior, but no detailed failure messages or error handling are provided. Dependencies are listed in pyproject.toml, but no availability guarantees are given. Thus, self-consistency scores 2, dependency availability and failure messages each score 1.
Evidence shows: README describes multiple use cases (e.g., 2048, email agent, MCP), but does not clearly define capability boundaries or trigger conditions. Environment fit is addressed with installation instructions and dependencies, but no detailed configuration for different environments. Thus, audience and scenarios score 2, capability boundaries and trigger precision each score 1, environment fit scores 2.
Evidence shows: README provides installation instructions, examples, and license information, but no changelog or version history. Naming stability is not explicitly stated. Known limitations mention Gemma 3 not supported, but not comprehensive. Thus, information architecture, install notes, examples, and license each score 2, naming stability, known limitations, versioning, and maintenance responsibility each score 1.
Evidence shows: README presents multiple examples and benchmarks, indicating high output usability. Marginal value is supported by claims of integration and performance improvements, but no cost-benefit analysis is provided. Thus, output usability and marginal value each score 2, cost-benefit scores 1.
Evidence shows: performance claims (e.g., 40% cost reduction, 28% training speedup) lack specific data or source citations. Benchmark charts exist, but raw data or reproduction methods are not provided. Thus, claim traceability, cross-source corroboration, and fact-inference separation each score 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- Performance claims (e.g., 40% cost reduction, 28% training speedup) lack specific data or citations; treat with caution.
- No permission model or user confirmation mechanism is provided; assess security risks yourself.
- Dependency versions are tightly pinned, which may affect environment compatibility.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
Release Highlights - Weave 0.54.0 or later is now required. Weave 0.53.9 and earlier fail in weave.init() with AttributeError: 'OTLPSpanExporter' object has no attribute '_session' once opentelemetry-exporter-otlp-proto-http 1.45 is installed. Installing ART now upgrades an older Weave instead of keeping it. …View on GitHub ↗ -
Release notes excerpt
Release Highlights - Model.delete_checkpoints works with polars 2.x. It averaged every history.jsonl column per step, and polars 2 raises on the string recorded_at column (InvalidOperationError: mean operation not supported for dtype str). It now averages only the checkpoint metric ([#1150](https://github.com/OpenPipe/ART/pull/1150)). What's Changed - fix: average only the metric in delete_checkpoints ([#1150](https://github.com/OpenPipe/ART/pull/1150)) Full Changelog: https://github.com/OpenPipe/ART/compare/v0.5.22...v0.5.23View on GitHub ↗ -
Release notes excerpt
Release Highlights - Model metrics now log to the model's W&B entity. In 0.5.21, a model created with an explicit entity that differed from the API key's default entity logged model.log() metrics to a run under the default entity instead ([#1134](https://github.com/OpenPipe/ART/pull/1134)). What's Changed - feat: support opt-in multi-adapter MoE LoRA waves ([#1139](https://github.com/OpenPipe/ART/pull/1139)) - fix: bound context-parallel admission and memory accounting ([#1138](https://github.com/OpenPipe/ART/pull/1138)) …View on GitHub ↗