rLLM: Reinforcement Learning Framework for LLM Agents

What does this agent do, and when should you use it?

rLLM is an open-source framework for training language agents with reinforcement learning. It provides a unified API to run your agent on any harness (Claude Code, Codex, Terminus-2, mini-swe-agent, opencode, etc.), execute in any sandbox (Docker, Daytona, Modal, or local), and switch between training backends (verl, tinker, fireworks) with a single flag. The framework includes 60+ integrated benchmarks across math, code, QA, search, VLM, and agentic tasks, and supports multiple training methods like GRPO, REINFORCE, RLOO, SFT, and on-policy distillation. A model gateway captures token IDs and logprobs, enabling the same agent code for both evaluation and training. rLLM has been used to train state-of-the-art open-source models like DeepScaleR-1.5B, DeepCoder-14B, and DeepSWE-32B, and is adopted by academic labs and industry teams.

rLLM runs a pipeline: run your agent → collect traces → compute rewards → update the model. The agent is defined via the @rllm.rollout decorator, and the reward function via @rllm.evaluator. During training, the model gateway routes LLM calls and captures token IDs and logprobs, organizing them into Episodes (one task), Trajectories (agent run), and Steps (LLM call). The training backends include verl (distributed multi-GPU), tinker (single-machine), and fireworks (Fireworks platform). For evaluation, it provides CLI commands like rllm eval <benchmark> and rllm train <benchmark>. The framework automatically pulls and runs benchmark datasets, and supports parallel rollout collection via a workflow engine.

1. Researchers want to train large language models with RL methods like GRPO on multi-GPU clusters.
2. Teams need to evaluate their agents on many benchmarks (e.g., SWE-bench, AIME) without writing separate code for each.
3. Developers have existing agent implementations (e.g., using OpenAI API or LangGraph) and want to add RL training without major refactoring.
4. Organizations that need flexibility to switch training backends (e.g., from single-machine tinker to distributed verl) based on resource availability.
5. Teams that require sandboxed execution (Docker, Modal) for safe and reproducible training runs.

What are this agent's strengths and limitations?

How do you install or deploy this agent?

Requires Python >= 3.11. Recommended installation using uv:

# Install core CLI (tinker backend)
uv pip install "rllm @ git+https://github.com/rllm-org/rllm.git"

# For distributed multi-GPU training (verl + vLLM/SGLang)
uv pip install "rllm[verl] @ git+https://github.com/rllm-org/rllm.git"

# For Fireworks platform
uv pip install "rllm[fireworks] @ git+https://github.com/rllm-org/rllm.git"

Alternatively, build from source or use Docker. See https://docs.rllm-project.com/installation for details.

How do you use this agent?

After installation, you can use the CLI or Python API.

CLI quickstart:

# Configure model provider
rllm model setup

# Evaluate on a benchmark (e.g., gsm8k)
rllm eval gsm8k

# Train on a benchmark
rllm train gsm8k

Python API example:

1. Define a rollout function with @rllm.rollout.
2. Define an evaluator with @rllm.evaluator.
3. Create an AgentTrainer and call train().

See the cookbooks directory for complete examples.

How does this agent compare with similar options?

FAQ