Open-AgentRL
RL training framework for reasoning, tool use, and interactive agent environments.
What does this agent do, and when should you use it?
Open-AgentRL brings together three agentic-RL research implementations: DemyAgent, RLAnything, and AutoTool. DemyAgent covers cold-start SFT and GRPO-TCR-based agentic RL, with evaluation workflows for math, science, and coding reasoning. RLAnything performs closed-loop optimization of policy, reward model, and environment for OSWorld computer control, AlfWorld text games, and coding tasks. AutoTool supplies a two-phase dynamic tool-selection framework that first stabilizes tool-integrated trajectories and then refines multi-step selection with KL-regularized Plackett–Luce ranking. It is a research training repository that requires users to supply models, data, configuration, and, for some scenarios, external execution infrastructure rather than a ready-to-deploy chat service.
For DemyAgent, the repository reads SFT, RL, and evaluation data in .parquet format, fine-tunes Qwen models through recipe/demystify/qwen3_4b_sft.sh or qwen2_7b_sft.sh, then runs GRPO-TCR with grpo_tcr_qwen2_7b.sh or grpo_tcr_qwen3_4b.sh. Code-execution tasks use a SandboxFusion /run_code endpoint, configured in sandbox_fusion_tool_config.yaml and verl/utils/reward_score/livecodebench/code_math.py. RLAnything launches training from scenario-specific YAML files through osworld_rl.py, alfworld_rl.py, and coding_rl.py, with matching *_eval.py evaluation programs. The AutoTool framework covers Phase I SFT-plus-RL trajectory stabilization and Phase II ranking-based tool-selection refinement; the repository says its full training data and toolset are still forthcoming.
- A research team reproducing agentic-reasoning experiments from 3K SFT and 30K RL datasets, then evaluating on AIME, GPQA-Diamond, or LiveCodeBench-v6.
- A team with a local or cloud SandboxFusion endpoint training Qwen-family models that execute code during reasoning.
- A team that has configured Volcengine cloud VMs, security groups, and an OSWorld image for RL training or evaluation of GUI computer-control policies.
- A researcher who has downloaded AlfWorld data and needs to train or evaluate RL policies in a text-based interactive game.
- A group investigating dynamic selection over a large, evolving tool library and willing to work before AutoTool’s full data and toolset are released.
What are this agent's strengths and limitations?
- Covers agentic reasoning, GUI computer control, text-game environments, coding, and dynamic tool selection in one repository rather than a single benchmark recipe.
- DemyAgent documents an end-to-end path from cold-start SFT through GRPO-TCR, including named datasets, model checkpoints, scripts, and configuration fields.
- RLAnything explicitly combines policy, reward-model, and environment adaptation in a closed loop, with separate training and evaluation entry points for OSWorld, AlfWorld, and coding.
- AutoTool specifies a two-phase approach with KL-regularized Plackett–Luce ranking for multi-step dynamic tool selection.
- It is not a plug-and-play inference product: adopters must provide models, `.parquet` data, absolute paths, and scenario configuration.
- Code-execution training depends on SandboxFusion or a Volcano Engine Cloud FaaS endpoint, with endpoint changes required in both tool configuration and reward-scoring code.
- The documented OSWorld experiment depends on Volcengine security groups, VM images, and a parallel cloud VM pool; infrastructure costs are not stated.
- AutoTool provides only an example format for its 200K tool-use trajectories; the full training data and toolset are described as coming soon.
How do you install or deploy this agent?
For DemyAgent:
git clone https://github.com/Gen-Verse/Open-AgentRL.git
conda create -n OpenAgentRL python=3.11
conda activate OpenAgentRL
cd Open-AgentRL
bash scripts/install_vllm_sglang_mcore.sh
pip install -e .[vllm]For RLAnything:
conda create --name rlanything python=3.10
source activate rlanything
pip install -r requirements_rlanything.txtThe supplied material does not state an API-key requirement. Code-execution RL additionally needs a SandboxFusion deployment or a Volcano Engine Cloud FaaS /run_code endpoint; the documented OSWorld setup uses Volcengine cloud VMs.
How do you use this agent?
Before DemyAgent cold-start SFT, set absolute TRAIN_DATA, EVAL_DATA, MODEL_PATH, and SAVE_PATH values in recipe/demystify/qwen3_4b_sft.sh or qwen2_7b_sft.sh, then run:
bash recipe/demystify/qwen3_4b_sft.shAfter configuring the SandboxFusion endpoint, RL data, model, benchmark paths, and default_local_dir, run:
bash recipe/demystify/grpo_tcr_qwen3_4b.shFor RLAnything, the first training invocation depends on the scenario; for example:
python osworld_rl.py config=configs/osworld_rl.yamlFor a single-node run, set num_node to 1 in the corresponding YAML configuration.
How does this agent compare with similar options?
The repository states that its implementation builds on VeRL and ReTool. The supplied material does not provide a feature-by-feature comparison with either project.