RLinf RL Infrastructure
Scalable reinforcement-learning infrastructure for embodied robotics and agentic AI training.
- Source repo
- RLinf/RLinf
- Stars
- ★ 5.6k
- Last updated
- today
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 35/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free, no paid service needed
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- A robotics research team wants to SFT and then RL-fine-tune OpenVLA or π₀/π₀.₅ policies in ManiSkill or LIBERO using PPO, SAC, or another listed method.
- Not a fit if
- Solo developers without GPU clusters or multi-node hardware
- Users wanting a ready-made chat UI instead of config scripts
- Teams unwilling to install simulators, robots, or camera stacks
- Source review
- 35/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
RLinf is an open-source reinforcement-learning infrastructure for embodied AI and agentic AI, spanning supervised fine-tuning, simulated RL, and real-world RL. It supports workflows including PPO, GRPO, SAC, and IQL while using macro-to-micro flow transformation to organize distributed execution. The repository documents FSDP with HuggingFace, SGLang, or vLLM, plus Megatron with SGLang or vLLM for larger-scale workloads. Its embodied examples cover simulators, VLA/VLM/world models, real robots, and data collection; its agentic examples include SearchR1, rStar2, Online Coder, math reasoning RL, and WideSeek-R1. Delivery is documented through a Docker image and a PyPI library, with end-to-end recipes run from provided configurations and scripts rather than through a conversational interface.
RLinf consumes training configurations and scripts, orchestrates policy training, rollouts, and inference backends, and runs workflows such as PPO, Async PPO, GRPO, SAC, IQL, CrossQ, RLPD, RECAP, STEAM, RLT, and OPD. It connects training to FSDP + HuggingFace/SGLang/vLLM or Megatron + SGLang/vLLM, with training designed to scale across GPU nodes. Its embodied workflows cover SFT, simulated RL, real-world RL, and data collection with environments including ManiSkill, LIBERO, RoboTwin, IsaacLab, BEHAVIOR, Genesis, and Polaris, and with models such as π₀/π₀.₅, OpenVLA, GR00T, Qwen-VL, OpenSora, and Wan. The repository also documents CI coverage for unit, agent-reasoning, embodied end-to-end, and scheduler tests, and its workflows produce trained policies for evaluation, deployment, or further fine-tuning; exact deployment commands are not included in the supplied material.
- A robotics research team wants to SFT and then RL-fine-tune OpenVLA or π₀/π₀.₅ policies in ManiSkill or LIBERO using PPO, SAC, or another listed method.
- An embodied-AI team needs a data-collection-to-SFT-to-RL-to-deployment workflow for Franka, XSquare Turtle2, DOS-W1, or GimArm hardware.
- An infrastructure team operating multiple GPU nodes needs to choose between FSDP or Megatron training while using SGLang or vLLM for inference.
- A researcher wants ready-made starting points for IQL, GRPO, Async PPO, DAPO, SAC-Flow, DSRL, RECAP, STEAM, or OPD experiments.
- An agent-training team needs examples for search, coding, or reasoning workflows such as SearchR1, rStar2, AgentLightning Calc-X, Online Coder, math reasoning RL, or WideSeek-R1.
How do you install or deploy this agent?
The supplied material confirms that RLinf is available as a PyPI library and recommends its Docker image because embodied-RL environments and dependencies are complex. It does not provide a copyable package-install command, image name or tag, Python/CUDA versions, or credential requirements. A first installation therefore requires following either the documented “Installation Method 1: Docker image” path or the “Install as a Library” path after choosing the intended runtime.
How do you use this agent?
After environment setup, the material identifies a simple embodied-RL example using the ManiSkill3 simulator as the first working path. It also states that users can directly run the provided configurations and scripts for end-to-end SOTA RL reproduction. No exact commands, configuration paths, required environment variables, dataset-download steps, or minimum hardware requirements are included in the supplied evidence, so a reliable copyable invocation cannot be stated here.
What are this agent's strengths and limitations?
- Covers both embodied robotics and agentic reinforcement learning instead of focusing on one simulator or model family.
- Documents two distinct backend paths: FSDP + HuggingFace/SGLang/vLLM for rapid adaptation, and Megatron + SGLang/vLLM for large-scale training.
- Lists a full embodied workflow from SFT and simulation RL through real-world RL and deployment, across many simulators, model families, and robot setups.
- Supports a broad set of named algorithms, including PPO, GRPO, SAC, IQL, Async PPO, DAPO, RLPD, DSRL, RECAP, STEAM, RLT, and OPD.
- Embodied-RL setup is explicitly described as complex enough to warrant the recommended Docker image, which can raise adoption effort compared with a pure Python training library.
- The supplied material lacks exact installation commands, Docker image tags, hardware specifications, CUDA/Python compatibility ranges, and credential requirements.
- Many documented capabilities depend on particular simulators, robots, cameras, grippers, or external model ecosystems, so unlisted platforms will require integration work.
- The stated throughput and speed figures apply to specific workflows or simulator scenarios; the supplied material does not provide a general benchmark configuration.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| RLinf RL Infrastructure This agent | 35 · Major gaps | Library / SDKFree | ★ 5.6k | today | Python | — |
| veRL Agent Training | 32 · Major gaps | CLIFree + model costs | ★ 2.4k | 5mo ago | Python | — |
| Relax Omni-Modal RL Engine | 64 · Some gaps | CLIFree | ★ 670 | 1d ago | Python | — |
| Mava | 41 · Major gaps | CLIFree | ★ 941 | 1mo ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: the repository does not provide permission models, user confirmation mechanisms, data flow transparency, sensitive data handling, dependency security audits, external effect controls, rollback mechanisms, or source attribution. All trust criteria are unmet, hence scores of 0.
Evidence shows: consistent naming and structure in README and config files, but no failure messages or error handling documentation. Dependency availability: pyproject.toml lists dependencies but lacks version pinning or mirror info. Thus, self_consistency scores 2, dependency_availability and failure_messages score 1.
Evidence shows: README lists many scenarios and models but does not clearly define capability boundaries. Trigger precision: config files provide detailed parameters but not trigger conditions. Environment fit: multiple installation options but not all environment requirements. Thus, audience_and_scenarios and capability_boundaries score 2, trigger_precision scores 1, environment_fit scores 2.
Evidence shows: README provides clear information architecture, but install notes are incomplete. Naming stability: version numbers exist but no naming rules. Examples and FAQ: multiple examples but no FAQ. Known limitations not explicitly stated. License is Apache-2.0, complete. Versioning changelog exists. Maintenance responsibility not clear. Thus, information_architecture, examples_and_faq, versioning_changelog score 2, install_notes, naming_stability, known_limitations, maintenance_responsibility score 1, license scores 3.
Evidence shows: output usability: config examples provided but no output format. Marginal value: multiple features but no comparison with other frameworks. Cost-benefit: no performance data. Thus, output_usability scores 1, marginal_value scores 2, cost_benefit scores 1.
Evidence shows: README claims lack specific evidence but cite papers and docs. Cross-source corroboration: external links but no independent verification. Fact-inference separation: not clearly distinguished. Thus, claim_traceability, cross_source_corroboration, fact_inference_separation all score 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- The repository does not provide permission models or user confirmation mechanisms, may not be suitable for sensitive data.
- Dependencies are not version-pinned, potential supply chain risk.
- No rollback mechanism provided, updates may not be reversible.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v0.4 RLinf v0.4 Release
Release notes excerpt
🚀 RLinf v0.4 is here. Reinforcement learning steps out of simulation and into agile control of the physical world. v0.3 connected the path from simulation to real robots. v0.4 is the step toward deploying embodied RL at real-world scale. The release rebuilds the stack around three pillars: a modular real-robot system, high-efficiency real-robot algorithms, and full-modality models on a heterogeneous compute base. …View on GitHub ↗ -
v0.3 RLinf v0.3 Release
Release notes excerpt
🎉 Introducing RLinf v0.3. This release completes an end-to-end real-world training pipeline, adds new real-world RL components and algorithms, and brings more simulators and SOTA models to simulation RL. All supported examples have been strictly validated for correctness and reproducibility (see the test results at the end). Embodied 1. Models Continuing to expand the model ecosystem, 6 new embodied models are added, covering world models, VLA models, and system-level acceleration. …View on GitHub ↗ -
v0.2 RLinf v0.2 Release
Release notes excerpt
🎉 Introducing RLinf v0.2, our second public release. RLinf v0.2 focuses on two major directions: Real-World RL and Multi-agent RL systems. To support these goals, RLinf now supports real-world platforms including XSquare Turtle2 Arms and the Franka Arm, while offering a richer set of embodied benchmarks, simulators, models, algorithms, together with native asynchronous training designed for high-throughput workloads. This release also strengthens real-world deployment, sim-to-real, and co-training workflows, alongside more robust data and replay infrastructure and improved …View on GitHub ↗