VMAS — Vectorized Multi-Agent Simulator
A PyTorch 2D vectorized, differentiable physics simulator that gives MARL research batched multi-robot benchmark scenarios with Gym, RLlib, and TorchRL wrappers.
- Source repo
- proroklab/VectorizedMultiAgentSimulator
- Stars
- ★ 628
- Last updated
- 5mo ago
- License
- GPL-3.0
- Primary language
- Python
- FA score
- 44/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free, no paid service needed
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- MARL researchers comparing MAPPO, IPPO, MADDPG, QMIX, or VDN on shared benchmark tasks can train VMAS scenarios via TorchRL or BenchMARL.
- Not a fit if
- Teams that need 3D rigid-body dynamics or high-fidelity robot simulation
- Users who want to learn low-level motor/joint control policies
- Teams whose runtime cannot provide Python 3.8-3.11 with PyTorch
- Source review
- 44/100 · Major gaps 3 safety controls not found
What does this agent do, and when should you use it?
VMAS (VectorizedMultiAgentSimulator) is a vectorized differentiable simulator built for efficient multi-agent reinforcement learning benchmarking. It pairs a fully differentiable 2D physics engine written in PyTorch tensor operations with a set of challenging multi-robot scenarios such as transport, give_way, wheel, flocking, and football. Because simulation runs as batched PyTorch ops, it can step tens of thousands of parallel environments on accelerated hardware, cutting rollout collection time in MARL training. Environments are created with make_env and ship with wrappers for OpenAI Gym, Gymnasium, RLlib, TorchRL, and the BenchMARL MARL training library. Setting grad_enabled=True makes the simulator differentiable so gradients flow from outputs back to inputs. New tasks are added by subclassing BaseScenario and implementing make_world, reset_world_at, observation, and reward, with interactive rendering available for debugging.
You call vmas.make_env(scenario=..., num_envs=..., device=..., wrapper=..., grad_enabled=..., dict_spaces=...) to get an environment object exposing step, reset, render, done, and get_from_scenario. Each env.step() takes a list or dict of per-agent action tensors shaped [num_envs, action_size]; the engine advances batched 2D physics — elastic collisions, joints, custom gravity — and returns per-agent observations, rewards, infos, and done. Observations are [num_envs, observation_size] tensors (or dicts of tensors) and rewards are [num_envs]. The optional wrapper converts the env into rllib, gym, gymnasium, or gymnasium_vec interfaces. render supports mode="rgb_array" or "human", can focus the camera on one agent via agent_index_focus, and can overlay a plotted function with plot_position_function. Scenario scripts can be executed directly for interactive debugging with arrow keys and TAB.
- MARL researchers comparing MAPPO, IPPO, MADDPG, QMIX, or VDN on shared benchmark tasks can train VMAS scenarios via TorchRL or BenchMARL.
- Paper authors need reproducible cooperative or competitive multi-robot benchmarks (transport, dispersion, flocking, football) to evaluate collective robot learning.
- Teams needing fast training throughput can run tens of thousands of parallel envs on GPUs to collect rollouts much faster.
- Researchers exploring gradient-based policy optimization or learned models can enable grad_enabled to backpropagate through the simulator.
- Instructors and tutorial authors can use the interactive renderer and Colab notebooks to demonstrate creating, stepping, and visualizing multi-agent environments.
- Multi-robot algorithm developers can implement custom tasks as BaseScenario subclasses and extend them with LIDARs, communication actions, controllers, and dynamics models.
How do you install or deploy this agent?
Install the latest release with pip:
pip install vmasTo install the current master version from source:
git clone https://github.com/proroklab/VectorizedMultiAgentSimulator.git
cd VectorizedMultiAgentSimulator
pip install -e .Only core requirements are installed by default; add optional extras as needed:
# install gymnasium for gymnasium wrappers
pip install vmas[gymnasium]
# install rllib for rllib wrapper
pip install vmas[rllib]
# install rendering dependencies
pip install vmas[render]
# install testing dependencies
pip install vmas[test]
# install all dependencies
pip install vmas[all]Training libraries are installed separately:
pip install benchmarl # For training in BenchMARL
pip install torchrl # For training in TorchRL
pip install "ray[rllib]"==2.1.0 # For training in RLlib. We support versions "ray[rllib]<=2.2,>=1.13"How do you use this agent?
Create and step an environment with make_env:
env = vmas.make_env(
scenario="waterfall", # can be scenario name or BaseScenario class
num_envs=32,
device="cpu", # Or "cuda" for GPU
continuous_actions=True,
wrapper=None, # One of: None, "rllib", "gym", "gymnasium", "gymnasium_vec"
max_steps=None, # Defines the horizon. None is infinite horizon.
seed=None, # Seed of the environment
dict_spaces=False, # By default tuple spaces are used with each element in the tuple being an agent.
grad_enabled=False, # If grad_enabled the simulator is differentiable and gradients can flow from output to input
terminated_truncated=False, # If terminated_truncated the simulator will return separate `terminated` and `truncated` flags
**kwargs # Additional arguments you want to pass to the scenario initialization
)Pass actions to env.step() as a list or dict of tensors shaped [num_envs, action_size]. Render with:
env.render(
mode="rgb_array", # "rgb_array" returns image, "human" renders in display
agent_index_focus=4, # If None keep all agents in camera, else focus camera on specific agent
index=0, # Index of batched environment to render
visualize_when_rgb: bool = False, # Also run human visualization when mode=="rgb_array"
)For headless servers, set up a virtual screen first:
export DISPLAY=':99.0'
Xvfb :99 -screen 0 1400x900x24 > /dev/null 2>&1 &Or wrap the command:
xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>Runnable examples are in examples/use_vmas_env.py, examples/rllib.py, and the per-scenario scripts.
What are this agent's strengths and limitations?
- Pure-PyTorch 2D physics engine steps batched environments in parallel and is documented to scale to tens of thousands of parallel envs.
- Fully differentiable when grad_enabled=True, so gradients can flow from outputs back to inputs.
- Wrappers for OpenAI Gym, Gymnasium, RLlib, TorchRL, and BenchMARL plug into a broad set of RL algorithms.
- Scenarios are modular BaseScenario subclasses with elastic collisions, joints, gravity, LIDAR sensors, and communication actions.
- All OpenAI MPE scenarios are ported and vectorized; papers, Colab notebooks, and interactive rendering aid onboarding.
- Restricted to 2D physics and holonomic point-robot models — no 3D or low-level motor-control simulation.
- README lists Python 3.8–3.11 as the supported range, excluding newer Python versions.
- RLlib integration pins ray[rllib]<=2.2,>=1.13, which may lag the current Ray ecosystem.
- Wrapped Gym/RLlib/TorchRL interfaces are less efficient than the unwrapped native API.
- Headless rendering needs OpenGL/Pyglet plus an Xvfb or EGL setup.
How does this agent compare with similar options?
The README notes that complex vectorized physics engines such as Brax do not scale efficiently with many agents, which motivated VMAS's lightweight custom PyTorch 2D engine; it also describes the implementation as inspired by OpenAI MPE and ports/vectorizes all MPE scenarios, making it a drop-in alternative to MPE.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| VMAS — Vectorized Multi-Agent Simulator This agent | 44 · Major gaps | Library / SDKFree | ★ 628 | 5mo ago | Python | — |
| skrl: Modular Reinforcement Learning Library | 43 · Major gaps | Library / SDKFree | ★ 1.1k | 2d ago | Python | — |
| MARTI: Multi-Agent Reinforced Training & Inference for LLMs | 36 · Major gaps | CLIFree | ★ 572 | 2mo ago | Python | — |
| Hands-On Modern RL | 30 · Major gaps | CLIFree | ★ 4.9k | 13d ago | Jupyter Notebook | — |
How does FollowAgents rate this agent?
Why each dimension lost points
README and code show a locally run PyTorch simulator with no credentials, exfiltration, or destructive defaults, so no red line. least_privilege scores 1: install notes suggest `pip install vmas[all]` plus ray/benchmarl/torchrl, a broad surface without a minimal-install statement; user_confirmation is 0: no high-impact action requires confirmation; data_flow_transparency is 1: README documents I/O spaces and rendering but not systematic data flow (e.g., telemetry); sensitive_data_handling is 0: simulator handles no sensitive data and makes no claim; dependency_security is 1: requirements.txt pins `pyglet<=1.5.27` and `gym-notices==0.0.8` but provides no hashes or vulnerability-scan evidence; external_effects is 1: rendering and interactive control open windows, but file/network side effects are undocumented; rollback is 0: no version rollback or state-recovery mechanism described; source_attribution is 2: GPL-3.0 LICENSE plus arXiv citation and authors give clear attribution.
self_consistency is 2: README descriptions of make_env parameters, I/O spaces, and rendering match test-code calls; dependency_availability is 1: requirements.txt lists numpy/torch/pyglet/gym/gym-notices/six but leaves most unpinned, and gym is deprecated, a availability risk; failure_messages is 1: tests and code contain assertions, but README does not describe error handling or failure-message strategy.
audience_and_scenarios is 2: README targets MARL researchers and scenario developers and lists VMAS and MPE scenarios; capability_boundaries is 2: explicitly states 2D, holonomic motion, and no low-level robot control; trigger_precision is 1: make_env has many parameters but no precise guidance on when to choose a wrapper or grad_enabled; environment_fit is 2: supports CPU/GPU and Gym/Gymnasium/RLlib/TorchRL integration, a broad fit.
information_architecture is 2: README has a table of contents and clear sections; install_notes is 2: pip and source installs plus optional dependency groups; naming_stability is 1: no statement on API naming stability or deprecation policy; examples_and_faq is 2: multiple Colab notebooks and example scripts; known_limitations is 1: mentions wrapper inefficiency but does not systematically list known limitations; license is 2: full GPL-3.0 LICENSE file; versioning_changelog is 1: PyPI version badge but no CHANGELOG; maintenance_responsibility is 1: publisher unverified and README does not state maintenance responsibility or support channels beyond a Discord link.
output_usability is 2: outputs are tensors/dicts with shapes and types documented in README; marginal_value is 2: a vectorized differentiable 2D physics engine has clear incremental value for MARL benchmarking; cost_benefit is 2: lightweight pure-tensor implementation keeps install and run cost relatively low, though optional dependencies are numerous.
claim_traceability is 1: README claims 'Tested' and links CI badges but does not map claims to tests; cross_source_corroboration is 1: test files cover only some scenarios (balance, lidar), not fully matching the all-scenario testing claim; fact_inference_separation is 1: README mixes factual description with performance claims (e.g., 'Faster than physics engines') without separating evidence from inference.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Dependencies are mostly unpinned (except pyglet and gym-notices), and gym is deprecated, limiting install reproducibility.
- README claims all-scenario testing, but visible tests cover only some scenarios; treat coverage claims cautiously.
- Publisher identity is unverified, and maintenance responsibility and support channels are unclear.
- No CHANGELOG or rollback guidance; upgrade risk must be assessed independently.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
What's Changed * [CI] Matplotlib import by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/165 * Fix: correct joint passage reward function (issue #145) by @Benabdellah22 in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/168 * [Copyright] Fix file headers by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/171 * [Experience] Disable gymnasium notice by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/172 New Contributors …View on GitHub ↗ -
Release notes excerpt
What's Changed * [Tutorial] New tutorial on creating a scenario and training it by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/151 * [Feature] Local seed by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/154 * [BugFix] Football u_shoot_multiplier by @matteobettini in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/156 * fix bug of road_traffic senario when using cuda by @Schopenhauer-loves-Hegel in https://github.com/proroklab/VectorizedMultiAgentSimulator/pull/157 …View on GitHub ↗ -
Release notes excerpt
Football revamp  We have just revamped VMAS football! New rewards, observations, options for physical differences and formation spawning and value function plotting. Try it out for some intense N vs M continuous control gameplay. We also provide an heuristic opponent with tunable strengths for precision, decision, and speed. Here are the key new points …View on GitHub ↗