MobileGym
A programmable mobile simulator for deterministic GUI-agent evaluation and online RL training.
- Source repo
- Purewhiter/mobilegym
- Stars
- ★ 856
- Last updated
- 1mo ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 67/100 · Some gaps
At a glance
- How it runs
- Works with
- Portable with changes
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A mobile-agent researcher comparing models on WeChat, Alipay, or cross-app tasks with identical initial state for every trial.
- Not a fit if
- Teams that need automation against real Android apps or live accounts
- Teams without GPU or high-concurrency server resources
- Source review
- 67/100 · Some gaps
What does this agent do, and when should you use it?
MobileGym is a browser-hosted Android-like simulation platform for mobile GUI-agent research. It combines a React/Vite simulator, a Python-and-Playwright benchmark runtime in bench_env, and an online RL pipeline in mobilegym-rl. The environment is exposed as structured JSON that can be read, patched, snapshotted, restored, and cloned; task judges evaluate resulting state and side effects programmatically. The repository includes 28 simulated apps, 416 parameterized task templates, and a 256-task test split, while its apps never connect to real services, accounts, or funds. It fits teams seeking repeatable evaluation and scalable browser rollouts, not teams that need automation against real Android apps or live consumer accounts.
The frontend composes a simulated phone from apps/, system/, and os/; bench_env drives it through Playwright using __SIM__, __OS__, and __SIM_INPUT__ to receive screenshots and issue tap, type, swipe, back, home, wait, drag, and complete actions. A task class supplies description, JSON state injection in setup, and deterministic check_goals() or get_answer() logic; python -m bench_env.run can execute individual tasks, suites, splits, parallel evaluations, pass@k, and retries. The agent package includes autoglm, uitars, venus, gui_owl, generic, generic_v2, mai_ui, and human adapters, and completed runs can be inspected in run_explorer.html with screenshots, action annotations, prompts, and model responses. mobilegym-rl/ contains the online RL training code.
- A mobile-agent researcher comparing models on WeChat, Alipay, or cross-app tasks with identical initial state for every trial.
- An RL team cloning task state into many browser instances to run parallel GRPO-style rollouts.
- A benchmark maintainer adding a simulated-app task with setup state injection and a deterministic check_goals() judge.
- A model-integration engineer evaluating a GUI model through an existing AutoGLM, UI-TARS, UI-Venus, GUI-Owl, or unified-JSON adapter.
- A developer using human mode to replay a task, debug its judge, and inspect the resulting trajectory in a browser.
How do you install or deploy this agent?
Prerequisites: Node.js >=22, Python >=3.11, a shell, and network access. Run: git clone https://github.com/Purewhiter/mobilegym.git; cd mobilegym; npm install; pip install -r bench_env/requirements.txt; playwright install chromium. The optional companion dataset is about 1.9 GB: curl -L -o mobilegym-data.tar.gz https://github.com/Purewhiter/mobilegym/releases/download/data-v0.1.0/mobilegym-data-v0.1.0.tar.gz; tar -xzf mobilegym-data.tar.gz. Simulator keys are optional; the canonical benchmark does not require credentials.
How do you use this agent?
First run npm run build && npm run preview -- --port 4173, then verify the setup in manual mode: python -m bench_env.run --task-id wechat.ReadMyWxid --agent human --env-url http://localhost:4173. List tasks with python -m bench_env.run --list. For the full test split: python -m bench_env.run --split test --parallel 8 --env-url http://localhost:4173 --agent autoglm --model-name autoglm-phone-9b. For higher concurrency, install nginx, run npm run build and ./scripts/server/start_nginx_gateway.sh, then use https://localhost:4180 as the environment URL.
What are this agent's strengths and limitations?
- It manages the full environment as structured JSON, enabling snapshot, restore, injection, and cloning rather than relying only on screenshots or accessibility trees.
- Tasks use programmatic check_goals() / get_answer() judges that can assess both outcomes and unexpected side effects without VLM judging.
- Its browser-hosted instances are designed for parallel rollouts; the README documents resource and startup figures for 256 instances on one server.
- Apps, OS functionality, benchmark tasks, and model adapters are separated into explicit repository layers, with documented locations for new apps and adapters.
- The apps are research surrogates rather than live integrations, so the platform cannot validate authentication, backend behavior, or payments in real production apps.
- Local use requires Node.js, Python, and Playwright Chromium; high-concurrency operation additionally requires nginx and may require Linux inotify tuning.
- The companion dataset is about 1.9 GB and the data/content license is CC BY-NC 4.0, which may limit commercial adoption.
- A new model must fit an existing adapter schema or require an adapter implemented and registered under bench_env/agent/.
How does this agent compare with similar options?
Compared with real-device or conventional emulator workflows, MobileGym is designed around writable, resettable, cloneable JSON state and deterministic state-diff evaluation. Its README identifies AppWorld, WebArena, VisualWebArena, AndroidWorld, AndroidLab, and A3 as inspirations.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| MobileGym This agent | 67 · Some gaps | CLIFree + model costs | ★ 856 | 1mo ago | Python | — |
| Sotopia | 46 · Major gaps | Library / SDKFree + model costs | ★ 342 | 10mo ago | Python | OpenAI API |
| RLinf RL Infrastructure | 54 · Major gaps | Library / SDKFree | ★ 5.6k | 2d ago | Python | — |
| MARO Resource Optimization Platform | 48 · Major gaps | Library / SDKFree | ★ 926 | 3y ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
The README and getting-started guide say the simulated apps do not connect to real services, accounts, or funds, and describe state snapshots, cloning, and rollback. Those are direct controls for privilege and external effects. Optional map, weather, and LLM API keys are supported, and the browser can make outbound requests, but the packet gives limited detail on secret protection and sensitive-data handling. Dependencies and version ranges are listed, but the packet does not establish a dependency security policy or lockfile contents. The Apache license, separate data license, and disclaimer provide attribution context; publisher identity and maintenance responsibility remain unclear.
The documentation provides installation, startup, evaluation, and scaling paths and calls out missing dataset content, gateway certificates, concurrency limits, and model-server capacity. The README includes news dated after the packet's current date, and the packet does not establish how its v0.1.0 release claims relate to the specified revision, so self-consistency is weak. The supplied material gives little evidence about runtime error handling or user-facing failure messages. The lack of executed tests was not treated as a deduction.
The packet clearly targets mobile GUI agent evaluation, research, and online RL, with examples for manual debugging, single tasks, small runs, and large parallel workloads. Browser simulation, structured state, extension contracts, and multiple agent adapters support a broad range of use cases. It also states that agents observe screenshots, users provide model endpoints, and datasets or optional external services affect the experience. Environment requirements are stated, though high concurrency requires Linux tuning and local Chromium and service configuration.
The README, getting-started guide, platform specifications, task-authoring documents, and benchmark documentation form a clear information structure. Installation steps, runtime requirements, commands, and known issues are well covered. The repository includes an Apache-2.0 license and identifies a separate data license. News and the v0.1.0 release note provide some version history, but the packet does not show a complete changelog or clear maintainer/support responsibility. Naming is generally understandable, though the packet does not independently corroborate some service-mode, capacity, and version claims.
Programmable simulation state, code-level judging, parallel rollouts, task organization, and multiple agent interfaces offer concrete value for mobile GUI research. Command examples report results and point to run trajectories, supporting practical use. The large dataset, Node/Python/Chromium setup, model endpoint, and gateway add meaningful setup cost, so the tradeoff is weaker for lightweight use or work that does not need a simulated mobile environment. Performance and transfer figures are primarily README claims in this packet and are not independently verified here.
The README links claims to repository paths for tasks, architecture, installation, known issues, and authoring specifications, and provides benchmark and training figures. The source excerpts do not cover most of the implementation and documentation those references point to, limiting cross-source corroboration. The README mixes simulator boundaries and resource estimates with paper and experiment claims; some are labeled as experimental results, but the packet does not include underlying logs or enough paper detail to verify them. Scores credit only what is traceable in the supplied material.
- The README contains news dated after the packet date, and the packet does not establish how those entries relate to the specified revision.
- When configuring optional external API keys or model endpoints, manage secrets and outbound requests for the deployment environment; the packet does not fully describe data retention or secret handling.
- The roughly 1.9 GB dataset, Chromium, model service, and high-concurrency gateway add setup and operational cost. Concurrency also depends on host and inference-backend capacity.
- Performance, success-rate, and sim-to-real figures are not independently verified by this static packet.
Maintain this project? Add the FARS badge to your README
[](https://followagents.com/en/agents/mobilegym)<a href="https://followagents.com/en/agents/mobilegym"><img src="https://followagents.com/badge/mobilegym.svg" alt="FARS review"></a>Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
data-v0.1.0 MobileGym data v0.1.0
-
Release notes excerpt
First stable release. Ships the online RL training code (mobilegym-rl).View on GitHub ↗
FAQ
Will it operate my real WeChat, Alipay, or eBay account?
Are API keys required?
Can I evaluate my own GUI model?
What changes for high-concurrency execution?
Thanks — we use this to improve the page.