MobileGym
A programmable mobile simulator for deterministic GUI-agent evaluation and online RL training.
What does this agent do, and when should you use it?
MobileGym is a browser-hosted Android-like simulation platform for mobile GUI-agent research. It combines a React/Vite simulator, a Python-and-Playwright benchmark runtime in bench_env, and an online RL pipeline in mobilegym-rl. The environment is exposed as structured JSON that can be read, patched, snapshotted, restored, and cloned; task judges evaluate resulting state and side effects programmatically. The repository includes 28 simulated apps, 416 parameterized task templates, and a 256-task test split, while its apps never connect to real services, accounts, or funds. It fits teams seeking repeatable evaluation and scalable browser rollouts, not teams that need automation against real Android apps or live consumer accounts.
The frontend composes a simulated phone from apps/, system/, and os/; bench_env drives it through Playwright using __SIM__, __OS__, and __SIM_INPUT__ to receive screenshots and issue tap, type, swipe, back, home, wait, drag, and complete actions. A task class supplies description, JSON state injection in setup, and deterministic check_goals() or get_answer() logic; python -m bench_env.run can execute individual tasks, suites, splits, parallel evaluations, pass@k, and retries. The agent package includes autoglm, uitars, venus, gui_owl, generic, generic_v2, mai_ui, and human adapters, and completed runs can be inspected in run_explorer.html with screenshots, action annotations, prompts, and model responses. mobilegym-rl/ contains the online RL training code.
- A mobile-agent researcher comparing models on WeChat, Alipay, or cross-app tasks with identical initial state for every trial.
- An RL team cloning task state into many browser instances to run parallel GRPO-style rollouts.
- A benchmark maintainer adding a simulated-app task with setup state injection and a deterministic check_goals() judge.
- A model-integration engineer evaluating a GUI model through an existing AutoGLM, UI-TARS, UI-Venus, GUI-Owl, or unified-JSON adapter.
- A developer using human mode to replay a task, debug its judge, and inspect the resulting trajectory in a browser.
What are this agent's strengths and limitations?
- It manages the full environment as structured JSON, enabling snapshot, restore, injection, and cloning rather than relying only on screenshots or accessibility trees.
- Tasks use programmatic check_goals() / get_answer() judges that can assess both outcomes and unexpected side effects without VLM judging.
- Its browser-hosted instances are designed for parallel rollouts; the README documents resource and startup figures for 256 instances on one server.
- Apps, OS functionality, benchmark tasks, and model adapters are separated into explicit repository layers, with documented locations for new apps and adapters.
- The apps are research surrogates rather than live integrations, so the platform cannot validate authentication, backend behavior, or payments in real production apps.
- Local use requires Node.js, Python, and Playwright Chromium; high-concurrency operation additionally requires nginx and may require Linux inotify tuning.
- The companion dataset is about 1.9 GB and the data/content license is CC BY-NC 4.0, which may limit commercial adoption.
- A new model must fit an existing adapter schema or require an adapter implemented and registered under bench_env/agent/.
How do you install or deploy this agent?
Prerequisites: Node.js >=22, Python >=3.11, a shell, and network access. Run: git clone https://github.com/Purewhiter/mobilegym.git; cd mobilegym; npm install; pip install -r bench_env/requirements.txt; playwright install chromium. The optional companion dataset is about 1.9 GB: curl -L -o mobilegym-data.tar.gz https://github.com/Purewhiter/mobilegym/releases/download/data-v0.1.0/mobilegym-data-v0.1.0.tar.gz; tar -xzf mobilegym-data.tar.gz. Simulator keys are optional; the canonical benchmark does not require credentials.
How do you use this agent?
First run npm run build && npm run preview -- --port 4173, then verify the setup in manual mode: python -m bench_env.run --task-id wechat.ReadMyWxid --agent human --env-url http://localhost:4173. List tasks with python -m bench_env.run --list. For the full test split: python -m bench_env.run --split test --parallel 8 --env-url http://localhost:4173 --agent autoglm --model-name autoglm-phone-9b. For higher concurrency, install nginx, run npm run build and ./scripts/server/start_nginx_gateway.sh, then use https://localhost:4180 as the environment URL.
How does this agent compare with similar options?
Compared with real-device or conventional emulator workflows, MobileGym is designed around writable, resettable, cloneable JSON state and deterministic state-diff evaluation. Its README identifies AppWorld, WebArena, VisualWebArena, AndroidWorld, AndroidLab, and A3 as inspirations.