Mobile-Agent: A Family of Powerful GUI Agents
A multi-platform GUI agent family by Alibaba's Tongyi Lab, automating mobile, desktop, and web tasks via multimodal models.
- Source repo
- X-PLUG/MobileAgent
- Stars
- ★ 9.3k
- Last updated
- 3mo ago
- License
- MIT
- Primary language
- Python
- FA score
- 29/100 · Major gaps
At a glance
- How it runs
- Works with
- Platform-specific
- Cost
- Free tier plus a paid hosted plan
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- Individuals: Use natural language commands to automate cross-app mobile tasks like flight searches, price comparisons, and scheduling.
- Not a fit if
- Teams that need fully offline operation or data that never leaves their machines
- Users without an Alibaba Cloud account or unwilling to use Bailian/ModelScope
- Individuals without GPUs large enough to run 32B-class models locally
- Source review
- 29/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
Mobile-Agent is a family of GUI agents developed by Alibaba's Tongyi Lab, covering various versions from single-agent Mobile-Agent-v1 to multi-agent v2, cross-platform v3, and the latest v3.5. It includes the self-developed GUI perception and operation model GUI-Owl (with 1.5 series), released on HuggingFace and ModelScope. The code is fully open-source under the MIT license, supporting Android, PC, and Web environments, with online demos and API services. Several versions have been accepted at top conferences like NeurIPS and ICLR, winning best demo awards.
Mobile-Agent reads screen captures and GUI element information, uses multimodal large models (e.g., Qwen3-VL) for perception, grounding, and decision-making, then outputs operations like clicks, swipes, and text input to control a mobile or PC to complete user tasks. It employs multi-agent frameworks with planning, progress management, reflection, and memory. Key components include GUI-Owl models (providing GUI perception, grounding, and end-to-end actions), Mobile-Agent-v3.5 (multi-platform foundational agents), PC-Agent (hierarchical multi-agent), UI-S1 (semi-online RL), and GUI-Critic-R1 (pre-operative error diagnosis). Online demos and APIs are available on ModelScope and Alibaba Cloud Bailian.
- Individuals: Use natural language commands to automate cross-app mobile tasks like flight searches, price comparisons, and scheduling.
- Office workers: Automate document creation, web research, and office software on PC/Web (e.g., WPS, Edge).
- Developers: Build custom GUI automation tools using GUI-Owl models and the Mobile-Agent framework.
- Enterprises: Integrate via Bailian API for cloud-based automation on cloud phones/desktops.
- Researchers: Reproduce or improve methods with open code, datasets, and models.
- Testers: Automate UI testing for mobile or desktop apps.
How do you install or deploy this agent?
Installation varies by version, but generally you clone the repo and install dependencies:\n``bash\ngit clone https://github.com/X-PLUG/MobileAgent\ncd MobileAgent\n# Follow the README in the specific subdirectory (e.g., Mobile-Agent-v3.5)\n``\nRequires Python, PyTorch, CUDA, and optionally a GPU. Model weights are downloaded from HuggingFace or ModelScope.
How do you use this agent?
Refer to each version's subdirectory README. Basic usage: load the GUI-Owl model, then run the main script with your natural language instruction. For example, in the Mobile-Agent-v3.5 directory, use the model API or local inference, then start the agent and input a task description. Online demos: ModelScope (https://modelscope.cn/studios/MobileAgentTest/computer_use) and Alibaba Cloud Bailian (https://bailian.console.aliyun.com/next?tab=demohouse#/experience/adk-computer-use/pc).
What are this agent's strengths and limitations?
- Comprehensive coverage: Supports mobile, PC, and web platforms with model sizes from 2B to 235B and both Instruct and Thinking variants.
- SOTA performance: Achieves state-of-the-art results on 20+ GUI benchmarks, with open evaluation code.
- Academic recognition: Multiple versions accepted at NeurIPS and ICLR, and won best demo awards.
- Online demos and APIs for quick trial and integration.
- Dependence on Alibaba ecosystem: Some demos and APIs are based on ModelScope and Bailian, potentially requiring an Alibaba account.
- High hardware requirements: Running large models needs substantial GPU memory (e.g., for 32B).
- Fragmented documentation: Instructions are spread across subdirectories, requiring manual research.
- Some features (like video demos) may not be fully open; actual deployment may need extra configuration.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Mobile-Agent: A Family of Powerful GUI Agents This agent | 29 · Major gaps | Hosted serviceFreemium | ★ 9.3k | 3mo ago | Python | — |
| OpenAdapt | 86 · Good | CLIFreemium | ★ 1.8k | today | Python | — |
| Clawd Cursor | 74 · Some gaps | MCP serverFree + model costs | ★ 404 | today | TypeScript | Codex · Claude Code |
| OpenGUI Mobile Operator | 45 · Major gaps | CLIFree + model costs | ★ 1.8k | today | HTML | Codex · Claude Code · OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
The repository provides no documentation on permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust criteria are unsupported, hence score 0.
self_consistency: 1 - Naming and descriptions across versions (v1, v2, v3, v3.5) and model series (GUI-Owl) are consistent, but detailed architecture docs are missing, limiting evidence. dependency_availability: 1 - Model weights links on HuggingFace and ModelScope are provided, but no dependency list or installation verification. failure_messages: 0 - No documentation on error handling or failure messages.
audience_and_scenarios: 2 - README clearly targets researchers and developers, with demos for phone, PC, and web scenarios. capability_boundaries: 1 - Capabilities of each version are described, but boundaries or limitations are not explicit. trigger_precision: 0 - No details on trigger mechanisms or instruction parsing. environment_fit: 1 - Mentions cloud phone and online demos, but no local deployment environment requirements.
information_architecture: 2 - README is well-structured with sections for news, results, features, series, demos, and citation. install_notes: 1 - No explicit installation steps, only online demos and API mentions. naming_stability: 2 - Version naming is consistent (v1, v2, v3, v3.5), and model series naming is clear. examples_and_faq: 2 - Multiple video demos and example tasks are provided, but no FAQ. known_limitations: 1 - No explicit list of known limitations, only some mentions in news. license: 2 - MIT license provided. versioning_changelog: 1 - News updates exist, but no formal changelog. maintenance_responsibility: 2 - Clearly attributed to Alibaba Group and Tongyi Lab, with contact info.
output_usability: 1 - Demo videos show outputs, but no output format or integration guide. marginal_value: 2 - Multiple versions and models cover various scenarios, providing research value. cost_benefit: 1 - No performance benchmarks or resource consumption data, making cost-benefit hard to assess.
claim_traceability: 2 - Paper links and model weights are provided, enabling traceability. cross_source_corroboration: 1 - Only relies on own repository, no third-party verification. fact_inference_separation: 1 - News and results sections mix facts and inferences without clear separation.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- The repository lacks any security or permission documentation; assess risks before use.
- Dependency and installation instructions are incomplete, potentially affecting reproducibility.
- Known limitations are not explicitly listed, which may hide potential issues.