Mobile-Agent: A Family of Powerful GUI Agents
A multi-platform GUI agent family by Alibaba's Tongyi Lab, automating mobile, desktop, and web tasks via multimodal models.
The repository provides no documentation on permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust criteria are unsupported, hence score 0.
self_consistency: 1 - Naming and descriptions across versions (v1, v2, v3, v3.5) and model series (GUI-Owl) are consistent, but detailed architecture docs are missing, limiting evidence. dependency_availability: 1 - Model weights links on HuggingFace and ModelScope are provided, but no dependency list or installation verification. failure_messages: 0 - No documentation on error handling or failure messages.
audience_and_scenarios: 2 - README clearly targets researchers and developers, with demos for phone, PC, and web scenarios. capability_boundaries: 1 - Capabilities of each version are described, but boundaries or limitations are not explicit. trigger_precision: 0 - No details on trigger mechanisms or instruction parsing. environment_fit: 1 - Mentions cloud phone and online demos, but no local deployment environment requirements.
information_architecture: 2 - README is well-structured with sections for news, results, features, series, demos, and citation. install_notes: 1 - No explicit installation steps, only online demos and API mentions. naming_stability: 2 - Version naming is consistent (v1, v2, v3, v3.5), and model series naming is clear. examples_and_faq: 2 - Multiple video demos and example tasks are provided, but no FAQ. known_limitations: 1 - No explicit list of known limitations, only some mentions in news. license: 2 - MIT license provided. versioning_changelog: 1 - News updates exist, but no formal changelog. maintenance_responsibility: 2 - Clearly attributed to Alibaba Group and Tongyi Lab, with contact info.
output_usability: 1 - Demo videos show outputs, but no output format or integration guide. marginal_value: 2 - Multiple versions and models cover various scenarios, providing research value. cost_benefit: 1 - No performance benchmarks or resource consumption data, making cost-benefit hard to assess.
claim_traceability: 2 - Paper links and model weights are provided, enabling traceability. cross_source_corroboration: 1 - Only relies on own repository, no third-party verification. fact_inference_separation: 1 - News and results sections mix facts and inferences without clear separation.
- The repository lacks any security or permission documentation; assess risks before use.
- Dependency and installation instructions are incomplete, potentially affecting reproducibility.
- Known limitations are not explicitly listed, which may hide potential issues.
What does this agent do, and when should you use it?
Mobile-Agent is a family of GUI agents developed by Alibaba's Tongyi Lab, covering various versions from single-agent Mobile-Agent-v1 to multi-agent v2, cross-platform v3, and the latest v3.5. It includes the self-developed GUI perception and operation model GUI-Owl (with 1.5 series), released on HuggingFace and ModelScope. The code is fully open-source under the MIT license, supporting Android, PC, and Web environments, with online demos and API services. Several versions have been accepted at top conferences like NeurIPS and ICLR, winning best demo awards.
Mobile-Agent reads screen captures and GUI element information, uses multimodal large models (e.g., Qwen3-VL) for perception, grounding, and decision-making, then outputs operations like clicks, swipes, and text input to control a mobile or PC to complete user tasks. It employs multi-agent frameworks with planning, progress management, reflection, and memory. Key components include GUI-Owl models (providing GUI perception, grounding, and end-to-end actions), Mobile-Agent-v3.5 (multi-platform foundational agents), PC-Agent (hierarchical multi-agent), UI-S1 (semi-online RL), and GUI-Critic-R1 (pre-operative error diagnosis). Online demos and APIs are available on ModelScope and Alibaba Cloud Bailian.
- Individuals: Use natural language commands to automate cross-app mobile tasks like flight searches, price comparisons, and scheduling.
- Office workers: Automate document creation, web research, and office software on PC/Web (e.g., WPS, Edge).
- Developers: Build custom GUI automation tools using GUI-Owl models and the Mobile-Agent framework.
- Enterprises: Integrate via Bailian API for cloud-based automation on cloud phones/desktops.
- Researchers: Reproduce or improve methods with open code, datasets, and models.
- Testers: Automate UI testing for mobile or desktop apps.
What are this agent's strengths and limitations?
- Comprehensive coverage: Supports mobile, PC, and web platforms with model sizes from 2B to 235B and both Instruct and Thinking variants.
- SOTA performance: Achieves state-of-the-art results on 20+ GUI benchmarks, with open evaluation code.
- Academic recognition: Multiple versions accepted at NeurIPS and ICLR, and won best demo awards.
- Online demos and APIs for quick trial and integration.
- Dependence on Alibaba ecosystem: Some demos and APIs are based on ModelScope and Bailian, potentially requiring an Alibaba account.
- High hardware requirements: Running large models needs substantial GPU memory (e.g., for 32B).
- Fragmented documentation: Instructions are spread across subdirectories, requiring manual research.
- Some features (like video demos) may not be fully open; actual deployment may need extra configuration.
How do you install or deploy this agent?
Installation varies by version, but generally you clone the repo and install dependencies:\n``bash\ngit clone https://github.com/X-PLUG/MobileAgent\ncd MobileAgent\n# Follow the README in the specific subdirectory (e.g., Mobile-Agent-v3.5)\n``\nRequires Python, PyTorch, CUDA, and optionally a GPU. Model weights are downloaded from HuggingFace or ModelScope.
How do you use this agent?
Refer to each version's subdirectory README. Basic usage: load the GUI-Owl model, then run the main script with your natural language instruction. For example, in the Mobile-Agent-v3.5 directory, use the model API or local inference, then start the agent and input a task description. Online demos: ModelScope (https://modelscope.cn/studios/MobileAgentTest/computer_use) and Alibaba Cloud Bailian (https://bailian.console.aliyun.com/next?tab=demohouse#/experience/adk-computer-use/pc).