Automation & Ops mobile-automation android ios adb natural-language-ui data-scraping

mobile-use: Automate Your Phone with Natural Language

An open-source AI agent that controls real Android/iOS apps using natural language, from sending messages to extracting structured data.

View on GitHub ↗ Quick start ↓

Source repo: minitap-ai/mobile-use
Stars: ★ 3.3k
Last updated: today
License: Apache-2.0
Primary language: Python
FA score: 33/100 · Major gaps

At a glance

How it runs: CLISelf-hosted service
Works with: Universal · cross-platformOpenAI API · Claude API
Cost: Free software; you pay for model usage
Setup effort: Medium · a few setup steps
You'll need: Python 3.12+uvADBDocker (optional)Shell / CLINetwork accessLocal filesystem
Typical use: A user on an Android phone asks the agent to open Gmail and list unread emails with sender and subject lines.
Not a fit if: Users who need to control physical iPhones; only iOS simulators work
Anyone automating games, which lack accessibility tree data
Non-technical users unwilling to set up ADB, env vars and API keys
Source review: 33/100 · Major gaps 8 safety controls not found

What does this agent do, and when should you use it?

mobile-use is a powerful open-source AI agent that controls your Android or iOS device using natural language. It understands commands and interacts with the UI to perform tasks, from sending messages to navigating complex apps. The agent is UI-aware, using accessibility tree data to navigate (though it has limited effectiveness with games). It supports data scraping, structuring extracted information into formats like JSON based on natural language descriptions. It is extensible and customizable, supporting multiple LLM providers including OpenAI, Google, xAI, OpenRouter, and MiniMax. mobile-use claims to be the first agentic framework to achieve 100% on the AndroidWorld benchmark. Deployment is flexible: you can use their cloud platform, run a Docker quickstart for Android, or set up from source for development.

mobile-use takes natural language commands and translates them into device operations. It connects to Android devices via ADB or iOS simulators via IDB, retrieves the UI hierarchy, and performs actions like taps and swipes. The main entry point is python ./minitap/mobile_use/main.py "<command>" [--output-description <format>], which can format output as structured data like JSON. It supports configuration through llm-config.override.jsonc to switch providers, and environment variables like OPENAI_API_KEY, ANTHROPIC_API_KEY, or MINIMAX_API_KEY. The system is built on a multi-agent architecture that decomposes tasks for better accuracy.

A user on an Android phone asks the agent to open Gmail and list unread emails with sender and subject lines.
A QA tester automates a series of UI actions on an iOS simulator using natural language, without writing code.
A data analyst extracts specific fields (e.g., product prices, contact info) from an app and outputs them as JSON.
A developer debugging their Android app guides the AI to perform a specific sequence of taps on an emulator.
A non-technical business user automates repetitive data collection tasks on a mobile app.
A researcher replicates the 100% accuracy result on the AndroidWorld benchmark for their own experiments.

How do you install or deploy this agent?

Clone the repository: git clone https://github.com/minitap-ai/mobile-use.git && cd mobile-use. 2. Copy .env.example to .env and add your API keys. 3. (Optional) Copy llm-config.override.template.jsonc to llm-config.override.jsonc to customize models. 4. Install [uv](https://github.com/astral-sh/uv). 5. Create a virtual environment: uv venv. 6. Activate it (macOS/Linux: source .venv/bin/activate, Windows: .venv\Scripts\activate). 7. Install dependencies: uv sync. For Android, you need [ADB](https://developer.android.com/studio/releases/platform-tools) and USB debugging enabled. For iOS simulators (macOS only), install [Xcode](https://developer.apple.com/xcode/) and [fb-idb](https://fbidb.io/docs/installation/) via brew install idb-companion.

How do you use this agent?

First, connect your device. For Android: connect via USB with USB debugging enabled, or launch an emulator. For iOS: launch a simulator and ensure idb_companion is in your PATH. Then run a command, e.g., python ./minitap/mobile_use/main.py "Go to settings and tell me my current battery level". To extract structured data, add --output-description, e.g., python ./minitap/mobile_use/main.py "Open Gmail, find all unread emails, and list their sender and subject line" --output-description "A JSON list of objects, each with 'sender' and 'subject' keys". If no model is configured, you will be prompted to choose one. Accept any ADB connection prompts on your device.

What are this agent's strengths and limitations?

Pros

First to achieve 100% accuracy on the AndroidWorld benchmark, demonstrating superior agentic task decomposition.
Supports multiple LLM providers (OpenAI, Google, xAI, OpenRouter, MiniMax) and OpenAI-compatible endpoints, avoiding vendor lock-in.
Offers a Docker quickstart script for simple deployment on Android.
Supports both Android and iOS simulators, expanding testing scenarios.
Provides a cloud platform and a clean CLI for easy adoption.

Limitations

Physical iOS devices are not supported yet, only simulators.
Limited effectiveness with games due to lack of accessibility tree data.
Requires external LLM API keys, incurring costs and network dependency.
Docker quickstart is Android-only; iOS requires manual setup on macOS.
Setup requires technical knowledge including ADB, environment variables, and package management.

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

Agent	Source review	Form / cost	Stars	Updated	Language	Full support on
mobile-use: Automate Your Phone with Natural Language This agent	33 · Major gaps	CLIFree + model costs	★ 3.3k	today	Python	OpenAI API · Claude API
Open-AutoGLM Phone Agent	55 · Major gaps	CLIFree + model costs	★ 26k	7mo ago	Python	—
ClickClickClick	23 · Major gaps	CLIFree + model costs	★ 714	7mo ago	Python	OpenAI API
OpenGUI Mobile Operator	45 · Major gaps	CLIFree + model costs	★ 1.8k	today	HTML	Codex · Claude Code · OpenAI API · Claude API

How does FollowAgents rate this agent?

FollowAgents source review · FARS-2.1

Major gaps

33/ 100 5-point scale 1.7 / 5

Trust 0/29

Reliability 5/14

Adaptability 9/18

Convention 9/18

Effectiveness 7/13

Verifiability 3/8

Why each dimension lost points

Trust0 / 29 · 0.0/5

Evidence shows: The repository does not provide any explicit permission model, user confirmation mechanism, data flow transparency, sensitive data handling, dependency security audit, external effect control, rollback mechanism, or source attribution. All trust criteria are unsupported, hence all scores are 0.

Reliability5 / 14 · 1.8/5

Evidence shows: README and pyproject.toml provide basic consistency information such as version, dependency list, and test configuration, but no detailed error handling or recovery mechanisms. Dependency list is complete but no availability guarantee. Failure messages: README provides some troubleshooting but not comprehensive. Hence each score is 1.

Adaptability9 / 18 · 2.5/5

Evidence shows: README clearly identifies target audience (developers and users) and multiple use cases (natural language control, data scraping), score 2. Capability boundaries: only mentions limited support for games, not detailed, score 1. Trigger precision: provides CLI examples but no trigger conditions or precision, score 1. Environment fit: supports multiple devices and platforms but no detailed configuration options, score 2.

Convention9 / 18 · 2.5/5

Evidence shows: README is well-structured with sections for installation, usage, architecture, etc., score 2. Install notes are detailed including Docker and manual setup, score 2. Naming stability: project name and version are clear but no API stability statement, score 1. Examples and FAQ: multiple examples and troubleshooting provided, score 2. Known limitations: only mentions limited game support, score 1. License: Apache-2.0, score 2. Versioning and changelog: version number present but no changelog, score 1. Maintenance responsibility: authors listed but no explicit maintenance responsibility, score 1.

Effectiveness7 / 13 · 2.7/5

Evidence shows: Output usability: provides structured output examples (e.g., JSON), score 2. Marginal value: provides unique features (e.g., data scraping), score 2. Cost-benefit: no performance or cost data, score 1.

Verifiability3 / 8 · 1.9/5

Evidence shows: README claims 100% on AndroidWorld benchmark but no verifiable test results or reproduction steps, score 1. Cross-source corroboration: provides paper and benchmark links but no independent verification, score 1. Fact-inference separation: README mixes facts and inferences without clear distinction, score 1.

Risks and how to mitigate them

Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
No permission model or user confirmation mechanism provided, may execute high-risk actions.
Dependency list includes many third-party libraries but no security audit or vulnerability scanning.
Benchmark claims in README lack reproducible test steps.

Evidence confidence: Low Reviewed Aug 09, 2026 Reviewed revision cc20e3d52bf6 New commits since this review; the score may not cover them

Review evidence README.md LICENSE pyproject.toml workflows/deployment.yml workflows/feature.yml mobile_use/test_idb_client.py

See the full review method →

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

v3.3.0 Release v3.3.0 2026-01-12

Release notes excerpt

What's Changed * fix: normalizing input for press key tool by @cguiguet in https://github.com/minitap-ai/mobile-use/pull/167 * feat(cortex): add form filling guidelines to prevent field misidentification by @cguiguet in https://github.com/minitap-ai/mobile-use/pull/168 * fix: Enter keycode implementation, better decision making for data related automations by @plfavreau in https://github.com/minitap-ai/mobile-use/pull/169 Full Changelog: https://github.com/minitap-ai/mobile-use/compare/v3.2.1...v3.3.0
View on GitHub ↗
v2.9.0 Release v2.9.0 2025-11-15

Release notes excerpt

What's Changed * feat(sdk): manual platform task can have a task name for tracing by @KYN4 in https://github.com/minitap-ai/mobile-use/pull/106 * docs: clarify API key and task configuration in platform example by @plfavreau in https://github.com/minitap-ai/mobile-use/pull/109 * fix: focus and input text now outputs the focus method to the agents by @plfavreau in https://github.com/minitap-ai/mobile-use/pull/107 * feat: updating Langgraph to alpha by @plfavreau in https://github.com/minitap-ai/mobile-use/pull/108 …
View on GitHub ↗
v2.6.0 Release v2.6.0 2025-10-20

Release notes excerpt

What's Changed * feat: bump to v2.5.0 by @cguiguet in https://github.com/minitap-ai/mobile-use/pull/79 * Fix iOS device detection by checking for Darwin instead of Linux by @yulingtianxia in https://github.com/minitap-ai/mobile-use/pull/80 * feat: prefer minitap default llm profile over openai config by @plfavreau in https://github.com/minitap-ai/mobile-use/pull/81 * feat: get streaming status endpoint by @plfavreau in https://github.com/minitap-ai/mobile-use/pull/82 * fix(sdk): adb host forwarded to maestro server by @KYN4 in https://github.com/minitap-ai/mobile-use/pull/83 …
View on GitHub ↗

FAQ

Which LLM providers are supported?

The agent supports OpenAI, Google, xAI, OpenRouter, MiniMax, and any OpenAI-compatible endpoint by setting OPENAI_BASE_URL.

Can I use it on a physical iPhone?

No, physical iOS devices are not yet supported; only iOS simulators on macOS are supported.

What are the costs involved?

The open-source framework is free, but you need your own API keys for LLM providers and pay for their usage.

How do I troubleshoot connection issues?

Ensure the device and computer are on the same Wi-Fi network and check firewall settings. If IP detection fails, use adb shell ip addr show up to find the interface and pass --interface to the script.

mobile-use: Automate Your Phone with Natural Language

At a glance

What does this agent do, and when should you use it?

How do you install or deploy this agent?

How do you use this agent?

What are this agent's strengths and limitations?

How does this agent compare with similar options?

How does FollowAgents rate this agent?

Recent releases

FAQ

Related agents

Open-AutoGLM Phone Agent

ClickClickClick

OpenGUI Mobile Operator

ESP32 iPhone Operator