OpenGUI Mobile Operator
An Android GUI framework that lets AI inspect screens, plan steps, and operate real mobile apps.
- Source repo
- Core-Mate/OpenGUI
- Stars
- ★ 1.8k
- Last updated
- 19d ago
- License
- NOASSERTION
- Primary language
- HTML
- FA score
- 45/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformCodex · Claude Code · OpenAI API · Claude API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A developer who needs an attached Android device to inspect its current app screen and return a summary.
- Not a fit if
- Users without an Android phone or emulator and USB debugging plus AccessibilityService access
- Users wanting a turnkey tool with no model API keys or backend setup
- Source review
- 45/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
OpenGUI is a mobile GUI agent framework for Android phones and emulators that understands live screens and operates app interfaces. Its backend uses a Plan Supervisor for task state, an Executor Graph for screenshot, vision, action, and call-user loops, and a Summarizer for structured results. The Android client keeps a standby connection through StandbySocketManager.kt and executes actions through GestureService.kt using AccessibilityService. Tasks can arrive through a local CLI, REST API, or Feishu, Telegram, and Discord dispatch paths to standby devices. The repository also documents bootstrap and remote-control workflows for Claude Code and Codex, with separate routing for planning and VLM execution.
A submitted goal is broken into executable steps by Plan Supervisor. Executor Graph reads the current Android UI and runs screenshot, vision, action, and call-user loops against live device state; execution review and retry feed back into supervision, and Summarizer returns a structured result. server/apps/backend/src/common/ws/standby.gateway.ts handles standby dispatch, StandbySocketManager.kt maintains the Android connection, and GestureService.kt executes Android-side gestures. Operators can list devices with pnpm opengui -- devices --json, dispatch work with pnpm opengui -- do "..." --json, and inspect a run with pnpm opengui -- status <executionId> --json.
- A developer who needs an attached Android device to inspect its current app screen and return a summary.
- A researcher collecting recent topic-related posts from X, Reddit, or Hacker News through a real Android phone.
- An operations team dispatching Android jobs to standby devices from Feishu, Telegram, Discord, or a REST API.
- An automation team running repetitive Android app workflows and receiving a structured execution result.
- A Codex or Claude Code user who wants guided local setup, APK build, installation, and debugging while handling only phone-side approvals.
How do you install or deploy this agent?
Prepare an Android phone or emulator, enable USB debugging and AccessibilityService, and provide model API keys for real task execution. Start the backend with cd server then ./start.sh; start the Android client with cd client then ./start.sh. The backend setup includes Docker-based PostgreSQL and Redis setup. For Discord dispatch, configure DISCORD_BOT_TOKEN; when it is empty, the backend starts without Discord.
How do you use this agent?
After the backend and client are running, from server run pnpm opengui -- devices --json to inspect available devices, pnpm opengui -- do "Observe the current Android screen and summarize what you see" --json to submit a first task, and pnpm opengui -- status <executionId> --json to track it. Codex and Claude Code can also use skills/open-gui-bootstrap/SKILL.md for setup or skills/open-gui-remote-control/SKILL.md for local-CLI Android task dispatch.
What are this agent's strengths and limitations?
- It is designed around a main graph plus an executor subgraph, with Plan Supervisor retaining task state, continuation, and review rather than only a short interactive loop.
- The device path is explicit: StandbySocketManager.kt maintains a standby connection and GestureService.kt performs actions through Android AccessibilityService.
- It provides local CLI, REST, Feishu, Telegram, and Discord task-entry paths, enabling standby phones to act as remotely dispatched workers.
- Planning and VLM execution can be routed separately across documented provider paths including Claude, GPT, Gemini, Kimi, MiniMax, and compatible models.
- It requires an Android phone or emulator, USB debugging, and AccessibilityService permissions.
- Execution quality depends on the selected model, app UI, network state, and task duration; long-running reliability still needs more real-world testing.
- It is not yet an always-on operating-system assistant; tasks are manually triggered or launched through configured channels.
- The repository uses BUSL-1.1, so production or commercial use, hosted services, and commercial-product integration require a separate Core-Mate license.
How does this agent compare with similar options?
Against the repository's described typical phone-agent demo, OpenGUI uses a main graph plus executor subgraph, manages task state in the backend, and includes Android standby connections with Feishu, Telegram, Discord, REST, and local dispatch paths.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| OpenGUI Mobile Operator This agent | 45 · Major gaps | CLIFree + model costs | ★ 1.8k | 19d ago | HTML | Codex · Claude Code · OpenAI API · Claude API |
| Mobile-Agent: A Family of Powerful GUI Agents | 29 · Major gaps | Hosted serviceFreemium | ★ 9.4k | 3d ago | Python | — |
| UFO³ Galaxy | 51 · Major gaps | CLIFree + model costs | ★ 10k | 23d ago | Python | OpenAI API |
| Clawd Cursor | 74 · Some gaps | MCP serverFree + model costs | ★ 406 | 1d ago | TypeScript | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: README mentions user authorization for USB debugging, AccessibilityService, etc., but does not explicitly state least privilege; user confirmation steps exist (e.g., authorization actions) but not systematic; data flow description (e.g., screenshots, task dispatch) exists but not transparent enough; sensitive data handling (e.g., API keys) mentioned but not detailed; dependency security not mentioned; external effects (e.g., remote control) described but risks not assessed; rollback mechanism not mentioned; source attribution (e.g., Core-Mate) mentioned but not verified. Deductions: lack of implementation details and evidence.
Evidence shows: README and code structure are consistent, but there are inconsistencies (e.g., README mentions 'Qwen 3.6 Plus' which may not exist); dependency availability not clear; failure messages mentioned (e.g., error reporting) but not detailed. Deductions: insufficient evidence for dependencies and failure handling.
Evidence shows: README clearly identifies target users (developers, AI agents) and use cases (Android automation); capability boundaries described (e.g., requires Android device, permissions); trigger methods (CLI, IM) explained but not precise; environment fit (e.g., Docker, local) mentioned. Deductions: insufficient detail on trigger precision and boundaries.
Evidence shows: Information architecture clear (README, docs, skills); installation notes exist (start.sh, get-started.md); naming stability not clear; examples and FAQ limited; known limitations listed; license clear (BUSL-1.1); versioning/changelog not provided; maintenance responsibility not clear. Deductions: insufficient evidence for naming stability, examples, changelog, maintenance.
Evidence shows: Output usability (structured results) described; marginal value (e.g., cost savings) mentioned; cost-benefit (e.g., model cost comparison) explained. Deductions: lack of actual runtime data.
Evidence shows: Claims in README (e.g., features, performance) have some code references but not fully mapped; cross-source verification insufficient; facts and inferences not clearly separated. Deductions: insufficient claim-code correspondence and independent verification.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- This repository uses BUSL-1.1 license, not OSI-approved open source, production use requires commercial license.
- Models mentioned in README (e.g., Qwen 3.6 Plus) may not exist, verify.
- Remote control features (e.g., Discord, REST API) may introduce security risks, configure carefully.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
OpenGUI for WorkBuddy 0.4.1 — MCP upgrade fix - Upgrading an earlier OpenGUI HTTP installation now succeeds when its server matches the installation receipt. The public runtime uses stdio and removes the old HTTP URL and authorization headers. - Unknown or edited opengui servers remain unchanged. Conflict errors provide backup and editing commands, with instructions to preserve the existing server under another name before retrying. …View on GitHub ↗ -
dsh-coremate-mobile-v0.1.13 dsh-coremate-mobile v0.1.13
-
dsh-coremate-mobile-v0.1.12 dsh-coremate-mobile v0.1.12