Agent Device Verification CLI
A CLI that lets coding agents inspect, control, and verify apps running on devices and simulators.
- Source repo
- callstack/agent-device
- Stars
- ★ 4.7k
- Last updated
- today
- License
- MIT
- Primary language
- TypeScript
- FA score
- 50/100 · Major gaps
At a glance
- Works with
- Universal · cross-platformCodex · Claude Code
- You'll need
- Typical use
- A React Native or Expo developer asks a coding agent to verify a changed form flow on an iOS Simulator or Android Emulator using snapshots, press, and fill.
- Main limitation
- Requires Node.js 22.12 or newer, and web automation raises that requirement to Node.js 24 or newer.
- Source review
- 50/100 · Major gaps
What does this agent do, and when should you use it?
agent-device is a command-line automation tool for coding agents and QA tools to inspect, control, and verify running applications. It supports iOS, Android, tvOS, Android TV, Amazon Vega OS TV through Vega Virtual Device, web, macOS, and Linux, with command and evidence availability varying by target. The CLI keeps device state in sessions and uses XCTest for iOS and tvOS, ADB plus a snapshot helper for Android, Vega CLI/VDA for VVD, a local macOS helper, and AT-SPI on Linux. It can produce accessibility snapshots, screenshots, video, logs, traces, network data, performance samples, crash details, and React profiles; successful workflows can be saved as .ad scripts or exported as strict Maestro YAML. Teams can use it locally, replay runs in CI/CD, or configure remote and cloud execution paths.
A session can begin with a command such as agent-device open Contacts --platform ios, followed by agent-device snapshot -i to read the accessibility tree and obtain current @eN refs. An agent can act on the latest refs or selectors with press, fill, scrolling, gestures, waits, assertions, and alert handling; --settle returns UI changes after an action. It can save a screenshot with screenshot and collect video, logs, traces, network data, performance samples, crash details, and React profiles. Workflows can be retained as .ad scripts for local or CI replay, or exported as strict Maestro YAML; Node.js applications can also use the typed client or public subpaths, including agent-device/android-adb for Android ADB, logcat, clipboard, keyboard, app, and port-reverse helpers.
- A React Native or Expo developer asks a coding agent to verify a changed form flow on an iOS Simulator or Android Emulator using snapshots, press, and fill.
- A mobile QA engineer saves a successful login or onboarding flow as an .ad script for CI replay with screenshots and logs retained as artifacts.
- A native iOS, Android, or Flutter team captures screenshots, video, and diagnostic data when accessibility information is insufficient for a reviewable bug report.
- A TV application team uses a Vega Virtual Device to discover an app, manage its lifecycle, and perform complete TV-remote interactions.
- A React Native performance team reproduces a UI issue while collecting performance samples, traces, network data, or a React profile.
How do you install or deploy this agent?
Install Node.js 22.12 or newer; web automation requires Node.js 24 or newer. Run: npm install -g agent-device@latest. Then verify the environment with: agent-device doctor. The supplied material does not state that CLI installation requires an account or credential; device and target-specific setup requirements are covered by the installation documentation.
How do you use this agent?
Run agent-device doctor first, then use agent-device help workflow for workflow help. A working iOS sequence shown in the supplied material is: agent-device open Contacts --platform ios; agent-device snapshot -i; agent-device press @e2 --settle; agent-device fill @e7 "Ada" --settle; agent-device screenshot ./contact-form.png; agent-device close. Use @eN refs only from the latest output; after --settle, use the refs returned in its diff.
What are this agent's strengths and limitations?
- Uses an inspect-act-verify workflow, so an agent can read live app state and choose each next command at runtime.
- Combines mobile, TV, desktop, and basic web automation in one CLI while using target-appropriate underlying automation interfaces.
- Provides token-efficient accessibility snapshots alongside refs, selectors, screenshots, and video for both interaction and reviewable evidence.
- Can preserve validated flows as .ad scripts and export strict Maestro YAML for repeatability and CI use.
- Includes React Native-oriented diagnostic surfaces such as component trees, React profiles, and performance-related evidence.
- Requires Node.js 22.12 or newer, and web automation raises that requirement to Node.js 24 or newer.
- Commands and evidence vary by target, so feature parity across platforms is not documented.
- Initial Vega OS support is limited to Vega Virtual Device discovery, app lifecycle, and complete TV-remote control; physical Fire TV, capture, and selector backends are unsupported.
- Element refs can become stale after UI changes, requiring refs from the newest snapshot or --settle diff.
- Poor accessibility labels, roles, or test IDs reduce snapshot reliability and may require screenshots or video as fallback evidence.
How does this agent compare with similar options?
Appium, Detox, and Maestro are described as tools teams use to write and maintain test suites. agent-device instead lets an agent read application state and select commands at runtime, while complementing those tools by saving .ad scripts or exporting strict Maestro YAML.
Key facts side by side with the most closely related agents.
| Agent | Source review | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|
| Agent Device Verification CLI This agent | 50 · Major gaps | ★ 4.7k | today | TypeScript | Codex · Claude Code |
| iOS Simulator Skill Pack | 62 · Some gaps | ★ 1.3k | 10d ago | Python | Claude Code |
| Expo Skills – Official AI Agent Skills for Expo | 49 · Major gaps | ★ 2.6k | today | Shell | Codex · Claude Code |
| Docker-Android | 41 · Major gaps | ★ 16k | 3d ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: README suggests running doctor first, but no explicit user confirmation for each action; CLI can perform device actions (press, fill) but no mention of least privilege or user confirmation. Data flow: README mentions saving evidence (screenshots, logs) but not where data goes. Sensitive data handling not mentioned. Only two runtime dependencies (@limrun/api, yaml) but no dependency security audit evidence. External effects: CLI controls devices, may have side effects, but no rollback mechanism mentioned. Source attribution: publisher identity not verified, but README and package.json clearly attribute to Callstack. Deductions: lack of explicit least privilege, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects control, rollback, and source verification.
Evidence shows: README and package.json descriptions are consistent, CLI command examples align with docs, good self-consistency. Dependency availability: few dependencies but no guarantee. Failure messages: README does not detail error handling or failure messages. Deductions: dependency availability and failure messages lack evidence.
Evidence shows: README clearly targets AI coding agents, scenarios include local, CI/CD, cloud. Capability boundaries: README lists supported platforms and commands, notes Vega OS initial limitations. Trigger precision: command examples clear but no detailed trigger conditions. Environment fit: requires Node.js 22.12+, web needs 24+, and target requirements. Deductions: trigger precision and environment fit details insufficient.
Evidence shows: README well-structured with quick start, commands, FAQ, docs links. Install notes clear (npm install -g). Naming stability: version exists but no guarantee. Examples and FAQ rich. Known limitations: README mentions Vega OS physical devices unsupported. License: MIT license file present. Versioning changelog: no CHANGELOG provided. Maintenance responsibility: README and SECURITY.md indicate Callstack maintains, but not verified. Deductions: missing version changelog and naming stability evidence.
Evidence shows: Output usability: CLI provides snapshots, screenshots, etc., useful for agents. Marginal value: compared to Appium/Detox/Maestro, offers agent-native interaction, unique value. Cost benefit: open source MIT, free, but no performance data. Deductions: cost benefit lacks quantitative data.
Evidence shows: README claims (e.g., supported platforms) align with package.json and code structure, but no independent verification. Cross-source: README references blogs and videos but no independent verification. Fact-inference separation: README distinguishes facts (supported platforms) and inferences (how it works) but not explicitly labeled. Deductions: lack of independent verification and explicit fact-inference separation.
- Publisher identity is unverified; confirm trustworthiness before use.
- CLI can perform device actions with potential side effects; use in isolated environments.
- No data flow transparency; evidence files may contain sensitive information.