macOS Harness
A minimal harness that hands an LLM the whole Mac through six primitives — no app-specific tools, no framework, no rails.
Per-dimension scores and reasoning
Positives: input events are posted to a target PID, untargeted input is refused (tested), doctor reports required permissions, telemetry is disclosed with a disable command. Deductions: the product is explicitly 'complete freedom' over a Mac with no per-action confirmation; telemetry is on by default; no rollback/undo mechanism exists; telemetry minimization is a README claim with no src code in evidence; attribution says 'Browser Use 2026' but publisher identity is unverified.
Tests cover key sequences, modifier transitions, focus-change interruption, and coordinate-space conversion; errors (MacOSError, FocusChangedError) carry clear messages; CI uses uv sync --locked. Deductions: the core runtime dependency browser-harness>=0.1.9 is an Alpha external package, a partial availability risk; only two exception types are visible.
Audience (developers) and scenario (macOS automation) are clear; the six-primitive surface is test-anchored (test_agent_surface_is_flat_and_explicit); environment constraints are precise (macOS, Python 3.11/3.12, conditional deps), earning 3. Deductions: 'complete freedom' means essentially no capability-boundary guardrails; only 'Experimental' flags this.
LICENSE is a complete MIT text (3); README/pyproject/CI/SECURITY.md are well organized; pyproject metadata is solid with a stable entry point. Deductions: no CHANGELOG, only version 0.1.2 Alpha; maintenance path is a single GitHub Security Advisories route with an unverified publisher; install.md is referenced but its contents are absent, so install notes are only partially visible.
Outputs are structured dicts (doctor results, see results with virtual_pointer/focus) directly consumable by an LLM; the deliberately thin 'no app-specific tools' design has clear marginal value over heavier frameworks; single-process, low overhead. Deductions: output usability depends on src code not shown; the permission cost (screen recording, accessibility, event posting) is nontrivial for users.
README claims (no cursor warping, no app activation, refusal of untargeted input, raw clicks bypassing AX) map to specific test assertions; facts and inference are well separated. Deductions: the core src/macos_harness implementation is not in the evidence set, so cross-source corroboration is limited to README↔tests↔pyproject and the telemetry-field claims cannot be independently verified.
- By design the tool grants full Mac control (keyboard, clicks, AppleScript, filesystem, logged-in browser) with no per-action confirmation or rollback; use only on isolated or acceptable machines.
- Anonymous telemetry is on by default; although the README claims minimal fields, the implementing code is not in the evidence set — consider running 'macos-harness telemetry disable' immediately after install.
- The core runtime dependency browser-harness is an Alpha-stage external package; supply-chain and availability risks are not independently verified.
- install.md and SKILL.md referenced by the README are not present in this evidence set, so the install and skill-registration flow could not be statically verified.
- Treat agent instructions and third-party UI content as untrusted input (as SECURITY.md itself advises) and grant the minimum macOS permissions needed.
What does this agent do, and when should you use it?
macOS Harness (browser-use/macos-harness) is an MIT-licensed, experimental Python tool that gives an LLM native control over a Mac. It runs a single persistent Python process exposing six primitives: see (CGWindow screenshots, including background windows), key and type (CGEvent input sent directly to an app's PID without moving the real cursor), click (coordinate clicks), ax (native Accessibility), and script (Apple Events/AppleScript). The same process also provides browser.* (a Browser Harness connected via CDP to real Chrome), plus Path and subprocess for files and shell. It ships no app-specific tools; the model writes any missing logic in ordinary Python mid-task. It is macOS-only and requires granting system permissions verified via macos-harness doctor.
Runs one persistent Python process connected directly to macOS, your real browser, and your files. Concretely it: captures app windows via CGWindow with mac.see("Spotify") without bringing them to the foreground; sends keyboard and coordinate input to a specific PID via CGEvent using mac.key("cmd+k", app="Spotify") and mac.type(...); clicks with mac.click(640, 420, app="Spotify"); falls back to native Accessibility with mac.ax.at(...) and Apple Events with mac.script('tell application "Spotify" to play') when vision is not enough; drives the real logged-in Chrome through browser.page_info() and the CDP-based Browser Harness; and uses ordinary Path and subprocess for filesystem and shell work in the same process. After install, macos-harness skill prints a skill to register, macos-harness doctor reports required macOS permissions, and the connection can be verified by capturing an already-running background app. It also draws an animated, click-through pointer without moving your physical cursor.
- A developer using Codex or Claude Code who wants an agent that installs itself, checks permissions, verifies the connection, and then controls the local Mac directly
- Users automating specific apps (Spotify, Slack, Final Cut) without building or waiting for app-specific integration tools
- Users who need the agent to run while they work — it captures background windows, never steals focus, and never moves the real pointer
- Automation scenarios requiring the real, logged-in browser (Chrome via CDP) for web tasks
- Technical users who want a fallback to Accessibility APIs and AppleScript when screenshots are insufficient
- Automation engineers combining filesystem and shell operations with GUI control in one Python process
What are this agent's strengths and limitations?
- Minimal, framework-free design: six primitives (see/key/type/click/ax/script) cover any app, so no Spotify tools, Slack tools, or Final Cut tools need to exist
- Captures background app windows without raising them, sends input directly to an app PID, and never moves the physical pointer — suitable for machines in active use
- Layered fallback: raw Apple Accessibility and Apple Events when vision is insufficient, plus a real logged-in browser via CDP
- macos-harness doctor documents exactly which permissions are needed, and anonymous telemetry can be disabled with one command; privacy boundaries are explicitly stated
- macOS-only and explicitly labeled Experimental — no stability or long-term maintenance guarantees
- Requires granting multiple macOS system permissions (detected by doctor), a necessary setup cost before adoption
- Browser control depends on real Chrome and CDP; other browsers or browserless environments cannot use that capability
- Anonymous telemetry is on by default; privacy-conscious users must actively run macos-harness telemetry disable
- No prebuilt app tools: complex tasks rely on the model writing Python mid-task, raising the capability bar; the README offers no benchmarks or failure-rate evidence
How do you install or deploy this agent?
Recommended (agent-driven): paste this into Codex or Claude Code: "Install or upgrade macOS Harness from https://github.com/browser-use/macos-harness with uv using Python 3.12. Register the skill printed by macos-harness skill, then run macos-harness doctor. Explain any missing macOS permissions and ask before requesting them. Finally, verify the harness by capturing one already-running app without bringing it to the foreground." Manual setup is documented in install.md. Requirements: macOS, Python 3.12, and uv; real Chrome is needed for browser control.
How do you use this agent?
Start a Python session via the CLI: macos-harness <<'PY' ... PY, then use the six primitives inside the process, e.g.: frame = mac.see("Spotify"); mac.key("cmd+k", app="Spotify"); mac.type("Alessia Cara", app="Spotify"); mac.click(640, 420, app="Spotify"); item = mac.ax.at(640, 420, app="Spotify"); mac.script('tell application "Spotify" to play'). browser.page_info(), Path, and subprocess are ready in the same process. First run macos-harness skill to register the workflow, then macos-harness doctor to check and grant macOS permissions. Disable telemetry with macos-harness telemetry disable if desired.