macOS Harness
A minimal harness that hands an LLM the whole Mac through six primitives — no app-specific tools, no framework, no rails.
- Source repo
- browser-use/macos-harness
- Stars
- ★ 1k
- Last updated
- 22d ago
- License
- MIT
- Primary language
- Python
- FA score
- 55/100 · Major gaps
At a glance
- How it runs
- Works with
- Platform-specificCodex · Claude Code
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A developer using Codex or Claude Code who wants an agent that installs itself, checks permissions, verifies the connection, and then controls the local Mac directly
- Not a fit if
- Users unwilling to grant macOS Accessibility and Screen Recording permissions
- Anyone needing stable, production-grade tooling rather than an experimental project
- Source review
- 55/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
macOS Harness (browser-use/macos-harness) is an MIT-licensed, experimental Python tool that gives an LLM native control over a Mac. It runs a single persistent Python process exposing six primitives: see (CGWindow screenshots, including background windows), key and type (CGEvent input sent directly to an app's PID without moving the real cursor), click (coordinate clicks), ax (native Accessibility), and script (Apple Events/AppleScript). The same process also provides browser.* (a Browser Harness connected via CDP to real Chrome), plus Path and subprocess for files and shell. It ships no app-specific tools; the model writes any missing logic in ordinary Python mid-task. It is macOS-only and requires granting system permissions verified via macos-harness doctor.
Runs one persistent Python process connected directly to macOS, your real browser, and your files. Concretely it: captures app windows via CGWindow with mac.see("Spotify") without bringing them to the foreground; sends keyboard and coordinate input to a specific PID via CGEvent using mac.key("cmd+k", app="Spotify") and mac.type(...); clicks with mac.click(640, 420, app="Spotify"); falls back to native Accessibility with mac.ax.at(...) and Apple Events with mac.script('tell application "Spotify" to play') when vision is not enough; drives the real logged-in Chrome through browser.page_info() and the CDP-based Browser Harness; and uses ordinary Path and subprocess for filesystem and shell work in the same process. After install, macos-harness skill prints a skill to register, macos-harness doctor reports required macOS permissions, and the connection can be verified by capturing an already-running background app. It also draws an animated, click-through pointer without moving your physical cursor.
- A developer using Codex or Claude Code who wants an agent that installs itself, checks permissions, verifies the connection, and then controls the local Mac directly
- Users automating specific apps (Spotify, Slack, Final Cut) without building or waiting for app-specific integration tools
- Users who need the agent to run while they work — it captures background windows, never steals focus, and never moves the real pointer
- Automation scenarios requiring the real, logged-in browser (Chrome via CDP) for web tasks
- Technical users who want a fallback to Accessibility APIs and AppleScript when screenshots are insufficient
- Automation engineers combining filesystem and shell operations with GUI control in one Python process
How do you install or deploy this agent?
Recommended (agent-driven): paste this into Codex or Claude Code: "Install or upgrade macOS Harness from https://github.com/browser-use/macos-harness with uv using Python 3.12. Register the skill printed by macos-harness skill, then run macos-harness doctor. Explain any missing macOS permissions and ask before requesting them. Finally, verify the harness by capturing one already-running app without bringing it to the foreground." Manual setup is documented in install.md. Requirements: macOS, Python 3.12, and uv; real Chrome is needed for browser control.
How do you use this agent?
Start a Python session via the CLI: macos-harness <<'PY' ... PY, then use the six primitives inside the process, e.g.: frame = mac.see("Spotify"); mac.key("cmd+k", app="Spotify"); mac.type("Alessia Cara", app="Spotify"); mac.click(640, 420, app="Spotify"); item = mac.ax.at(640, 420, app="Spotify"); mac.script('tell application "Spotify" to play'). browser.page_info(), Path, and subprocess are ready in the same process. First run macos-harness skill to register the workflow, then macos-harness doctor to check and grant macOS permissions. Disable telemetry with macos-harness telemetry disable if desired.
What are this agent's strengths and limitations?
- Minimal, framework-free design: six primitives (see/key/type/click/ax/script) cover any app, so no Spotify tools, Slack tools, or Final Cut tools need to exist
- Captures background app windows without raising them, sends input directly to an app PID, and never moves the physical pointer — suitable for machines in active use
- Layered fallback: raw Apple Accessibility and Apple Events when vision is insufficient, plus a real logged-in browser via CDP
- macos-harness doctor documents exactly which permissions are needed, and anonymous telemetry can be disabled with one command; privacy boundaries are explicitly stated
- macOS-only and explicitly labeled Experimental — no stability or long-term maintenance guarantees
- Requires granting multiple macOS system permissions (detected by doctor), a necessary setup cost before adoption
- Browser control depends on real Chrome and CDP; other browsers or browserless environments cannot use that capability
- Anonymous telemetry is on by default; privacy-conscious users must actively run macos-harness telemetry disable
- No prebuilt app tools: complex tasks rely on the model writing Python mid-task, raising the capability bar; the README offers no benchmarks or failure-rate evidence
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| macOS Harness This agent | 55 · Major gaps | CLIFree + model costs | ★ 1k | 22d ago | Python | Codex · Claude Code |
| SwiftAutoGUI Desktop Automation | 60 · Some gaps | CLIFree + model costs | ★ 94 | 19d ago | Swift | Claude Code · OpenAI API |
| Agent Desktop | 85 · Good | CLIFree | ★ 17k | 1d ago | Rust | — |
| Clawd Cursor | 74 · Some gaps | MCP serverFree + model costs | ★ 406 | 1d ago | TypeScript | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Positives: input events are posted to a target PID, untargeted input is refused (tested), doctor reports required permissions, telemetry is disclosed with a disable command. Deductions: the product is explicitly 'complete freedom' over a Mac with no per-action confirmation; telemetry is on by default; no rollback/undo mechanism exists; telemetry minimization is a README claim with no src code in evidence; attribution says 'Browser Use 2026' but publisher identity is unverified.
Tests cover key sequences, modifier transitions, focus-change interruption, and coordinate-space conversion; errors (MacOSError, FocusChangedError) carry clear messages; CI uses uv sync --locked. Deductions: the core runtime dependency browser-harness>=0.1.9 is an Alpha external package, a partial availability risk; only two exception types are visible.
Audience (developers) and scenario (macOS automation) are clear; the six-primitive surface is test-anchored (test_agent_surface_is_flat_and_explicit); environment constraints are precise (macOS, Python 3.11/3.12, conditional deps), earning 3. Deductions: 'complete freedom' means essentially no capability-boundary guardrails; only 'Experimental' flags this.
LICENSE is a complete MIT text (3); README/pyproject/CI/SECURITY.md are well organized; pyproject metadata is solid with a stable entry point. Deductions: no CHANGELOG, only version 0.1.2 Alpha; maintenance path is a single GitHub Security Advisories route with an unverified publisher; install.md is referenced but its contents are absent, so install notes are only partially visible.
Outputs are structured dicts (doctor results, see results with virtual_pointer/focus) directly consumable by an LLM; the deliberately thin 'no app-specific tools' design has clear marginal value over heavier frameworks; single-process, low overhead. Deductions: output usability depends on src code not shown; the permission cost (screen recording, accessibility, event posting) is nontrivial for users.
README claims (no cursor warping, no app activation, refusal of untargeted input, raw clicks bypassing AX) map to specific test assertions; facts and inference are well separated. Deductions: the core src/macos_harness implementation is not in the evidence set, so cross-source corroboration is limited to README↔tests↔pyproject and the telemetry-field claims cannot be independently verified.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- By design the tool grants full Mac control (keyboard, clicks, AppleScript, filesystem, logged-in browser) with no per-action confirmation or rollback; use only on isolated or acceptable machines.
- Anonymous telemetry is on by default; although the README claims minimal fields, the implementing code is not in the evidence set — consider running 'macos-harness telemetry disable' immediately after install.
- The core runtime dependency browser-harness is an Alpha-stage external package; supply-chain and availability risks are not independently verified.
- install.md and SKILL.md referenced by the README are not present in this evidence set, so the install and skill-registration flow could not be statically verified.
- Treat agent instructions and third-party UI content as untrusted input (as SECURITY.md itself advises) and grant the minimum macOS permissions needed.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
Your agent's own interrupt is now the stop button. Upgrade: after upgrading, run macos-harness install --agent <yours> again. Old Codex hooks do not stop typing on Esc. Changed - Ctrl+C, or Esc in Claude Code, Codex or Pi, stops the running action (a long type_text, a drag, a hold) before its next key or click. - Removed: Escape twice, the menu bar item, stop and resume. An old stop file is no longer read, so a stop left over from earlier can no longer block every call. - Every refusal the agent sees now says what happened and what to do next. …View on GitHub ↗ -
v0.2.0 macOS Harness v0.2.0
Release notes excerpt
A rebuild of the harness around one helper process. Agents drive Mac apps in the background: the real pointer does not move and no app comes to the front, so the person keeps working beside the agent. - Read: an app's window as a numbered text tree from the Accessibility API, then only what changed, with a screenshot. - Act: click, type_text, press_key, set_value, scroll, drag, select_text and menu actions, sent to the target app's process. …View on GitHub ↗ -
v0.1.2 macOS Harness v0.1.2
Release notes excerpt
Fix the animated project banner on PyPI by using the same absolute static-asset pattern as Browser Harness.View on GitHub ↗