Clawd Cursor
Compiles the screen into one addressable UI map so any AI agent can safely drive the real desktop — local, cross-platform, model-agnostic, with every action verified.
Evidence shows: single safety.evaluate() chokepoint (allow/confirm/block), HTTP bound to 127.0.0.1 with bearer-token auth, screen text wrapped in <untrusted-screen-content> tags, log redaction of password fields, explicit confirm/block tier table, revocable one-time consent (--revoke), uninstall cleanup path, MIT with named author. Deductions: the safety gate and platform adapter are described mainly in the README; gate implementation code is not in the sample; the granted desktop-wide permissions (Accessibility, screen recording, synthetic input) inherently exceed least privilege — scored 2.
CI matrix covers 3 OS × 2 Node versions with lint, typecheck, npm audit gate, coverage ratchet and perf smoke; the sampled tests (pidfile, orphan teardown, credentials) are high quality with honest skip rationales. Deductions: the '1,000+ tests' claim cannot be verified from the sample; a known native-module segfault on headless Linux teardown is admitted in test comments.
Clear target audience (MCP agent users), honest 'last-mile fallback' positioning, cross-platform matrix (Windows/macOS/Linux/Wayland) with per-OS prerequisites and known limits (ydotool on Wayland, headless Linux segfault). Deductions: trigger precision and capability boundaries rest on narrative rather than a specification file.
README is well structured (install/how-it-works/tools/platforms/safety/architecture/CLI); package. metadata is complete (repository, bugs, bin, files, engines, overrides); full MIT LICENSE, SECURITY.md with reporting channels, CHANGELOG referenced, version-sync script. Deductions: CHANGELOG and SKILL.md contents are not in the sample; maintenance responsibility is only indirectly evidenced by CI config; publisher is unverified.
A11y+OCR fusion, stable el_NN addressing, expect-based verification with DEVIATION reporting, and tiered perception to control token cost give clear marginal value over screenshot-first approaches, with an honest comparison table. Deductions: token/performance benefits are asserted, not demonstrated; cost-benefit depends on scenario (Node 20+, native deps per OS).
Claims mostly trace to concrete mechanisms (safety gate, consent file, pidfile tests, CI workflows consistent with README); test comments candidly record failed attempts and skip rationale; accepted audit advisories (issue #117) are disclosed. Deductions: performance/cost/reliability conclusions are inference; static review cannot execute them, hence low confidence.
- This tool requires desktop-wide OS permissions (Accessibility, Screen Recording, synthetic input); any vulnerability gives an attacker the same control as the logged-in user — only enable it for agents you trust on that machine.
- The allow/confirm/block behavior of the safety gate is described mainly in the README; the gate implementation was not in the reviewed sample. Inspect the SafetyLayer in src/core yourself and verify there is no bypass path before deployment.
- The HTTP-mode bearer token lives at ~/.clawdcursor/token; watch for leakage via logs, error responses, or local web pages (CORS), and periodically confirm the port binds only to 127.0.0.1.
- CI currently accepts two moderate advisories (unpatched file-type infinite loop, ip-address XSS); re-check the status of these transitive dependencies before production use.
- A known native-module segfault exists on headless-Linux teardown and Wayland support depends on external ydotool; run clawdcursor status to self-check before relying on it.
What does this agent do, and when should you use it?
Clawd Cursor is a local MCP server that gives any tool-calling agent — Claude Code, Cursor, Windsurf, Zed, OpenAI Codex — safe control of the real desktop: clicking, typing, reading the screen, and opening apps. Its core innovation is the UI State Compiler, which fuses the accessibility tree with OCR into a confidence-scored map where every element carries a stable el_NN id; the agent acts by element id instead of pixel coordinates, dropping to the expensive screenshot/vision tier only for canvas-only apps. Every call routes through a single safety.evaluate() chokepoint (allow / confirm / block), and consequential actions accept an expect parameter that re-checks the live screen, returning a DEVIATION when reality didn't match. It runs on Windows, macOS, and Linux, exposes stdio and HTTP MCP transports, and ships 7 compound tools alongside 98 granular primitives. While an agent drives, an on-screen banner with a blinking red dot appears and can be double-clicked as a kill switch.
At runtime, compile_ui fuses the accessibility tree (free) with OS-level OCR (cheap) into a confidence-scored element map with stable el_NN ids; find_button / find_field locate targets by meaning. Agents call seven compound tools — computer, accessibility, window, system, browser, task, batch — each with an action enum of roughly 10–20 verbs (click, type, key, invoke, open_app, etc.), or the 98 granular primitives directly. Every call first passes the safety.evaluate() gate with three tiers: Allow (executes immediately), Confirm (sends, deletes, purchases, sensitive apps pause for human approval), and Block (Ctrl+Alt+Del-class sequences refused outright). Actions carrying expect re-read the live screen after a short settle window and return DEVIATION if state didn't obey. A sparse a11y tree triggers system.detect_webview to switch Electron/WebView2 apps to CDP-based browser.* tools; canvas-only apps fall back to screenshot plus coordinate click as a last resort. The deployment boundary is fully local: stdio MCP spawned by the editor host, or clawdcursor agent serving HTTP MCP on 127.0.0.1:3847 with Bearer-token auth — no telemetry by default.
- A developer using Claude Code or Cursor who wants the agent to operate native apps like Outlook or Notepad that expose no API or CLI
- Business-process automation over legacy desktop software, e.g. finding a form field by name and typing into it instead of brittle pixel-coordinate scripts
- Privacy-sensitive setups leveraging per-OS OCR (Apple Vision on macOS, Windows.Media.Ocr on Windows, Tesseract on Linux) so screen content never leaves the machine
- Hybrid orchestration teams that plan with an expensive model (Claude) and delegate grunt work via task() to the built-in loop on a cheaper LLM
- Auditable ops automation: every step passes one safety gate, with email, banking, and password-manager apps auto-elevated to human confirmation
- Integrators running headless agents (Agent SDK, custom loops) against the same tool catalog over HTTP MCP at 127.0.0.1:3847
What are this agent's strengths and limitations?
- Cheapest-tier-first perception: accessibility tree (free) → OCR (cheap) → screenshot (expensive, the only tier that puts pixels in model context), so token cost tracks task difficulty
- Unique reactive verification: pass expect on a consequential action and it re-checks the live screen, reporting DEVIATION instead of hollow success
- Single safety chokepoint: stdio, HTTP, and built-in loop calls all pass safety.evaluate() (allow/confirm/block); sensitive apps auto-elevate to human confirmation and the agent cannot bypass it
- Human always informed: a desktop-control banner with blinking red dot appears while an agent drives — double-click to stop
- Fully local, model-agnostic, cross-OS: with a local model nothing leaves the machine, and one MCP config works with any MCP host
- macOS requires Accessibility permission (mandatory) and Screen Recording (optional), plus Xcode Command Line Tools for the screenshot/vision tier — real first-run friction
- Linux depends on system packages npm cannot install (tesseract-ocr, python3-gi, gir1.2-atspi-2.0, plus ydotool on Wayland)
- Perception quality depends on each OS's native stacks; dark or non-standard UIs with sparse a11y trees force tier-climbing, and canvas-only apps are limited to coordinate clicks
- Verification and safety gating add round-trips; batch collapses calls but still re-perceives and gates each step
- Requires Node.js 20+ and a recommended global install (rather than auto-fetching via npx), an extra operational constraint in locked-down environments
How do you install or deploy this agent?
Requires Node.js 20+. Global install (recommended — pinnable and inspectable on disk):
npm i -g clawdcursor
clawdcursor consent --accept # one-time desktop-control consent (required)
clawdcursor grant # macOS only: approve Accessibility + Screen RecordingPer-OS prerequisites: Windows installs clean (sharp and @nut-tree-fork/nut-js ship prebuilt binaries); macOS needs Xcode Command Line Tools (xcode-select --install) for screenshots/vision; Linux needs tesseract-ocr, python3-gi, gir1.2-atspi-2.0, plus ydotool on Wayland.
Add to Claude Code:
claude mcp add clawdcursor -s user -- clawdcursor mcp --compactAdd to OpenAI Codex (~/.codex/config.toml):
[mcp_servers.clawdcursor]
command = "clawdcursor"
args = ["mcp", "--compact"]Cursor / Windsurf / Claude Desktop: add {"mcpServers":{"clawdcursor":{"command":"clawdcursor","args":["mcp","--compact"]}}} to the host's MCP config; Zed uses the context_servers key. Or install the bundled plugin in one step: claude plugin marketplace add AmrDab/clawdcursor && claude plugin install clawdcursor@clawdcursor.
How do you use this agent?
Once installed, the editor host spawns clawdcursor mcp over stdio on demand. Issue natural-language tasks directly, e.g. "open Outlook and reply to the latest email from Sarah." Headless/daemon mode: run clawdcursor agent (HTTP MCP on 127.0.0.1:3847, Bearer token at ~/.clawdcursor/token, POST JSON-RPC to /mcp), or clawdcursor agent --no-llm for the tool surface only. Example calls:
computer({ action: "key", combo: "mod+s" }) // Cmd+S / Ctrl+S, resolved per-OS
accessibility({ action: "invoke", name: "Send" }) // click by name, not pixels
window({ action: "open_app", name: "Outlook" })
task({ instruction: "open Notepad and type hello" }) // delegate to the built-in loopCollapse known step sequences with batch({steps:[…]}) — each step is still perceived and safety-gated. Diagnose with clawdcursor doctor (configures the LLM for agent mode), clawdcursor status (readiness check), or clawdcursor stop (stop all modes).
How does this agent compare with similar options?
The README contrasts Clawd Cursor with several named alternatives: Windows-MCP and Terminator (peer desktop MCP servers); browser-use and Playwright MCP (browser-only); OmniParser and UI-TARS (vision-centric parsing approaches needing a screenshot in model context for every observation); and Anthropic computer-use (Claude-only, screens sent to a sandbox/cloud). Its stated differentiators: any desktop app (not just the web), cross-OS, vision-model-free perception (a11y + OCR fusion), self-verified actions, one safety gate, any model/vendor, MCP-native, and local-only.