Clawd Cursor
Compiles the screen into one addressable UI map so any AI agent can safely drive the real desktop — local, cross-platform, model-agnostic, with every action verified.
- Source repo
- AmrDab/clawdcursor
- Stars
- ★ 408
- Last updated
- today
- License
- MIT
- Primary language
- TypeScript
- FA score
- 72/100 · Some gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformCodex · Claude Code
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A developer using Claude Code or Cursor who wants the agent to operate native apps like Outlook or Notepad that expose no API or CLI
- Not a fit if
- macOS users unwilling to grant Accessibility and Screen Recording permissions
- Linux hosts where system packages like tesseract-ocr and python3-gi can't be installed
- Website
- clawdcursor.com ↗ set on GitHub
- Maintenance
- Commits in the last 30 dayslast commit 2026-11-10latest release v1.5.27 (2026-11-10)
- Source review
- 72/100 · Some gaps
What does this agent do, and when should you use it?
Clawd Cursor is a local MCP server that gives any tool-calling agent — Claude Code, Cursor, Windsurf, Zed, OpenAI Codex — safe control of the real desktop: clicking, typing, reading the screen, and opening apps. Its core innovation is the UI State Compiler, which fuses the accessibility tree with OCR into a confidence-scored map where every element carries a stable el_NN id; the agent acts by element id instead of pixel coordinates, dropping to the expensive screenshot/vision tier only for canvas-only apps. Every call routes through a single safety.evaluate() chokepoint (allow / confirm / block), and consequential actions accept an expect parameter that re-checks the live screen, returning a DEVIATION when reality didn't match. It runs on Windows, macOS, and Linux, exposes stdio and HTTP MCP transports, and ships 7 compound tools alongside 98 granular primitives. While an agent drives, an on-screen banner with a blinking red dot appears and can be double-clicked as a kill switch.
At runtime, compile_ui fuses the accessibility tree (free) with OS-level OCR (cheap) into a confidence-scored element map with stable el_NN ids; find_button / find_field locate targets by meaning. Agents call seven compound tools — computer, accessibility, window, system, browser, task, batch — each with an action enum of roughly 10–20 verbs (click, type, key, invoke, open_app, etc.), or the 98 granular primitives directly. Every call first passes the safety.evaluate() gate with three tiers: Allow (executes immediately), Confirm (sends, deletes, purchases, sensitive apps pause for human approval), and Block (Ctrl+Alt+Del-class sequences refused outright). Actions carrying expect re-read the live screen after a short settle window and return DEVIATION if state didn't obey. A sparse a11y tree triggers system.detect_webview to switch Electron/WebView2 apps to CDP-based browser.* tools; canvas-only apps fall back to screenshot plus coordinate click as a last resort. The deployment boundary is fully local: stdio MCP spawned by the editor host, or clawdcursor agent serving HTTP MCP on 127.0.0.1:3847 with Bearer-token auth — no telemetry by default.
- A developer using Claude Code or Cursor who wants the agent to operate native apps like Outlook or Notepad that expose no API or CLI
- Business-process automation over legacy desktop software, e.g. finding a form field by name and typing into it instead of brittle pixel-coordinate scripts
- Privacy-sensitive setups leveraging per-OS OCR (Apple Vision on macOS, Windows.Media.Ocr on Windows, Tesseract on Linux) so screen content never leaves the machine
- Hybrid orchestration teams that plan with an expensive model (Claude) and delegate grunt work via task() to the built-in loop on a cheaper LLM
- Auditable ops automation: every step passes one safety gate, with email, banking, and password-manager apps auto-elevated to human confirmation
- Integrators running headless agents (Agent SDK, custom loops) against the same tool catalog over HTTP MCP at 127.0.0.1:3847
Key setup facts
Pulled from the install and usage notes below. The project README has the current values.
- Install / run
npm i -g clawdcursor
- Config files
~/.codex/config.toml~/.clawdcursor/token- Runtime
- Node.js 20+
- Install notes cover
- Windows · macOS · Linux
How do you install or deploy this agent?
Requires Node.js 20+. Global install (recommended — pinnable and inspectable on disk):
npm i -g clawdcursor
clawdcursor consent --accept # one-time desktop-control consent (required)
clawdcursor grant # macOS only: approve Accessibility + Screen RecordingPer-OS prerequisites: Windows installs clean (sharp and @nut-tree-fork/nut-js ship prebuilt binaries); macOS needs Xcode Command Line Tools (xcode-select --install) for screenshots/vision; Linux needs tesseract-ocr, python3-gi, gir1.2-atspi-2.0, plus ydotool on Wayland.
Add to Claude Code:
claude mcp add clawdcursor -s user -- clawdcursor mcp --compactAdd to OpenAI Codex (~/.codex/config.toml):
[mcp_servers.clawdcursor]
command = "clawdcursor"
args = ["mcp", "--compact"]Cursor / Windsurf / Claude Desktop: add {"mcpServers":{"clawdcursor":{"command":"clawdcursor","args":["mcp","--compact"]}}} to the host's MCP config; Zed uses the context_servers key. Or install the bundled plugin in one step: claude plugin marketplace add AmrDab/clawdcursor && claude plugin install clawdcursor@clawdcursor.
How do you use this agent?
Once installed, the editor host spawns clawdcursor mcp over stdio on demand. Issue natural-language tasks directly, e.g. "open Outlook and reply to the latest email from Sarah." Headless/daemon mode: run clawdcursor agent (HTTP MCP on 127.0.0.1:3847, Bearer token at ~/.clawdcursor/token, POST JSON-RPC to /mcp), or clawdcursor agent --no-llm for the tool surface only. Example calls:
computer({ action: "key", combo: "mod+s" }) // Cmd+S / Ctrl+S, resolved per-OS
accessibility({ action: "invoke", name: "Send" }) // click by name, not pixels
window({ action: "open_app", name: "Outlook" })
task({ instruction: "open Notepad and type hello" }) // delegate to the built-in loopCollapse known step sequences with batch({steps:[…]}) — each step is still perceived and safety-gated. Diagnose with clawdcursor doctor (configures the LLM for agent mode), clawdcursor status (readiness check), or clawdcursor stop (stop all modes).
What are this agent's strengths and limitations?
- Cheapest-tier-first perception: accessibility tree (free) → OCR (cheap) → screenshot (expensive, the only tier that puts pixels in model context), so token cost tracks task difficulty
- Unique reactive verification: pass expect on a consequential action and it re-checks the live screen, reporting DEVIATION instead of hollow success
- Single safety chokepoint: stdio, HTTP, and built-in loop calls all pass safety.evaluate() (allow/confirm/block); sensitive apps auto-elevate to human confirmation and the agent cannot bypass it
- Human always informed: a desktop-control banner with blinking red dot appears while an agent drives — double-click to stop
- Fully local, model-agnostic, cross-OS: with a local model nothing leaves the machine, and one MCP config works with any MCP host
- macOS requires Accessibility permission (mandatory) and Screen Recording (optional), plus Xcode Command Line Tools for the screenshot/vision tier — real first-run friction
- Linux depends on system packages npm cannot install (tesseract-ocr, python3-gi, gir1.2-atspi-2.0, plus ydotool on Wayland)
- Perception quality depends on each OS's native stacks; dark or non-standard UIs with sparse a11y trees force tier-climbing, and canvas-only apps are limited to coordinate clicks
- Verification and safety gating add round-trips; batch collapses calls but still re-perceives and gates each step
- Requires Node.js 20+ and a recommended global install (rather than auto-fetching via npx), an extra operational constraint in locked-down environments
How does this agent compare with similar options?
The README contrasts Clawd Cursor with several named alternatives: Windows-MCP and Terminator (peer desktop MCP servers); browser-use and Playwright MCP (browser-only); OmniParser and UI-TARS (vision-centric parsing approaches needing a screenshot in model context for every observation); and Anthropic computer-use (Claude-only, screens sent to a sandbox/cloud). Its stated differentiators: any desktop app (not just the web), cross-OS, vision-model-free perception (a11y + OCR fusion), self-verified actions, one safety gate, any model/vendor, MCP-native, and local-only.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Clawd Cursor This agent | 72 · Some gaps | MCP serverFree + model costs | ★ 408 | today | TypeScript | Codex · Claude Code |
| Open Computer Use (MCP desktop service) | 79 · Good | CLIFree + model costs | ★ 2.9k | 1d ago | Swift | Codex · Claude Code |
| macOS Harness | 54 · Major gaps | CLIFree + model costs | ★ 1k | 27d ago | Python | Codex · Claude Code |
| AIHawk – AI Browser Agent | 52 · Major gaps | MCP serverFree + model costs | ★ 32k | 1d ago | Python | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
The code and documentation show a central safety gate, window-focus checks before input, pause/stop controls, confirmation for sensitive actions, and loopback binding. These support meaningful least privilege and user control. Privacy statements say screenshots stay in memory and there is no telemetry, but the supplied material does not fully map data flows across MCP transports, so data-flow transparency and sensitive-data handling are not fully established. Dependency security is addressed through a lockfile audit, a high-severity threshold, and disclosure of accepted moderate issues, but the lockfile and current audit results are not supplied. External effects have confirmation controls; rollback is only mentioned as undo without a stated recovery scope. Author information is present, but publisher identity is unverified and maintenance responsibility and update commitments are limited.
The workflows, CI configuration, tests, and security policy provide reliability evidence, but README/plugin/skill material has version and feature-timing inconsistencies: package and plugin metadata say 1.5.26, while the README describes a 1.5.27 deprecation; the skill says compact mode has seven tools but lists six compound tools plus batch/task. Dependency requirements and platform prerequisites are specific. Failure examples cover permissions, OCR, and window focus, though the sample cannot establish the full failure behavior.
The materials identify target users, MCP hosts, desktop-control scenarios, and all three operating systems. The skill explicitly prioritizes APIs, CLIs, file edits, and browser automation, and states capability and action rules, giving it precise trigger boundaries. Platform prerequisites and permissions are documented, but the README says some macOS/Linux scrolling behavior has not yet run on a real desktop, so environment fit is not fully demonstrated.
The README organizes installation, operation, tools, and platform information into clear sections; it includes host-specific configuration, per-OS prerequisites, examples, and release history. Known limitations include Linux system packages, macOS permissions, vision dependencies, and deprecation plans. The MIT license is explicit. Naming is mostly stable, but version labels and tool counts conflict; examples are extensive while FAQ coverage is limited. The author and a security-reporting channel are named, but long-term maintenance responsibility is not fully specified.
UI structure, OCR, stable element identifiers, verification, and batching are designed to reduce desktop-task round trips. Local cross-application control provides a clear marginal benefit where APIs or CLIs do not exist. Structured text, errors, and deviation states should aid use, but the supplied sample lacks a complete output schema or representative end-to-end output. Installation, permissions, and Linux system packages add cost, and screenshot/vision paths can add model cost, so cost-benefit is not fully established.
README claims can be partially cross-checked against the skill, package metadata, source snippets, security policy, CI workflows, and test examples. Source excerpts show platform selection and capability matching, while CI describes intended checks. Many product-effectiveness claims remain documentation assertions; the packet contains no independently verifiable run results. Version inconsistencies weaken corroboration. Documentation sometimes distinguishes tested from not-yet-run platform behavior, but it does not systematically label fact versus inference, so none of these criteria merits full marks.
- This is a static review; code was not executed and product claims were not independently tested. The supplied source is a sample.
- Package and plugin metadata say 1.5.26, while the README refers to a 1.5.27 deprecation; check the intended release and matching skill instructions before installation.
- The product receives desktop input and screen-reading permissions; confirm host consent, OS permissions, and sensitive-action confirmation behavior meet your expectations.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
Fixed - Typing goes only where the agent was working. Windows set_value's keyboard fallback types only when the target's own window is in front, and an owned dialog in front (Save As) counts as that window (#314, #331). A typed value that cannot be read back is reported as not verified instead of set. - A stop always stops, and says so first. A stop also refuses hover, clipboard writes, page script and tab switches (#319). A stopped call now answers "stopped by the user" before any other refusal (#329). …View on GitHub ↗ -
Release notes excerpt
Security - Windows: text could run as PowerShell code (GHSA-9ppf-38v8-h97v, high). Clipboard writes (type_text, smart_type, copy_all_text and the restore of the user's clipboard), app launches, window-title lookups and relaunch_with_cdp quoted values into PowerShell source, and the quoting could be broken by other characters PowerShell treats as quotes, so text from a document, a page or the clipboard could break out and run commands with no confirmation. Values now reach PowerShell only as data: clipboard …View on GitHub ↗ -
Release notes excerpt
Changed - A step the agent may not take is handed over, not the whole task. Agents turned down whole tasks (an APNs key, an App Store review account, notarization) because one step was something their own rules forbid: creating an account, downloading a private key, entering a secret. The MCP instructions, the pause_for_user description and the skill now say that such a step is a roadblock like a sign-in: hand over that step, as wide as the agent's rules draw it, and do the rest they allow. When a secret must go into a field or a terminal …View on GitHub ↗
FAQ
Where does my screen data go?
Does it use screenshots or the accessibility tree, and what does it cost in tokens?
Can the agent delete files or send email by mistake?
How do I stop an agent mid-task?
Which hosts and run modes are supported?
Is Clawd Cursor open source?
Is Clawd Cursor safe to use?
Is Clawd Cursor still maintained?
Who makes Clawd Cursor?
What is Clawd Cursor's official website?
Thanks — we use this to improve the page.