Nuphus MCP Desktop Automation
Give any MCP client control of desktops, input devices, windows, and Chrome.
Per-dimension scores and reasoning
The project plainly states that it grants access comparable to a remote operator and recommends avoiding elevation and restricting stdin access. It also describes stdio transport, local OCR, optional external vision calls, model-download locations, and browser attachment. Path validation, destructive annotations, clipboard cleanup, and dependency auditing are substantive protections, supporting moderate scores for flow disclosure, sensitive-data handling, dependency security, and external effects. Deductions apply because the capability is inherently machine-wide, write operations run without confirmation by default, and strict confirmation is opt-in. Clicks, typing, navigation, and cookie changes generally lack a universal rollback path. Attribution reaches the repository, npm package, and “Nuphus Team,” but the publisher is unverified and individual stewardship is unclear.
The README, security policy, and workflows are broadly consistent about architecture, platforms, confirmation, and test scope. Missing browsers, unconfigured vision, failed model downloads, and broken external CDP connections are said to produce explicit errors. Prerequisites, browser discovery, prebuilt packages, and release-version consistency are adequately handled. Deductions reflect partial Linux desktop support, first-use network dependencies for OCR/YOLO, the non-blocking real-Chrome job in ordinary CI, and the absence of implementation code demonstrating the claimed error quality.
Documentation thoroughly covers generic MCP clients, Claude Desktop, DSH, managed and external Chrome, fingerprint browsers, persistent sessions, vision providers, and multiple operating systems. It clearly distinguishes desktop from DOM automation and documents Chrome 136+, platform, BYOK, and browser-ownership boundaries. Environment-specific guidance is unusually detailed. Trigger precision receives only a moderate score because the supplied evidence omits TOOLS.md schemas and per-tool invocation rules, preventing confirmation of every high-risk tool's constraints.
The README has strong organization, repository mapping, prerequisites, platform tables, installation paths, client configurations, demo material, and test commands. Product, command, environment-variable, and tool-prefix naming is stable. Limitations and the full MIT license are present, while release automation enforces version agreement and generates release notes. Deductions apply because no standalone FAQ or changelog is supplied. Maintenance is attributed mainly to “Nuphus Team”; the security channel, response commitments, and latest-only support policy establish an update path, but ownership remains nonspecific.
Combining desktop control, window management, local OCR, optional semantic vision, and Chrome CDP behind MCP offers substantial marginal value over a single-mode automation tool. PNG/base64 screenshots, accessibility references, OCR elements, JSON-RPC, and structured errors indicate usable outputs. The complete tool reference and representative responses are absent, however, so uniform downstream usability cannot be established. Core automation needs no API key, but vision usage, downloads, storage, initial setup, and machine-control risk impose real costs; consequently the “zero-cost” characterization is only narrowly supported.
Several README claims about safety, platforms, testing, and releases are corroborated by SECURITY, workspace metadata, and CI/release workflows; license metadata is also backed by the license text. The workflows provide static evidence of test, audit, and publication gates. Core implementation, Cargo.lock, the audit exception file, TOOLS.md, and test sources are absent, so claims such as 38 tools, 60 tests, validation details, and error behavior remain largely self-reported. The documentation usually distinguishes defaults, recommendations, limitations, and optional features, but phrases such as “battle-tested” and local 7/7 results lack independent support in the supplied files.
- Write-capable tools execute without confirmation by default. Enable --confirm-write or NUPHUS_MCP_CONFIRM_WRITE=1, and treat every process able to write to the server's stdin as having machine-control authority.
- desktop_vision sends screen content to the user-configured external model endpoint. Screenshots may expose credentials, private messages, or other sensitive data, so review the endpoint and its retention policy before use.
- Browser tools can read or change cookies and act through authenticated sessions; copying a real browser profile increases credential exposure. Prefer an isolated profile and a dedicated low-privilege account.
- OCR/YOLO models are downloaded on first use, while the supplied evidence omits the lockfile, audit exceptions, and download-integrity mechanism. Verify dependencies, model provenance, checksums, and current audit results before deployment.
- Linux desktop support is explicitly partial, macOS requires Accessibility permission, and the real-Chrome integration job is not yet a hard gate in ordinary CI.
What does this agent do, and when should you use it?
nuphus-mcp is a lightweight, cross-platform MCP server exposing 38 automation tools: 15 for the desktop and 23 for the browser. It runs as a single binary and exchanges newline-delimited JSON-RPC 2.0 over stdio, without a daemon or HTTP service. Its workspace combines the nuphus-mcp server, the chromiumoxide CDP-based nuphus-browser component, and desktop-api for native desktop control. It can capture screens, manipulate windows and input devices, manage the clipboard, and use Chrome DevTools Protocol for navigation, accessibility snapshots, interaction, scripts, tabs, cookies, uploads, and downloads. Core desktop and browser automation requires no API key; OCR runs locally with PaddleOCR, while semantic vision connects to a user-supplied OpenAI-compatible or Anthropic-native model. Desktop support is complete on Windows, requires Accessibility permission on macOS, and is limited for some window and input operations on Linux.
An MCP client launches nuphus-mcp over stdio and invokes initialize, tools/list, and tools/call. The desktop path uses desktop-api to read screen and window state, produce PNG/base64 screenshots, move and resize windows, operate the mouse and keyboard, and write or clear the clipboard. desktop_perceive runs local PaddleOCR to return recognized elements with pixel coordinates and can add optional YOLO icon detection; desktop_vision sends a screenshot to a configured OpenAI-compatible or Anthropic-native vision endpoint for semantic interpretation. The browser path uses nuphus-browser and CDP to launch or attach to Chrome/Edge and perform navigation, accessibility-tree snapshots, clicks, typing, extraction, screenshots, JavaScript evaluation, waits, history navigation, cookie operations, uploads, tab management, and downloads. The process reads one-line JSON requests from stdin, writes JSON-RPC responses to stdout, and sends logs to stderr; with --confirm-write enabled, mutating calls must include confirm: true.
- Teams using Claude Desktop, Cursor, VS Code, Copilot, or another MCP client can let an agent click, type, switch windows, and capture screens across local applications.
- Browser testers and operations staff can automate Chrome through CDP to navigate sites, inspect accessibility snapshots, complete forms, manage cookies, upload files, and handle downloads.
- Automation developers working with desktop software that has no DOM can combine desktop_perceive with mouse and keyboard tools to act on coordinates returned by local OCR.
- Users facing visually complex interfaces can connect desktop_vision to their own OpenAI-compatible or Anthropic vision model, then use desktop_perceive for precise coordinates.
- Teams operating anti-detect or fingerprint browsers can attach through NUPHUS_MCP_BROWSER_CDP_URL and configure browser identity data for live-port rediscovery.
- DeepSeek Harness users seeking native tool mounting should install the dedicated dsh-nuphus-mcp package rather than treating this repository as a DSH/Cordis plugin.
What are this agent's strengths and limitations?
- One stdio MCP server combines 15 native desktop tools with 23 CDP browser tools, covering both non-web interfaces and browser-aware automation.
- Core desktop and browser operations need no API key, and OCR runs locally; only semantic vision analysis requires user-provided model credentials.
- desktop_vision supports both OpenAI-compatible and Anthropic-native protocols instead of tying the feature to one model vendor.
- It can attach to externally managed Chrome or fingerprint browsers and rediscover changed ports using the executable path, profile directory, and DevToolsActivePort.
- It provides strict confirmation for mutating calls, MCP destructive-operation annotations, and path validation for screenshots, uploads, and file drags.
- Windows is the only documented platform with full desktop control; macOS requires an additional permission, and Linux window/input capabilities are partial.
- Browser tools depend on Chrome or Edge. Chrome 136+ cannot expose its default user-data directory through remote debugging, requiring a dedicated or copied profile.
- desktop_vision depends on an external model endpoint, credentials, and network access, and may incur provider charges.
- Local OCR and optional YOLO assets are downloaded on first use; failed downloads require manual remediation, and YOLO failure leaves only OCR results.
- Mutating tools run without confirmation by default, so adopters must explicitly enable --confirm-write or NUPHUS_MCP_CONFIRM_WRITE=1.
- External-browser attachment never falls back to managed Chrome; without identity-based recovery, a stale CDP URL must be updated manually.
How do you install or deploy this agent?
The recommended installation uses the prebuilt npm package:
npm install -g @nuphus/nuphus-mcp
nuphus-mcp --confirm-writePrebuilt binaries are provided for Windows x64/arm64, macOS arm64, and Linux x64/arm64, so this route does not require Rust. Chrome or Edge is required for browser tools. To build from source instead, install the stable Rust toolchain and run:
cargo build --release -p nuphus-mcpThe binary is written to target/release/nuphus-mcp, with .exe on Windows. On its first call, desktop_perceive downloads PaddleOCR models and icon_detect.onnx into %APPDATA%\Nuphus\models or NUPHUS_MODELS_DIR. Only desktop_vision requires NUPHUS_MCP_VISION_API_KEY and NUPHUS_MCP_VISION_MODEL; NUPHUS_MCP_VISION_BASE_URL and NUPHUS_MCP_VISION_PROVIDER are optional.
How do you use this agent?
Add the server to an MCP client's mcpServers configuration:
{
"mcpServers": {
"nuphus-mcp": {
"command": "nuphus-mcp",
"args": ["--confirm-write"]
}
}
}
After installation, a first smoke test is:
echo '{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test"}}}' | nuphus-mcpOnce the client has completed initialize and notifications/initialized, call tools/list and then tools/call with the desired desktop_* or browser_* tool. To attach to a user-managed browser, set NUPHUS_MCP_BROWSER_CDP_URL, such as http://127.0.0.1:9222; Chrome 136+ also requires remote debugging to use a non-default --user-data-dir. On macOS, grant the host process Accessibility permission before using desktop input.
How does this agent compare with similar options?
Unlike dsh-nuphus-mcp, nuphus-mcp is a general-purpose stdio MCP server rather than a DeepSeek Harness/Cordis plugin; DSH users wanting zero-configuration native mounting should choose the dedicated plugin. Its two browser-control paths have different boundaries: CDP offers DOM-adjacent capabilities such as accessibility snapshots, cookies, and tabs but requires a debuggable Chrome/Edge profile, while desktop_perceive plus mouse and keyboard can operate an already-running real browser or any desktop application without DOM access. For persistent sessions, the managed profile is the simplest option; copying a real profile preserves more extensions and login state but consumes storage and cannot run alongside the original profile.