Taskhomie
A local AI desktop assistant that operates your screen, browser, and terminal from natural-language instructions.
What does this agent do, and when should you use it?
Taskhomie is a local desktop AI agent built with Tauri, React, and Rust. It accepts natural-language instructions and can take screenshots, move the mouse, click, type, and run terminal commands. Computer Use Mode takes control of the screen, cursor, and keyboard for tasks spanning applications, while Background Mode automates the web through Chrome DevTools Protocol and uses the terminal for other work without taking over input devices. It runs locally as a Tauri app in development or production builds, but model access requires an Anthropic API key; the UI offers Haiku, Sonnet, and Opus selections.
After a user provides a natural-language task, the app started with npm run tauri dev can run one of two workflows. Computer Use Mode reads the current desktop through screenshots, then moves the mouse, clicks, types, and can run terminal commands; it directly controls the cursor and keyboard. Background Mode uses Chrome DevTools Protocol for web automation and the terminal for everything else, so it does not touch the mouse or keyboard. Shortcuts are ⌃⇧C for Computer Use Mode, ⌃⇧B for Background Mode, ⌘⇧H for help mode, and ⌘⇧S to stop the agent.
- An individual completing a task across several desktop applications can use Computer Use Mode to let the agent act from what it sees in screenshots.
- A developer working across a browser and terminal can run Background Mode for web automation and terminal work while retaining use of the mouse and keyboard.
- A macOS user who wants to start a spoken task quickly can use `⌃⇧C` or `⌃⇧B` to choose foreground or background execution.
- A user who wants a screenshot-based quick prompt can invoke help mode with `⌘⇧H`.
- A user supervising desktop automation can stop the agent immediately with `⌘⇧S`.
What are this agent's strengths and limitations?
- It offers two distinct execution models: direct desktop control in Computer Use Mode and input-device-free Background Mode.
- Background Mode explicitly combines Chrome DevTools Protocol web automation with terminal-based work.
- The Tauri, React, Rust, and Tokio stack supports both `npm run tauri dev` development runs and `npm run tauri build` production builds.
- An Anthropic API key is required, creating a dependency on Anthropic as the model provider.
- Computer Use Mode directly controls the screen, cursor, and keyboard, so those devices must be relinquished while it runs.
- macOS requires Accessibility permission, and the supplied material does not document support for other operating systems.
- The license information conflicts: repository metadata says NOASSERTION while the README links to Apache License 2.0.
How do you install or deploy this agent?
Requirements: Rust and Cargo, Node.js and npm, and an Anthropic API key.
Run:npm installecho "ANTHROPIC_API_KEY=your-key-here" > .env
For development: npm run tauri dev
For a production build: npm run tauri build
On macOS, grant Accessibility permission when prompted in System Settings → Privacy & Security → Accessibility.
How do you use this agent?
Start the app and give it a natural-language task. Press ⌃⇧C to use push-to-talk with Computer Use Mode, which directly operates the screen, mouse, and keyboard. Press ⌃⇧B for Background Mode, which uses Chrome DevTools Protocol and the terminal without controlling input devices. Use ⌘⇧H for a screenshot plus quick prompt, and ⌘⇧S to stop the agent.
How does this agent compare with similar options?
The project presents two internal alternatives: Computer Use Mode directly controls the desktop, while Background Mode runs asynchronously through Chrome DevTools Protocol and the terminal without using the mouse or keyboard.