Open Computer Use
An open-model agent that operates an isolated cloud desktop through keyboard, mouse, and shell actions.
What does this agent do, and when should you use it?
Open Computer Use is a cloud Linux computer powered by E2B Desktop Sandbox and controlled by open-source LLMs. The agent operates the sandbox through keyboard, mouse, and shell commands, while its display is live-streamed to the client computer. It includes a web interface where users can pause the agent and provide another prompt at any time. Ubuntu is the documented operating system, although the project says the design is intended to work with other operating systems. Model selection is configured in os_computer_use/config.py, with provider integrations defined in providers.py.
After receiving a prompt, the agent controls the cloud Linux computer supplied by E2B Desktop Sandbox using keyboard, mouse, and shell commands, and streams the sandbox display to the client. A user can pause execution and send an additional prompt. os_computer_use/config.py selects the models, for example with providers.OSAtlasProvider() for grounding and providers.GroqProvider("llama3.2") or providers.GroqProvider("llama3.3") for vision and actions. The documented providers in providers.py include Fireworks, OpenRouter, Llama API, Groq, DeepSeek, Google, OpenAI, Anthropic, HuggingFace Spaces, Moonshot, and Mistral AI.
- A developer who needs an LLM to operate keyboard, mouse, and shell controls inside an isolated cloud Ubuntu desktop.
- An operator who wants to watch the remote desktop live and interrupt the agent when a task needs correction.
- A team evaluating Groq, OpenAI, Anthropic, Gemini, or another documented provider by changing model configuration in config.py.
- A researcher experimenting with OS-Atlas or ShowUI as the agent's grounding model.
- A user who wants to start the web interface with an initial CLI prompt, such as asking the agent to use a web browser for a weather lookup.
What are this agent's strengths and limitations?
- Uses E2B Desktop Sandbox for an isolated cloud Linux desktop and live-streams that desktop to the client.
- Separates grounding, vision, and action model configuration, enabling documented provider and model combinations in config.py.
- Documents grounding support for OS-Atlas and ShowUI alongside multiple API-based model providers.
- Users can pause the agent and send another prompt while it is operating the computer.
- Core operation depends on E2B Desktop Sandbox and requires an E2B API key.
- Except for HuggingFace Spaces, selected model providers require separate API credentials, adding account, billing, and secret-management overhead.
- The documented setup uses brew and requires Poetry and ffmpeg; equivalent instructions for other installation environments are not supplied.
- Ubuntu is the explicitly documented operating system; support details for other operating systems are not provided.
How do you install or deploy this agent?
Prerequisites are Python 3.10 or later, git, an E2B API key, and an API key for the selected LLM provider. Install Poetry and ffmpeg with: brew install poetry ffmpeg. Clone the project with: git clone https://github.com/e2b-dev/open-computer-use/. Enter it with: cd open-computer-use. Create a .env file containing E2B_API_KEY="your-e2b-api-key", then add the key required by the provider selected in config.py, such as GROQ_API_KEY, OPENAI_API_KEY, or ANTHROPIC_API_KEY. For HuggingFace Spaces, the documented setup requires HF_TOKEN to bypass Gradio rate limits. Run: poetry install.
How do you use this agent?
From the project directory, run: poetry run start. This starts the web interface, which opens and asks for the first instruction. To supply one at launch, run: poetry run start --prompt "use the web browser to get the current weather in sf". The sandbox display stream should appear a few seconds after the Python program starts. Change grounding_model, vision_model, and action_model in os_computer_use/config.py as needed, and ensure the matching provider credentials are in .env.