Agent-E Web Automation

What does this agent do, and when should you use it?

Agent-E is a self-hosted, AG2-based computer automation system whose current scope is the web browser. Its documented open-source architecture combines a User proxy that executes skills with a Browser navigation agent that owns browser-interaction skills. The system distills the browser accessibility tree into compact text, input-field, or all-field views before selecting actions such as clicking, entering text, and opening URLs. Tasks can be submitted through a browser chat interface or through a FastAPI `/execute_task` endpoint that streams results. It is a practical fit for browser-based retrieval and workflows, but not yet a general desktop agent: single-tab state, dynamic controls, Shadow DOM, PDFs, and canvas-based Google applications remain documented limitations or planned work.

Agent-E accepts a natural-language task and uses AG2 agents to select and execute defined skills. geturl, get_dom_with_content_type, and the optional get_user_input inspect the URL, page state, and user feedback; click, enter_text, bulk_enter_text, enter_text_and_click, and openurl manipulate the browser. get_dom_with_content_type can return text_only, input_fields, or all_fields; its distillation process uses the DOM Accessibility Tree and injects an mmid attribute so the model can target elements with DOM query selectors. Each skill returns a natural-language account of its outcome, giving the agent context for its next decision. Users can start tasks from the browser chat UI or send them to FastAPI with POST /execute_task; API callers may also supply separate llm_config values for planner_agent and browser_nav_agent. Execution changes browser and form state, while chat logs can be written to files or printed to standard output according to configuration.

1. An operations specialist handling repetitive web forms can have Agent-E collect supplied information, populate fields, and pause for review before submission.
2. A purchasing or e-commerce researcher can ask it to find products, apply sorting criteria, open a result, and perform an action such as adding the item to a cart.
3. A researcher can use it to visit news, sports, university, or local-business websites, locate requested facts, and report visible page content.
4. A project team working in a browser-based platform such as JIRA can use natural-language requests to navigate pages and filter issues.
5. An engineering team can expose the self-hosted FastAPI endpoint inside an internal workflow and provide separate model settings for the planner and browser navigation agents.
6. A contributor evaluating browser behavior can run the JSON tasks under test/tasks against live websites with python -m test.run_tests.

How do you install or deploy this agent?

The documented runtime is Python 3.10 or newer, with Python 3.11 recommended. On macOS or Linux, run ./install.sh from the project root, or ./install.sh -p to install Playwright support without another prompt. On Windows PowerShell, use .\win_install.ps1 or .\win_install.ps1 -p. For manual setup, install uv, then run uv venv --python 3.11, source .venv/bin/activate, uv pip compile pyproject.toml -o requirements.txt, and uv pip install -r requirements.txt; use .venv\Scripts\activate on Windows. Run playwright install if Google Chrome is unavailable. Copy the configuration with cp .env-example .env, then set at least AUTOGEN_MODEL_NAME and AUTOGEN_MODEL_API_KEY. For a non-OpenAI endpoint, also configure AUTOGEN_MODEL_BASE_URL and, when required by the provider, AUTOGEN_MODEL_API_TYPE and AUTOGEN_MODEL_API_VERSION. To reuse local Chrome, obtain its profile path from chrome://version/ and assign it to BROWSER_STORAGE_DIR.

How do you use this agent?

After configuration, start Agent-E from the repository root with python -m ae.main; on macOS, python -u -m ae.main is the documented workaround for BlockingIOError. When the Agent-E icon appears in the browser, open its chat interface and try a command such as open youtube and search for funny cat videos. For programmatic use on Linux or macOS, launch uvicorn ae.server.api_routes:app --reload --loop asyncio; on Windows, use uvicorn ae.server.api_routes:app --loop asyncio. A first API invocation is: curl --location 'http://127.0.0.1:8000/execute_task' --header 'Content-Type: application/json' --data '{"command":"go to espn, look for soccer news, report the names of the most recent soccer champs"}'. The request may additionally contain an llm_config object with separate planner_agent and browser_nav_agent settings. Run the complete test suite with python -m test.run_tests.

What are this agent's strengths and limitations?

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

How does FollowAgents rate this agent?

FAQ