Agent-E Web Automation
Turn natural-language requests into multi-step browser research and actions.
- Source repo
- EmergenceAI/Agent-E
- Stars
- ★ 1.3k
- Last updated
- 1y ago
- License
- MIT
- Primary language
- Python
- FA score
- 46/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- An operations specialist handling repetitive web forms can have Agent-E collect supplied information, populate fields, and pause for review before submission.
- Not a fit if
- Teams needing full desktop automation, not just browser tasks
- Users who need reliable multi-tab or PDF form handling
- Source review
- 46/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
Agent-E is a self-hosted, AG2-based computer automation system whose current scope is the web browser. Its documented open-source architecture combines a User proxy that executes skills with a Browser navigation agent that owns browser-interaction skills. The system distills the browser accessibility tree into compact text, input-field, or all-field views before selecting actions such as clicking, entering text, and opening URLs. Tasks can be submitted through a browser chat interface or through a FastAPI /execute_task endpoint that streams results. It is a practical fit for browser-based retrieval and workflows, but not yet a general desktop agent: single-tab state, dynamic controls, Shadow DOM, PDFs, and canvas-based Google applications remain documented limitations or planned work.
Agent-E accepts a natural-language task and uses AG2 agents to select and execute defined skills. geturl, get_dom_with_content_type, and the optional get_user_input inspect the URL, page state, and user feedback; click, enter_text, bulk_enter_text, enter_text_and_click, and openurl manipulate the browser. get_dom_with_content_type can return text_only, input_fields, or all_fields; its distillation process uses the DOM Accessibility Tree and injects an mmid attribute so the model can target elements with DOM query selectors. Each skill returns a natural-language account of its outcome, giving the agent context for its next decision. Users can start tasks from the browser chat UI or send them to FastAPI with POST /execute_task; API callers may also supply separate llm_config values for planner_agent and browser_nav_agent. Execution changes browser and form state, while chat logs can be written to files or printed to standard output according to configuration.
- An operations specialist handling repetitive web forms can have Agent-E collect supplied information, populate fields, and pause for review before submission.
- A purchasing or e-commerce researcher can ask it to find products, apply sorting criteria, open a result, and perform an action such as adding the item to a cart.
- A researcher can use it to visit news, sports, university, or local-business websites, locate requested facts, and report visible page content.
- A project team working in a browser-based platform such as JIRA can use natural-language requests to navigate pages and filter issues.
- An engineering team can expose the self-hosted FastAPI endpoint inside an internal workflow and provide separate model settings for the planner and browser navigation agents.
- A contributor evaluating browser behavior can run the JSON tasks under
test/tasksagainst live websites withpython -m test.run_tests.
How do you install or deploy this agent?
The documented runtime is Python 3.10 or newer, with Python 3.11 recommended. On macOS or Linux, run ./install.sh from the project root, or ./install.sh -p to install Playwright support without another prompt. On Windows PowerShell, use .\win_install.ps1 or .\win_install.ps1 -p. For manual setup, install uv, then run uv venv --python 3.11, source .venv/bin/activate, uv pip compile pyproject.toml -o requirements.txt, and uv pip install -r requirements.txt; use .venv\Scripts\activate on Windows. Run playwright install if Google Chrome is unavailable. Copy the configuration with cp .env-example .env, then set at least AUTOGEN_MODEL_NAME and AUTOGEN_MODEL_API_KEY. For a non-OpenAI endpoint, also configure AUTOGEN_MODEL_BASE_URL and, when required by the provider, AUTOGEN_MODEL_API_TYPE and AUTOGEN_MODEL_API_VERSION. To reuse local Chrome, obtain its profile path from chrome://version/ and assign it to BROWSER_STORAGE_DIR.
How do you use this agent?
After configuration, start Agent-E from the repository root with python -m ae.main; on macOS, python -u -m ae.main is the documented workaround for BlockingIOError. When the Agent-E icon appears in the browser, open its chat interface and try a command such as open youtube and search for funny cat videos. For programmatic use on Linux or macOS, launch uvicorn ae.server.api_routes:app --reload --loop asyncio; on Windows, use uvicorn ae.server.api_routes:app --loop asyncio. A first API invocation is: curl --location 'http://127.0.0.1:8000/execute_task' --header 'Content-Type: application/json' --data '{"command":"go to espn, look for soccer news, report the names of the most recent soccer champs"}'. The request may additionally contain an llm_config object with separate planner_agent and browser_nav_agent settings. Run the complete test suite with python -m test.run_tests.
What are this agent's strengths and limitations?
- DOM Distillation can reduce the accessibility tree to text, interactive inputs, or all distilled fields instead of passing an unfiltered HTML document to the model.
- Browser behavior is exposed through named sensing and action skills rather than unrestricted model-generated code, creating a clearer execution boundary.
- It provides both an interactive browser experience and a self-hosted FastAPI
/execute_taskinterface for application integration. - Model configuration covers OpenAI and custom compatible endpoints, with a documented LiteLLM and Ollama route for local open-source models.
- The repository includes live-site JSON tasks, test-range controls, result identifiers, and an option to capture screenshots after operations.
- The present scope is browser automation rather than complete desktop control, and form handling does not yet cover PDFs; canvas-based Google applications remain planned work.
- The documented implementation currently handles only one tab reliably and can lose state when another tab opens.
- Live-site changes make test outcomes variable, while complex dropdowns, Shadow DOM, and losses during DOM distillation are acknowledged areas for improvement.
- Adoption requires a Python environment, model configuration, and either Chrome or Playwright; remote-model deployments also depend on network access and a model provider.
- The LiteLLM and Ollama path is documented but explicitly described as not thoroughly tested, so moving to local or less capable models may require prompt simplification or skill refactoring.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Agent-E Web Automation This agent | 46 · Major gaps | CLIFree + model costs | ★ 1.3k | 1y ago | Python | OpenAI API |
| Camoufox CLI | 56 · Major gaps | CLIFree | ★ 360 | 25d ago | Python | Codex · Claude Code |
| Auto Browser | 76 · Some gaps | Self-hosted serviceFree + model costs | ★ 924 | 1d ago | Python | OpenAI API · Claude API |
| Heym | 63 · Some gaps | Self-hosted serviceFree + model costs | ★ 1.4k | today | Python | Codex · Claude.ai |
How does FollowAgents rate this agent?
Why each dimension lost points
The README describes a bounded set of browser skills, says configured skills are intended to avoid running unknown code, and shows a form example that waits for user review before submission. However, it gives no general confirmation policy, permission boundaries, or rollback mechanism; it also allows extra skills and use of a local Chrome profile. Logging is enabled by default, while handling of model API and browser data is only lightly described, so most criteria score 1. License and author details support source attribution (2). The packet does not show evidence sufficient to identify a red-line risk.
The README says DOM distillation is still being refined, live-site tests may be inconsistent, and lists some environment problems and remedies, supporting scores of 1 for consistency, dependency availability, and failure guidance. There are visible documentation gaps: the README says Python 3.10+ should work and package metadata requires >=3.10, but the dependency list does not establish full compatibility or availability guarantees. The supplied test files are excerpts and do not support higher static scores.
The README targets natural-language browser automation and lists forms, search, media, and project-management tasks. It also says the focus is currently the browser, PDF forms are unsupported, and local models have not been thoroughly tested, supporting 2 for scenarios and capability boundaries. Triggering is described mainly through user commands or an HTTP endpoint, without precise applicability or refusal boundaries, so trigger precision scores 1. Installation instructions cover macOS/Linux and Windows and offer local Chrome or Playwright, supporting environment fit at 2.
The README is organized around installation, configuration, operation, advanced usage, troubleshooting, demos, and architecture, with commands and examples; it also states some limitations. The MIT license text is complete and scores 3. Naming and release information are limited to the project name and version 0.1.0; the README has typos and inconsistencies, and no changelog or clear maintenance responsibility is supplied, so those criteria score 1.
The README describes atomic browser actions returning natural-language outcomes and demonstrates research, form-filling, and shopping tasks, supporting scores of 2 for output usability and marginal value. Cost-benefit evidence is limited: the project depends on several model and service options, and local models are described as insufficiently tested, but there is no cost or performance tradeoff data, so cost-benefit scores 1.
README claims about features, installation, and limitations map to the skill list, dependency configuration, and test entry point, supporting claim traceability at 2. The test-file excerpt is incomplete and provides limited independent corroboration of effectiveness, so cross-source corroboration scores 1. The documentation explicitly identifies known limitations and insufficiently tested behavior, helping separate stated facts from unconfirmed outcomes; fact-inference separation scores 2.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- The README enables chat-log saving by default and supports a local Chrome profile; clarify how log contents and browser session data are handled before use.
- Browser automation can perform actions with external effects. One example waits for review before submission, but the README does not establish this as a general confirmation requirement or describe rollback.
- The project says DOM distillation is still being refined, live-site tests may be inconsistent, and local models are not thoroughly tested; static evidence does not establish runtime outcomes.
Maintain this project? Add the FARS badge to your README
[](https://followagents.com/en/agents/agent-e)<a href="https://followagents.com/en/agents/agent-e"><img src="https://followagents.com/badge/agent-e.svg" alt="FARS review"></a>FAQ
Do I have to buy the managed service to use Agent-E?
Is an OpenAI model mandatory?
AUTOGEN_MODEL_BASE_URL and related variables support other compatible services. A LiteLLM-plus-Ollama local route is also documented, although the project says it has not been thoroughly tested.What sensitive resources can it access?
BROWSER_STORAGE_DIR points to a local Chrome profile, automation runs in that browser-profile environment, so adopters should control the selected profile, website sessions, model API credentials, and generated logs.Why might a browser task fail intermittently?
Can it wait for a person before submitting a form?
PLANNER_USER_INPUT_SKILL_ENABLED.Thanks — we use this to improve the page.