AIDE ML
Iteratively writes, evaluates, and improves machine-learning code through tree search.
- Source repo
- WecoAI/aideml
- Stars
- ★ 1.6k
- Last updated
- 23d ago
- License
- MIT
- Primary language
- Python
- FA score
- 53/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API · Claude API
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- An ML practitioner with a tabular dataset who wants to explore modeling code from a request such as churn prediction measured by AUROC.
- Not a fit if
- Users who want a hosted service instead of running code locally
- Teams needing production-grade experiment tracking and controls
- Source review
- 53/100 · Major gaps
What does this agent do, and when should you use it?
AIDE ML is the open-source reference implementation of the AIDE algorithm for machine-learning engineering tasks defined by a dataset, goal, and evaluation metric. It uses LLM-guided tree search over code: Python scripts become solution-tree nodes, generated patches create children, and metric feedback guides or prunes exploration. The repository includes the aide CLI, the aide.Experiment Python API, an HTML solution-tree visualizer, and a Streamlit web UI. A run writes the best code to logs/<id>/best_solution.py and an inspectable tree to logs/<id>/tree_plot.html. It is positioned for research and prototyping; the repository positions Weco as the related platform for broader code optimization, experiment tracking, and enhanced control.
A user supplies a data directory, natural-language goal, and metric through aide data_dir=... goal=... eval=... or aide.Experiment(data_dir=..., goal=..., eval=...). AIDE ML treats each Python script as a solution-tree node, calls the configured coding model to generate patches and child nodes, evaluates the code, and uses the resulting metric to steer search. Experiment.run(steps=...) returns the best solution, including valid_metric and code. CLI runs write logs/<id>/best_solution.py and logs/<id>/tree_plot.html; the web UI accepts an API key and uploaded data, lets users set Goal and Metric, and displays live logs, the solution tree, and the best code.
- An ML practitioner with a tabular dataset who wants to explore modeling code from a request such as churn prediction measured by AUROC.
- An agent-architecture researcher who wants to swap search heuristics, evaluators, or LLM backends and inspect the resulting code-search tree.
- A researcher reproducing the AIDE paper or testing a new code-generation and evaluation strategy.
- A team testing
qwen2.5through a local Ollama endpoint as the coding model for iterative ML optimization. - A user who prefers a browser workflow for uploading data, setting a goal and metric, starting AIDE, and watching results in Streamlit.
How do you install or deploy this agent?
Python 3.10+ is required. Install with pip install -U aideml. For the documented OpenAI quick start, set export OPENAI_API_KEY=<your-key>. Run: aide data_dir="example_tasks/house_prices" goal="Predict the sales price for each house" eval="RMSE between log-prices". For development installation: git clone https://github.com/WecoAI/aideml.git, then cd aideml && pip install -e ..
How do you use this agent?
Use agent.code.model to choose a coding model and agent.steps to control iterations, for example: aide agent.code.model="claude-4-sonnet" agent.steps=50 data_dir=... goal=... eval=.... In Python, create aide.Experiment(...) and call exp.run(steps=2). For the UI, change to aide/webui and run streamlit run app.py. For Ollama through an OpenAI-compatible endpoint, set OPENAI_BASE_URL="http://localhost:11434/v1" and use agent.code.model="qwen2.5"; for a fully local code-and-evaluator setup, also set agent.feedback.model="qwen2.5".
What are this agent's strengths and limitations?
- Its code-tree search makes scripts, LLM-generated patches, and metric feedback explicit, rather than presenting exploration as a single linear edit sequence.
- It provides a CLI, the
aide.ExperimentAPI, an HTML tree visualizer, and a Streamlit UI for scripted, embedded, and interactive workflows. - The documentation names OpenAI, Anthropic, Gemini, and OpenAI API-compatible local-model paths, with a concrete Ollama configuration.
- Runs preserve both the best Python code and the full solution tree for reviewing the final result and the search path.
- Initial use needs access to an LLM service and its applicable credentials; the documented default coding model is
gpt-4-turbo. - Local-model mode can still default to
gpt-4ofor evaluation; a fully local setup requires an extra feedback-model setting, and the documentation warns of possible performance loss. - Evaluation depends on the user-supplied data directory, goal, and metric; the supplied material does not establish universal support for arbitrary data formats or task types.
- The repository describes itself as a lean, research-friendly implementation; broader code optimization, experiment tracking, and enhanced controls are positioned in a separate platform product.
How does this agent compare with similar options?
The README cites an MLE-Bench result across 75 Kaggle competitions in which AIDE tree search earned four times as many medals as the best linear agent, OpenHands. That comparison concerns medals in that benchmark and does not establish a general performance result for every ML task.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| AIDE ML This agent | 53 · Major gaps | CLIFree + model costs | ★ 1.6k | 23d ago | Python | OpenAI API · Claude API |
| AI Engineering Bootcamp | 38 · Major gaps | FrameworkFree | ★ 956 | 28d ago | Jupyter Notebook | — |
| K-Dense BYOK | 73 · Some gaps | Desktop appFree + model costs | ★ 1.4k | 23d ago | TypeScript | ChatGPT · Codex · Claude.ai |
| BambooAI Data Analyst | 70 · Some gaps | Web appFree + model costs | ★ 842 | today | Python | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
The code sends task and execution-review requests through the configured model backend and runs generated scripts through an interpreter. The README documents external APIs and local endpoints, plus where logs and results appear, so data flows are partly visible (data_flow_transparency: 2). The supplied code does not show sandboxing or permission isolation, pre-run confirmation, general sensitive-data protection, limits on external side effects, or recovery. Removing API keys from the subprocess is shown only in a particular E2E test, not as a general runtime safeguard, so those criteria score low. The README cites the algorithm paper and requests citation, and the license names a copyright holder, supporting source attribution (2); maintainer identity and update path remain unclear.
agent.py warns and falls back when metric inference fails. The README describes errors for missing structured responses, truncation, and refusals, and bounded retries; visible tests cross-check metric classification and Anthropic tool requests, so self-consistency and failure messages score 2. The dependency list contains many broad or unpinned ML packages and several lower-bound versions, with no complete lock or security process shown, so dependency availability scores 1.
The README names agent-architecture researchers and ML practitioners and provides CLI, Python, Streamlit, and local-model paths. Goals, metrics, providers, and iteration counts are configurable, with concrete environment examples, supporting scores of 2 for audience, trigger precision, and environment fit. Capability boundaries are less clear: generated programs may use arbitrary relevant packages and are executed by the interpreter, while the sample does not describe execution isolation or firm task limits, so that criterion scores 1.
The README organizes quick start, UI, CLI, Python, extras, and citation material, and gives installation, examples, configuration guidance, and a local-model performance caveat. The full MIT license is present and scores 3. Version history is only weakly supported by the supplied material; the README gives current options and test workflow but no clear maintainer responsibility or update channel, so those criteria score 1.
The product describes tree search, iterative code improvement, metric feedback, logs, and visualization, and offers reusable CLI and Python interfaces, showing practical output value and research/engineering utility (2). Running it requires model calls and training-script execution. The README gives usage steps, an example call cap, and local-model options, but no general cost estimate or cost-control commitment, so cost-benefit scores 1.
The README links the main algorithm claims to a paper and distinguishes the algorithm, repository implementation, and Weco product. The workflow, tests, and source corroborate some configuration and behavior. The sample distinguishes mocked regression checks from real API checks and limits what the latter establish, supporting scores of 2 for traceability, corroboration, and fact/inference separation. The paper and performance claims are not independently validated by this static review; these scores reflect only the evidence trail in the supplied material.
- Generated training scripts are executed; the supplied material does not show general runtime sandboxing, permission limits, or side-effect recovery.
- Runs may send task and data context to model providers and incur API costs; confirm data may be sent externally and set an appropriate budget.
- The dependency list includes many ML packages and does not pin every version; the supplied material shows no comprehensive dependency lock or vulnerability-handling evidence.
Maintain this project? Add the FARS badge to your README
[](https://followagents.com/en/agents/aide)<a href="https://followagents.com/en/agents/aide"><img src="https://followagents.com/badge/aide.svg" alt="FARS review"></a>Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
What's Changed * :bug: minor fix for o1 models by @dexhunter in https://github.com/WecoAI/aideml/pull/42 * :bug: better handling for function calling models (#43) by @dexhunter in https://github.com/WecoAI/aideml/pull/44 * ⬆️ Bump scikit-learn from 1.2.2 to 1.5.0 by @dependabot[bot] in https://github.com/WecoAI/aideml/pull/46 * :bug: fix outdated error by @dexhunter in https://github.com/WecoAI/aideml/pull/49 * ⬆️ Bump tqdm from 4.66.2 to 4.66.3 by @dependabot[bot] in https://github.com/WecoAI/aideml/pull/48 …View on GitHub ↗ -
v0.2.0 aideml v0.2.0
Release notes excerpt
What's Changed * pass the cfg.desc_file to open by @thesofakillers in https://github.com/WecoAI/aideml/pull/7 * Direct any model (named without claude-) to the OpenAI backend. by @namin in https://github.com/WecoAI/aideml/pull/14 * Configure and enable final report generation from journal. by @namin in https://github.com/WecoAI/aideml/pull/15 * :memo: Update README to include local llm usage by @dexhunter in https://github.com/WecoAI/aideml/pull/17 * :whale: Add Dockerfile by @dexhunter in https://github.com/WecoAI/aideml/pull/18 …View on GitHub ↗ -
v0.1.4 aideml v0.1.4
FAQ
What credentials does it need?
OPENAI_API_KEY. The repository also documents configuration for Anthropic, Gemini, and OpenAI API-compatible local models; its Ollama example points OPENAI_BASE_URL at a local endpoint.Can it run fully locally?
OPENAI_BASE_URL to Ollama and set both agent.code.model and agent.feedback.model to a local model. The documentation notes that performance may decline.What artifacts does a run produce?
logs/<id>/best_solution.py and logs/<id>/tree_plot.html. The Python API also exposes the returned best solution's validation metric and code.Is it a complete production ML platform?
Thanks — we use this to improve the page.