MLE-Agent
A CLI-oriented ML engineering assistant for building baselines, researching methods, and advancing local project workflows.
- Source repo
- MLSysOps/MLE-agent
- Stars
- ★ 1.6k
- Last updated
- 2mo ago
- License
- MIT
- Primary language
- Python
- FA score
- 41/100 · Major gaps
At a glance
- How it runs
- Works with
- Portable with changesOpenAI API · Claude API (Partial support)
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- An ML engineer who has a vague request, such as predicting stock prices from historical data, and wants to prototype and test a baseline locally.
- Not a fit if
- Users who want a GUI instead of a terminal-driven workflow
- Teams needing Hugging Face, S3, MLflow or W&B integrations out of the box
- Source review
- 41/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
MLE-Agent is presented as a pairing LLM agent for machine learning engineers and researchers. Its `mle` command-line interface creates and starts project directories and includes interactive terminal chat. The repository describes baseline construction, code execution and debugging on a local machine or cloud environment, plus research retrieval through Arxiv, Papers with Code, and general keyword search. It also documents Kaggle workflows, weekly-report generation, and local RAG support, while listing OpenAI GPTs, Anthropic Claude 3.5 Sonnet, and Ollama Llama3 among supported models. It fits ML teams that want one project-local workflow for planning, research, execution, and reporting, but the supplied material does not document provider credentials or model configuration.
After mle new <project name> creates a directory, mle start begins the ML/AI project workflow and mle chat opens an interactive terminal session. The project describes building ML/AI baselines from requirements, retrieving methods through Arxiv, Papers with Code, and general keyword search, and executing, debugging, and fixing code on a local machine or cloud environment. mle kaggle starts a Kaggle task, while mle kaggle --auto accepts dataset paths, a description, submission file, submission example, and competition ID for an automated workflow spanning data preparation, training, and debugging. For reporting, mle report starts a local web application at http://localhost:3000/ for GitHub-based reports, and mle report-local --email=<git email> --start-date=YYYY-MM-DD --end-date=YYYY-MM-DD <path_to_git_repo> creates a report from a local Git repository.
- An ML engineer who has a vague request, such as predicting stock prices from historical data, and wants to prototype and test a baseline locally.
- A Kaggle participant who has joined a competition and has the datasets and submission artifacts ready for the parameterized
mle kaggle --autoworkflow. - A researcher planning an ML approach who wants to consult Arxiv, Papers with Code, and keyword-search results alongside implementation work.
- A developer maintaining an existing ML project who wants a terminal-based
mle chatinteraction mode within that project directory. - A team member who needs a weekly work report covering development progress, communication notes, references, and to-do items from GitHub or a local Git repository.
How do you install or deploy this agent?
Install from PyPI:
pip install -U mle-agentOr with uv:
uv pip install -U mle-agentInstall from source:
git clone https://github.com/MLSysOps/MLE-agent.git
cd MLE-agent
uv venv .venv
source .venv/bin/activate
pip install -e .The supplied material lists support for providers including OpenAI, Anthropic, and Ollama, but does not document required credentials, environment variables, or model-configuration steps.
How do you use this agent?
Create and start a project:
mle new <project name>
cd <project name>
mle startStart terminal chat in that project:
mle chatStart a Kaggle workflow:
mle kaggleGenerate a report from a local Git repository:
mle report-local --email=<git email> --start-date=YYYY-MM-DD --end-date=YYYY-MM-DD <path_to_git_repo>What are this agent's strengths and limitations?
- Combines ML baseline creation, code execution and debugging, research retrieval, and project guidance in one project-level CLI workflow.
- Explicitly integrates Arxiv, Papers with Code, and general keyword search for bringing research references into implementation planning.
- Provides a parameterized
mle kaggle --autoflow that accepts practical competition inputs such as datasets, descriptions, and submission artifacts. - Includes terminal chat, a GitHub-report web entry point, and a local-Git reporting command rather than focusing only on coding.
- The README lists OpenAI GPTs, Anthropic Claude 3.5 Sonnet, and Ollama Llama3, indicating a multi-model integration direction.
- The supplied material does not document provider credentials, environment variables, configuration files, or a default model, so first-run setup requires further verification.
- The Kaggle automation mode requires that the user has joined the competition and prepared datasets, submission files, and a submission example.
- Its documented local filesystem and code-execution capabilities require evaluating the execution boundary for project code and data before adoption.
- Several integrations, including cloud data and testing platforms, Hugging Face, AWS S3, MLflow, and W&B, remain unchecked roadmap items.
- Although the README describes end-to-end Kaggle task completion, the supplied evidence provides no benchmark, success-rate, or resource-consumption data.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| MLE-Agent This agent | 41 · Major gaps | CLIFree + model costs | ★ 1.6k | 2mo ago | Python | — |
| optim-agent | 65 · Some gaps | Library / SDKFree + model costs | ★ 939 | 1mo ago | Python | Codex · Claude Code |
| Metaflow | 77 · Good | Library / SDKFree | ★ 10k | 12d ago | Python | — |
| Karpathy Agentic ML Engineer | 37 · Major gaps | CLIFree + model costs | ★ 1.6k | 1mo ago | Python | Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: API keys (e.g., LITELLM_API_KEY) and environment variables are used, but least privilege is not explicitly addressed; interactive CLI and user confirmation (e.g., mle start) exist, but not detailed; data flow transparency is limited, no explanation of data handling or storage; sensitive data handling is not explicit; many dependencies, but no security audit; external effects (e.g., code execution, network access) exist, but no explicit restrictions; no rollback mechanism; source attribution is partial, authors listed but not verified. Deductions: lack of specific security measures and transparency.
Evidence shows: Tests exist (e.g., test_litellm_model.py), but coverage is limited; many dependencies, but no availability guarantee; error messages are partial, e.g., tests handle auth errors, but not comprehensive. Deductions: insufficient test coverage, dependency availability not verified.
Evidence shows: Targeted at ML engineers and researchers, scenarios clear (e.g., Kaggle, report generation); capability boundaries partial, feature list but no explicit limits; trigger precision moderate, CLI commands but not detailed; environment fit partial, supports multiple models and OS, but not detailed. Deductions: capability boundaries and trigger precision not explicit enough.
Evidence shows: Information architecture clear, README and docs links; install notes detailed; naming stable, CLI commands; examples and FAQ present; known limitations partial, roadmap but not explicit; license is MIT, but pyproject.toml says Apache-2.0, inconsistency; versioning changelog partial, milestones but no detailed changelog; maintenance responsibility partial, authors listed but not explicit. Deductions: license inconsistency, changelog not detailed.
Evidence shows: Output usability high, CLI and Web UI; marginal value high, automates ML tasks; cost-benefit not assessed. Deductions: cost-benefit not evaluated.
Evidence shows: Claim traceability partial, docs and tests; cross-source corroboration partial, tests and CI; fact-inference separation not explicit. Deductions: lack of explicit separation of facts and inferences.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- License inconsistency: README and LICENSE are MIT, but pyproject.toml declares Apache-2.0, needs clarification.
- Many dependencies without security audit, recommend checking dependency security.
- Data flow and sensitive data handling are opaque, assess privacy risks before use.
FAQ
Which model-provider credentials do I need?
Does it access or execute local content?
Can it run a Kaggle competition fully automatically?
mle kaggle --auto, but requires that you have joined the competition and have datasets, a description, submission file, submission example, and competition ID ready.