A-Evolve
Evolves an agent’s prompts, skills, and memory from benchmark feedback.
What does this agent do, and when should you use it?
A-Evolve is Python infrastructure for evolving an existing agent, not a standalone end-user agent. Developers provide an agent workspace and benchmark adapter to `ae.Evolver`, which runs a Solve, Observe, Evolve, Gate, Reload loop. Its filesystem workspace contract covers `manifest.yaml`, prompts, `SKILL.md` skills, tool configuration, and JSONL memory. Evolution engines modify those files from trajectories and evaluation feedback, then validate candidates on holdout tasks and can roll back regressions through Git. The repository includes adapters and seed workspaces for SWE-bench Verified, MCP-Atlas, Terminal-Bench, SkillsBench, and CL-Bench, while exposing interfaces for custom agents, benchmarks, and algorithms.
When Evolver.run(cycles=10) is called, A-Evolve has the agent run solve() on task batches, collects Trajectory objects and benchmark feedback, and records structured observations. An EvolutionEngine.step() implementation can inspect the workspace, observations, history, and trial context before changing prompt, skill, or memory files. The Gate phase validates a candidate on holdout tasks; regressed mutations are reverted through Git, while accepted versions are tagged as evo-1, evo-2, and so on. A custom agent implements BaseAgent.solve(self, task: Task) -> Trajectory; a custom benchmark implements BenchmarkAdapter.get_tasks() and .evaluate().
- A research team improving a code-repair agent on SWE-bench Verified can begin with the built-in `swe-verified` seed workspace and adapter.
- A developer building an MCP tool-calling agent can use `mcp-atlas` and let evaluation feedback drive changes to skill files.
- An algorithm researcher testing a new self-improvement strategy can implement `EvolutionEngine.step()` while reusing trial, history, and Git rollback primitives.
- A team with an existing custom agent that wants a benchmark-driven loop for prompts, skills, and memory can wrap it as a `BaseAgent`.
- An evaluator comparing evolution approaches for terminal-operation or skill-discovery tasks can use the Terminal-Bench or SkillsBench examples.
What are this agent's strengths and limitations?
- Uses a standard directory as the evolvable state, so prompts, skills, and memory can be processed through one workflow without requiring knowledge of the agent’s internals.
- Separately pluggable `BaseAgent.solve()`, `BenchmarkAdapter`, and `EvolutionEngine.step()` interfaces let adopters replace the agent, evaluation, or evolution strategy.
- Candidate changes are checked on holdout tasks, with Git rollback for regressions and `evo-*` tags for accepted versions.
- Includes benchmark adapters and seed workspaces spanning code repair, MCP tool calling, terminal operations, skill discovery, and continual-learning evaluation.
- Outcomes depend on the benchmark, base model, evolution algorithm, and task setup; the reported results do not establish comparable gains for a custom agent.
- It automatically changes workspace files and relies on Git-style version control, so teams must define which prompts, skills, and memory files are safe to evolve.
- Although Anthropic, OpenAI, and AWS Bedrock are listed as provider examples, the supplied material does not specify credential setup, model selection, or cost controls.
- The Terminal-Bench domain explicitly involves Docker, so adopting that adapter may require a container runtime.
How do you install or deploy this agent?
Python 3.11+ is required. Install the core package with pip install a-evolve; Claude support with pip install a-evolve[anthropic]; MCP-Atlas support with pip install a-evolve[mcp]; SWE-bench support with pip install a-evolve[swe]; or all extras with pip install a-evolve[all]. For source development, run git clone https://github.com/A-EVO-Lab/a-evolve.git && cd a-evolve, then pip install -e ".[all,dev]". The README names Anthropic, OpenAI, and AWS Bedrock as LLM Provider examples, but does not document the exact credential configuration; configure authentication for the selected provider before use.
How do you use this agent?
For a minimal run, import with import agent_evolve as ae, create evolver = ae.Evolver(agent="swe-verified", benchmark="swe-verified"), then run results = evolver.run(cycles=10) and inspect results.final_score and results.converged. To evolve your own agent, subclass BaseAgent, implement solve() to return a Trajectory, and pass it as ae.Evolver(agent=MyAgent("./my_workspace"), benchmark="mcp-atlas"). The agent workspace should follow the documented filesystem contract and be reloadable by the agent after each cycle; accepted changes are Git-tagged.