A-Evolve

The framework reads and rewrites agent workspace files, and a custom evolution engine may choose whether to use validation and version control. The README describes holdout gates, Git rollback, and mutation tags, but the supplied material does not establish permissions or confirmation across all execution paths, so least privilege, user confirmation, and external effects score 1–2. The MCP agent obtains server keys through a key registry and imports a secret-redaction utility, but the excerpt does not establish end-to-end sensitive-data protection, so that criterion scores 1. Dependencies have minimum version bounds but no lock or security policy, scoring 1. The project cites a paper and names A-EVO Lab authors, but publisher identity is unverified, so source attribution scores 1.

The README's general Python 3.11+ requirement and CLAUDE.md's ARC SDK requirement of Python 3.12+ are not necessarily contradictory, but the general installation description does not clearly explain this branch-specific exception. More directly, the README says CC BY-NC 4.0 while pyproject declares MIT; this direct license conflict limits self-consistency to 1. Dependencies have optional extras and minimum versions, but no lockfile or evidence of availability across environments, scoring 1. The CLI gives a clear invalid-boolean message, and the ARC agent returns a structured error trajectory when arc-agi is missing, so failure messages score 2.

The framework explicitly targets researchers, agent authors, and benchmark authors, with custom agent, benchmark, and algorithm interfaces plus several built-in benchmark paths; audience and scenario coverage scores 3. The extension interfaces are described, but custom engines can bypass shared validation and rollback primitives, leaving capability boundaries incomplete at 2. The CLI exposes task directories, filters, modes, and scoring choices, but the general quick start does not clearly define when to use each, so trigger precision scores 2. The README's Python 3.11+ guidance differs from the ARC-specific environment requirement, and examples depend on external model, Docker, or benchmark environments, so environment fit scores 1.

The README is clearly organized around use cases, architecture, installation, algorithms, and community information, scoring 3. It provides PyPI extras and source installation, but benchmark and model environment requirements are spread across dedicated guides, scoring 2. The product, package, and import names are A-Evolve, a-evolve, and agent_evolve, and the packet refers to different branches, so naming stability scores 1. Quick starts, interface examples, and benchmark guide links are useful, though there is no FAQ, scoring 2. CLAUDE.md records incomplete ARC attempts and weak pure-LLM results, but the README does not systematically present framework limitations, scoring 1. The README's CC BY-NC 4.0 and pyproject's MIT declaration conflict, so license scores 0. News lists dated announcements, but there is no clear release-matched changelog, scoring 1. Issue and PR links exist, but ongoing maintenance responsibility and update process are not clearly stated, scoring 1.

The framework API, CLI JSON results, standard workspace contract, and benchmark adapters provide usable integration outputs. However, the supplied material does not support underlying reports for README benchmark figures and claims such as SOTA, so output usability scores 2. The unified agent, evaluation, and evolution interfaces have a clear potential reuse benefit, but the benefit is mostly asserted in the supplied material, scoring 2. Repeated evaluation cycles, model calls, containers, and external benchmarks can incur cost; the CLI exposes token, retry, task, and cycle settings, but cost estimates and budget controls are not sufficiently described, scoring 1.

Paper citations and algorithm and benchmark guides provide some paths to trace claims, but several README rankings and scores are not tied to verifiable reports or results in the supplied packet, scoring 1. The README, source, and tests complement one another on interfaces, workspaces, and some GEPA behavior, but tests cover sampled components and do not substantiate broad benchmark claims, scoring 1. The README gives a results-check date and some baseline comparisons, but the supplied material does not consistently separate evidence from promotional claims, scoring 1.