Kapso
A self-improving AI software factory for measurable objectives: state a goal and it iterates experiments, coding agents, and accumulated knowledge until the goal is met and the result ships.
README documents CLI-agent dependencies, an inbox pause for human confirmation, local-only knowledge bank by default, and a name-collision warning against the unrelated PyPI 'kapso' package — all good. Deductions: no core source provided, so least privilege, data flow, and external effects (deploy, web research, KG writes) are narrative-only; API-key-into-.env guidance lacks further sensitive-data handling detail.
kapso doctor preflight, configurable live model probes, and fail-fast on missing CLIs show deliberate failure-message design. Deduction: requirements.txt contradicts pyproject.toml (it still lists aider-chat, google-genai, openhands while pyproject claims aider moved to an extra), hurting self-consistency.
Audience (ML engineers/researchers) is clear; examples and benchmarks span scenarios; doctor adapts to config, custom model configs supported. Deductions: capability boundaries (subscription caps, timeout recalibration, ingest scaling) are scattered prose without a systematic boundaries statement; trigger precision cannot be checked without core code.
Complete MIT license, thorough install notes (name collision, optional extras, source install), heavily annotated pyproject, docs-coverage CI gate — clear strengths. Deductions: no in-repo CHANGELOG (only a releases link), version 0.4.4 is Beta, and maintenance responsibility rests on an unverified publisher.
explain(), campaign watching, and a CLI give usable output surfaces. Deductions: the core value claims (self-improvement, benchmark firsts) are unverifiable from the provided files; cost/benefit (multi-model subscriptions, hours-long ingests) is only prose, with no quantified evidence.
README separates claims from explanation and links a technical report and benchmark pages. Deductions: benchmark results (MLE-Bench #1, IOAI superhuman, RelBench vs KumoRFM) cannot be cross-verified from these files; the arXiv ID 2601.21526 is future-dated and unverifiable; missing core code makes claims untraceable.
- requirements.txt contradicts pyproject.toml on dependencies; verify the actual dependency source before installing.
- Benchmark and IOAI claims cannot be verified from in-repo files; do not take them at face value.
- Usage requires multiple API keys and third-party CLIs; mind key management and vendor lock-in.
- Publisher identity is unverified, and 'kapso' on PyPI is an unrelated package — install only as leeroo-kapso.
What does this agent do, and when should you use it?
Kapso is an open-source Python framework by Leeroo (published on PyPI as leeroo-kapso, MIT licensed) described as a self-improving software factory for measurable objectives. You state an objective in a sentence and it runs a campaign: designing candidate solutions, having coding agents implement them, scoring each against the objective, and refining the closest until the objective is met, with the result deployed to your infrastructure. It is built on four pillars: evolve (tree-search experiments), learn/learn_knowledge (mining finished campaign trajectories into evidence-priced knowledge cards, plus importing repositories and papers into a Knowledge Graph), research (deep web research), and deploy (local, Docker, or Modal). Inference runs through coding-agent CLIs — Claude Code for ideation and implementation by default, Codex for research and judging — rather than direct model API calls. The project reports #1 rankings among open-source systems on MLE-Bench and ALE-Bench, results above KumoRFM-v2 on RelBench, and a Grand Master Trophy at IOAI 2026's AI track with a total score above all 471 human contestants.
The end-to-end flow: (1) kapso.evolve(goal=..., initial_repo=..., output_path=..., time_budget_minutes=...) launches a campaign that uses tree search to generate candidates, coding agents to implement them, and measurable scoring to refine the best; watch it live with kapso watch ./campaign, and when a session needs something only you can provide it pauses and resumes via kapso inbox reply. (2) kapso.learn(solution) mines the trajectory and banks evidence-priced knowledge cards into a local git-repo lesson bank (shareable via kapso bank connect or kapso bank create). (3) kapso.research(query, mode=["idea","implementation"]) runs deep web research returning structured findings. (4) kapso.learn_knowledge(Source.Repo(...), ...) ingests repositories and findings into a Knowledge Graph backed by local Weaviate + Neo4j (started via scripts/start_infra.sh), which later campaigns consult automatically. (5) kapso.deploy(solution, strategy=DeployStrategy.LOCAL) turns a solution into running software on local, Docker, or Modal targets. The CLI includes kapso doctor (per-verb checks plus model probing), kapso watch, kapso inbox, and kapso bank; model selection is centralized in one editable YAML config.
- An ML engineer who needs end-to-end delivery of a prediction model — data prep, features, training, validation — evolved toward a measurable target such as accuracy > 0.80.
- A performance engineer optimizing CUDA kernels or PyTorch wall-clock time and memory, using the bundled cuda_optimization and pytorch_optimization examples.
- A research or algorithms team iterating contest heuristics over hours-long searches on AtCoder-style optimization problems (the ALE-Bench scenario).
- A data science team running outcome prediction, forecasting, and recommendation over enterprise multi-table databases (RelBench covers SAP, Amazon, H&M data).
- A platform team improving agents with agents: evolving workflows, tools, and prompts until the metric climbs (agent_optimization example).
- An enterprise that wants every completed task to feed a living knowledge bank of internal systems and hard-won lessons, shared across teams via the git-backed lesson bank.
What are this agent's strengths and limitations?
- Verifiable benchmark track record: the README claims #1 open-source on MLE-Bench and ALE-Bench, RelBench results published on the official leaderboard, and an IOAI 2026 Grand Master Trophy (top 3 among AI systems) with a 536.07 total above all 471 human contestants.
- A genuine closed-loop self-improvement mechanism: learn() distills trajectories into evidence-priced knowledge cards, and a lesson stays trusted only while it keeps holding up, guarding against experience decay.
- Pre-flight safety: kapso doctor runs static checks and an optional live model probe, so missing CLIs or credentials fail in seconds instead of hours into a run.
- A clear deployment boundary: solutions ship via .deploy() to local, Docker, or Modal, and solution.explain() documents what was produced.
- Hard dependency on two third-party coding-agent CLIs with logged-in subscriptions (Claude Code and Codex); there is no direct-API fallback, so without either subscription it will not run.
- Model names and crew timeout caps were calibrated on the default models (claude-opus-5, claude-fable-5, gpt-5.6-sol); swapping models requires config edits and possibly raised timeouts.
- learn_knowledge() ingest is hours-long and scales with extractable substance, not the research depth flag — a small findings set can still yield dozens of wiki pages.
- An unrelated PyPI package named kapso (a WhatsApp tool) shadows the kapso command, creating a real mis-installation risk.
How do you install or deploy this agent?
1) Install Node.js 18+ and log in to both coding-agent CLIs: npm install -g @openai/codex && codex login; npm install -g @anthropic-ai/claude-code && claude auth login. 2) Add OPENAI_API_KEY to .env (used for embeddings). 3) On Python 3.10+, run pip install leeroo-kapso (if the unrelated same-named WhatsApp tool is present, run pip uninstall kapso first). 4) Verify with kapso doctor; optionally bash scripts/start_infra.sh to start the Weaviate+Neo4j KG backends; optionally connect Leeroopedia MCP with pip install leeroopedia-mcp and LEEROOPEDIA_API_KEY in .env. For development: git clone https://github.com/leeroo-ai/kapso.git then pip install -e .
How do you use this agent?
Minimal loop: from kapso import Kapso; kapso = Kapso(). Then solution = kapso.evolve(goal="Optimize the model in train.py; target accuracy > 0.80 on evaluate.py", initial_repo="./my_project", output_path="./campaign", time_budget_minutes=120). Watch with kapso watch ./campaign from another terminal; reply to pauses with kapso inbox reply. Then lesson = kapso.learn(solution); set learning.serving.enabled: true in config and run the next evolve() to serve earned cards. Deploy: from kapso import DeployStrategy; deployed = kapso.deploy(solution, strategy=DeployStrategy.LOCAL); result = deployed.run({"input": "data"}); deployed.stop(). To change models, copy the packaged config to kapso-config.yaml, edit, and pass Kapso(config_path=...); before long runs probe with kapso doctor --models or enable preflight.live_model_probe: true in config.
How does this agent compare with similar options?
The README's MLE-Bench chart compares Kapso against R&D-Agent, AIRA-dojo, ML-Master, and AIDE; on RelBench it is compared against NVIDIA's KumoRFM-v2. If you need a single-shot ML-engineering run rather than a continuously self-improving experimental factory, those systems or bare coding-agent CLIs are lighter-weight alternatives.