Arbor Research Optimizer
A hypothesis-tree agent for running, validating, and retaining metric-driven improvements.
What does this agent do, and when should you use it?
Arbor is an autonomous research runtime for tasks that have a measurable target and an executable evaluation path. Its Coordinator manages an Idea Tree and research decisions, while independent Executors implement one hypothesis at a time in isolated Git worktrees. The Python CLI covers intake, configuration, dashboards, checkpoints, reports, replay, export, and read-only session monitoring, with run artifacts stored under .arbor/sessions/. It can use OpenAI, Anthropic, or OpenAI-compatible providers in its native runtime, or operate inside Codex and Claude Code through skills and optional deterministic MCP tools. It is best suited to benchmark or repository optimization work with a clean Git baseline and dev/held-out evaluation discipline, rather than tasks lacking a defined metric.
The arbor command begins with an intake conversation that turns the objective, target directory, metric, baseline, budget, dev/test rules, and artifact paths into an Arbor Research Contract. The Coordinator executes the Observe, Ideate, Select, Dispatch, Backpropagate, and Decide cycle over the Idea Tree, then dispatches chosen hypotheses to Executors. An Executor changes code and runs the task in its own branch and Git worktree, iterating on the dev signal and reporting evidence; a change is retained only after held-out test validation and the configured threshold are satisfied. arbor replay replays a session timeline, arbor report renders REPORT.md, arbor export produces self-contained HTML or JSONL, and arbor web serves a read-only monitor. arbor idea-check can query the built-in alphaXiv backend for a concise related-work and novelty assessment.
- An ML research engineer with a runnable training benchmark wants to test optimizer or architecture hypotheses under a fixed cycle budget while protecting a held-out test split.
- A developer maintaining an executable coding benchmark needs candidate changes evaluated in separate Git worktrees before any run result is promoted toward main.
- A team improving a retrieval, browsing, or terminal-evaluation harness wants a durable record of failed directions, scores, and distilled lessons rather than an unstructured series of attempts.
- A Codex or Claude Code subscriber without a separate model API key wants the host coding model to run an Arbor-style research loop backed by skills and optional MCP tools.
- A researcher considering a new technical branch wants to run arbor idea-check before allocating compute, using the documented alphaXiv-based related-work check.
What are this agent's strengths and limitations?
- The Coordinator/Executor split is paired with an Idea Tree that preserves results, failure modes, and propagated insights across research cycles.
- Each experiment runs in an isolated Git worktree, develops against the dev signal, and is retained only after held-out validation clears a configurable merge threshold.
- The repository documents three delivery modes: a native CLI, a standalone skill suite, and optional MCP tools, with native support for OpenAI, Anthropic, and OpenAI-compatible backends.
- Sessions leave inspectable, resumable artifacts including REPORT.md, events.jsonl, run_stats.json, the Idea Tree, and per-experiment outputs; HTML and JSONL export are available.
- Native research runs require provider/model configuration and credentials; the keyless route depends on a host coding agent such as Codex or Claude Code for reasoning.
- Adoption presumes a clean Git repository and a runnable evaluation setup, ideally with dev and held-out test data; projects without them must establish that infrastructure first.
- Built-in alphaXiv search requires Python 3.12+; on Python 3.10 or 3.11 the documented backend degrades, and in-run network search is disabled by default.
- Validated changes are merged into a per-run trunk rather than main, so final promotion remains an explicit user-managed Git decision.
How do you install or deploy this agent?
Requirements are Python 3.10+ and Git. Install the native CLI with:
pip install arbor-agent
arbor doctor
Then run arbor quickstart to set up a free key or local Ollama, or use arbor setup to configure provider, model, base_url, and API key. Configuration is written to ~/.arbor/config.yaml. For a keyless demo, run arbor replay --demo. To install the skill suite for Codex or Claude Code, run pip install arbor-agent followed by arbor install. Optional MCP support requires pip install "arbor-agent[mcp]" and registration of arbor mcp with an MCP-capable host.
How do you use this agent?
From a clean Git repository containing a runnable evaluation script and evaluation data, preferably with dev and held-out test splits, run:
arbor "improve validation score without touching the test split" --cwd ./benchmark
Confirm the resolved research contract before experiments launch. For repeatable settings, place task, coordinator.max_cycles, coordinator.max_depth, merge_threshold, and executor.max_turns in research_config.yaml, then run:
arbor --cwd ./benchmark --config research_config.yaml
Resume an interrupted run with arbor --resume --run-name <run_name>. When satisfied, promote its separate trunk with git merge research/run_xxx/trunk.
How does this agent compare with similar options?
The repository reports higher held-out scores for Arbor than its listed Claude Code and Codex baselines on six tasks. For example, it reports 67.67 on BrowseComp versus 53.33 for Claude Code and 50.00 for Codex, and 77.36 on Terminal-Bench 2.0 versus 71.70 and 73.59. These are repository-reported benchmark results, not evidence of a general performance guarantee for every project.