Codex Autoresearch
Run measurable optimization loops inside your codebase: give it a goal, a benchmark, and a boundary, and it experiments, keeps evidence, and leaves a reviewable patch.
- Source repo
- TheGreenCedar/codex-autoresearch
- Stars
- ★ 835
- Last updated
- 20d ago
- License
- Apache-2.0
- Primary language
- TypeScript
- FA score
- 91/100 · Excellent
At a glance
- How it runs
- Works with
- Platform-specificCodex
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- A backend engineer shortening a slow test suite that already has a repeatable benchmark script and tests
- Not a fit if
- Teams that need to run outside the Codex ecosystem or in another agent runtime
- One-off edits or taste-based work with no repeatable measurement
- Benchmarks so slow or noisy that each measurement adds little information
- Source review
- 91/100 · Excellent
What does this agent do, and when should you use it?
Codex Autoresearch is a Codex plugin that runs bounded optimization loops against a repeatable benchmark in a local codebase. You supply a workload, correctness checks, an edit boundary, and a time budget; it records a baseline, evaluates small candidate changes one at a time, and delivers a reviewable patch with the evidence behind it. It targets test runtime, build speed, bundle size, model loss, memory use, query latency, or any metric a script can print. Session records live in the target project: temporary packet state sits under .git/autoresearch/ in Git repositories, falling back to local worktree files otherwise. An optional read-only dashboard shows improvements, passes, blockers, and the next action, while commands still run through Codex and the CLI. Version 3.0 shipped on engineering verification alone, with no model-driven comparison and no claim of better outcomes or lower cost than ordinary Codex or 2.9.0.
After installing, you issue a contract via /goal: goal, benchmark command, metric, check commands, scope, and budget; Codex presents the full contract for approval first. Once accepted, the flow is setup -> accept contract -> state -> next -> log -> state -> finalize-preview. Setup prepares an experiment contract covering goal, repository and checkout, typed metric semantics, evaluator, independent checks, scope, noise, keep and stop rules, and budgets. State compiles current files and Git state into one decision; next runs only the evaluator and checks the contract accepts; log records whether a result was a baseline, keep, discard, or failure; finalize-preview returns a read-only evidence receipt of accepted changes, exclusions, and blockers. The benchmark must print at least one line like METRIC seconds=12.34; the primary metric decides direction, checks protect correctness, and secondary metrics catch known tradeoffs. Keeping a result can create a commit limited to configured paths; a plain measurement never stages, commits, or reverts anything.
- A backend engineer shortening a slow test suite that already has a repeatable benchmark script and tests
- A frontend developer reducing build time or bundle size while keeping functionality correct
- An ML researcher lowering model loss on a fixed dataset through many small candidate changes
- A performance engineer tuning query latency or memory use, guarding against tradeoffs like faster runtime at much higher memory
- A team with a vague outcome (e.g., a faster, less memory-hungry indexer) asking the plugin to help design a trustworthy benchmark first
- A maintainer preferring several small, bounded experiments over one large rewrite
How do you install or deploy this agent?
Open the plugin picker in Codex: type /plugins, choose TheGreenCedar -> codex-autoresearch -> Install plugin, then start a new Codex task in the repository you want to improve. If your Codex build supports terminal marketplace management, register the source marketplace first: codex plugin marketplace add TheGreenCedar/AgentPluginMarketplace --ref main. This repository is the plugin source; the marketplace lives in TheGreenCedar/AgentPluginMarketplace. Source development requires Node.js 24+, npm, and Git.
How do you use this agent?
In the target repository, issue a contract via /goal @Codex Autoresearch, e.g.: run a measured loop to reduce parser runtime; Benchmark: node bench/parser.mjs; Metric: seconds (s), lower is better; Checks: node --test tests/parser.test.mjs; Scope: src/parser.mjs; protect bench/parser.mjs and tests/parser.test.mjs; measure two baselines and repeat each candidate before a keep; stop after 5 packets or 30 minutes. Codex presents the complete contract for review; after acceptance, Autoresearch measures the baseline before any candidate. If you lack a benchmark, ask Codex to inspect relevant files and propose commands and paths with citations, as a direct review first — no session is created until you approve the complete contract. Tasks like architecture reviews and documentation continue directly through ordinary Codex without session files.
What are this agent's strengths and limitations?
- Every kept change ships with a reviewable evidence receipt; baselines, keeps, discards, and failures are all logged
- The contract mechanism forces explicit goal, metric semantics, scope, noise, keep/stop rules, and budgets before execution
- Secondary metrics catch known tradeoffs such as lower runtime at much higher memory use
- State, doctor, recommendations, finalization, and the dashboard share one snapshot with retries on concurrent source changes
- Inspired by karpathy/autoresearch and pi-autoresearch, but built around Codex, local repositories, and ordinary reviewable Git work
- Tightly coupled to the Codex ecosystem: installed via /plugins and run inside a Codex session, not portable to other agent runtimes
- Only fits tasks with repeatable measurement, a reasonably stable benchmark, and a nameable edit boundary; one-off edits and taste-based work are a poor fit
- Approved benchmark commands run with your local permissions and can read files, start processes, use credentials, contact paid external services
- 3.0 has no model-driven comparison data backing claims of better outcomes or lower cost than ordinary Codex
- Benchmarks that are slow or noisy add little information per measurement, limiting loop value
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Codex Autoresearch This agent | 91 · Excellent | Agent plugin / skillFree + model costs | ★ 835 | 20d ago | TypeScript | Codex |
| FableCodex | 82 · Good | Agent plugin / skillFree + model costs | ★ 437 | 2mo ago | Python | Codex |
| CORAL Autonomous Research Infrastructure | 60 · Some gaps | CLIFree + model costs | ★ 1.1k | 1mo ago | Python | Codex · Claude Code |
| Compound Engineering | 57 · Major gaps | Agent plugin / skill | ★ 26k | today | TypeScript | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows least-privilege design (packets get minimal env by default, full env inheritance requires --packet-env-mode inherit, SHA-pinned actions with minimal workflow permissions, check runner refuses unlisted commands); contracts require explicit acceptance before runs, finalize starts with a read-only preview, branches only after approval; privacy docs disclose local session records and the separate Codex data path; plain measurements never stage or commit, keeps are limited to configured paths, and rollback-safe directory swaps are proven by phase-fault injection tests; external effects (credential access, network calls, monetary cost) are explicitly disclosed. Deductions: sensitive-data handling is best-effort redaction that the authors themselves call not a security boundary, and CI uses --no-audit with no shown dependency vulnerability scanning or mitigation, so sensitive_data_handling and dependency_security get 2.
README, CI, and tests are highly consistent: attestation verify args, checksum manifest parsing, and per-phase fault recovery for runtime swaps all have targeted behavioral tests; dependencies are lockfile-pinned, actions SHA-pinned, and hydration fails closed when gh is unavailable; tests extensively assert actionable error messages (stale rollback directories, specific npm resolution guidance). No deduction identified.
Fit and misfit scenarios are clearly delineated (repeatable bounded experiments vs one-off or taste-driven work); direct vs measured routing in 3.0 is explicit; triggers are precisely controlled via fit -> contract flow; deduction: environment demands are high (Node 24+, Codex host, Chrome for dashboard tests) and no degraded or minimal path is offered, so environment_fit gets 2.
Documentation is complete (start/walkthrough/operate/finish/troubleshooting/architecture index), install/update/uninstall notes include marketplace-vs-workspace caveats; naming is stable; Apache-2.0 full text with copyright is present; CHANGELOG plus auto-release enforces version synchronization across package./package-lock/plugin. and rejects non-strictly-increasing versions; limitations are honest (synthetic screenshot labeled, no comparative-experiment claim). Deductions: no standalone FAQ (troubleshooting substitutes), and maintenance responsibility rests on one named individual whose publisher identity is unverified, so examples_and_faq and maintenance_responsibility get 2.
Output usability is strong: evidence receipts, reviewable patches, read-only dashboard, and audit tracing are documented and tested. Deductions: the README admits 3.0 has no model-driven comparison and claims no advantage over ordinary Codex or 2.9.0, so marginal value is unproven (marginal_value 2); cost control exists only as in-contract budgets/time caps with disclosed possible external charges and no measured cost data (cost_benefit 2).
Claims are traceable: engineering verification is explicitly separated from the not-yet-run comparison; proposals must cite source files; the repo's own verification chain (attestation pinned to the signer workflow, checksum validation, release provenance policy tests) is unusually complete; README, CI workflows, and tests corroborate each other; synthetic data is labeled and fact/inference separation is clean. No deduction identified.
- Publisher identity is unverified by the curated registry; verify the TheGreenCedar/codex-autoresearch attestation mapping yourself before use.
- Redaction is best-effort, not a security boundary: keep secrets out of benchmark commands, outputs, and notes; dashboard exports and ledgers may contain paths and command excerpts.
- Approved commands run with your local permissions, can access credentials, contact external services, and incur charges; read the trust/privacy/terms docs before use on sensitive repos or expensive workloads.
- Version 3.0 has no comparative-experiment data; its advantage over ordinary Codex is unproven and should be assessed with your own small trials.
- Removing a marketplace registration does not uninstall an already-installed workspace plugin; use the plugin UI.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
Codex Autoresearch 3.0 adds investigations for work where the goal is clear and the method may change. Accept the objective, allowed changes, evidence requirements, and budget once. Preparation, failed attempts, repairs, confirmation, and delivery all use that same allowance. - Define success with numeric targets or explicit conditions and counterexamples. - Resume recorded actions without starting them twice. A packaged worker keeps process actions running when the observing CLI closes. …View on GitHub ↗ -
Release notes excerpt
Measured loops now have a clearer first run, stricter repeat evidence, and a simpler handoff. Existing sessions keep working with their current note paths; new sessions put notes and ideas in .autoresearch/. - The documented first run now starts a fresh measured session. Missing benchmark, checks, and scope can be proposed from bounded read-only discovery before acceptance. - Experiment notes no longer count as product-grade or broad-improvement proof. Goal wording no longer invents retrieval-specific requirements. …View on GitHub ↗ -
Release notes excerpt
2.8.0 - 2026-08-29 The loop now knows when not to loop Autoresearch 2.8.0 makes a hard distinction between work that needs a measured experiment and work that just needs to be done. One-shot architecture, documentation, UX, product, and research tasks stay direct, with a compact evidence trail instead of an empty Autoresearch session. When a loop is the right tool, 2.8.0 is much stricter about what counts. The benchmark, checks, metric, scope, keep rules, stop rules, and budget move together as one accepted contract. A plausible number cannot sneak through as a win. …View on GitHub ↗