OrcaReplay
Time travel for AI coding agents: record, replay offline, and fork any run from any checkpoint onto different models — byte-for-byte, to find out exactly why it failed.
Deny-by-default env capture, auth dropped before the writer, opt-in ephemeral-CAR TLS interception, loopback-only binds and no telemetry make least privilege strong; deducted one point each for user_confirmation and external_effects because replay executes recorded tool calls for real and side effects are documented as out of scope rather than confirmed. Sensitive-data handling earns full marks: SECURITY.md details redaction in the write path, scrub limitations stated plainly. Dependency security only shows devDependencies and a CI pip install list — no lockfile, runtime dependency manifest or audit evidence, so 1. Source attribution is limited to in-repo brand links with an unverified publisher, so 1.
Tests align tightly with README claims (alias resolution, openCode overlay, Windows path regression) with motivation recorded, full marks for self-consistency. Dependency availability: CI skips missing integration frameworks (now failing loudly) and some integrations are labelled 'should work', so 2. Failure messages are tested to name typeable ids and fixes, full marks.
Audience and scenarios are crisply defined with a quickstart on-ramp; capability boundaries are unusually explicit (not a sandbox, no HTTP/2, LLM-layer-only for browser-use, id_ecdsa gap); adapter detection order is fully tested. Environment fit is Node 20/22 only, with platform tests but limited platform documentation, so 2.
Clear information architecture (i18n READMEs, dedicated capture/spec docs), naming stability enforced by alias and contract tests, rich examples and comparison tables, frank known-limitations sections, complete Apache-2.0 license. Version is 0.2.3 but no CHANGELOG or version history file is in evidence, so 1; maintenance shows CI, a security policy and good-first-issues but no visible maintainer/contributing file, so 2.
Strong output usability (show/graph/export, self-contained HTML, SVG/PNG cards with CI asserting no external refs); marginal value versus observability tools is argued concretely; cost/benefit uses real token/price tables and refuses to invent prices, full marks across the three.
README's 'in CI' claims are actually enforced by ci.yml with a --require-all guard, and validation claims point at named files, so claim traceability is full; the recorded-vs-inferred edge distinction is exemplary fact/inference separation. However this is a static review — 'byte-for-byte' and CI-passing claims cannot be independently reproduced, and cross-source corroboration is confined to in-repo files with an unverified publisher, so 1.
- Replay executes recorded tool calls for real (shell, file writes) and is explicitly not a sandbox; run it in isolation.
- --tls-intercept decrypts child-process TLS; off by default with an ephemeral CA, but still highly invasive — review the host allowlist per run.
- Redaction is best-effort, not a guarantee: id_ecdsa is absent from the filesystem exclusion list; treat traces as sensitive.
- Publisher identity is unverified and the default gateway points at the affiliated OrcaRouter product; assess the data path independently before enterprise adoption.
- This is a static review; byte-for-byte replay and all 'in CI' claims are unverified by execution.
What does this agent do, and when should you use it?
OrcaReplay is an open-source agent recording and replay tool from Continuum-AI-Corp (npm package orcareplay), built by the OrcaRouter.ai team. It records complete runs of coding agents — Claude Code, Codex CLI, LangGraph, OpenHands and more — through a local proxy and five capture layers (base-URL environment variables, a PATH shim, an MCP JSON-RPC tee, a shadow git snapshot index, and a fetch hook), without patching the agent. Traces land as self-describing files under .orca/runs/ in your project, holding the event timeline, filesystem snapshots and an integrity digest. On replay, egress is blocked by default (egress=blocked): no model is called, no tokens are spent, and the run reproduces byte-for-byte; forking from any checkpoint lets a different model take over, graded by a command you choose such as npm test. It also provides a causal graph (orca graph), shareable cards, single-file HTML export, JSON output, and an MCP server (orca mcp) so an agent can read, explain and replay its own runs. Code is Apache-2.0; the trace spec is CC BY 4.0.
orca record <agent> stands up a local proxy and sets two environment variables for the child process (e.g. ANTHROPIC_BASE_URL, OPENAI_BASE_URL), capturing every model request, streamed response, tool call and result at the process and socket boundary; a PATH shim records shell exit codes, timing and stdout/stderr splits, and a shadow git index snapshots the workspace every turn. Events write into .orca/runs/run_<id>/ as manifest., events.l, blobs/ and fs/. orca replay last replays the whole run from disk (egress=blocked); orca replay last --from 4 --model X forks at checkpoint 4 onto a new model; orca compare does this across several models and grades each with --verify. orca show prints the event timeline, orca graph builds a causal graph distinguishing recorded from inferred edges, and orca export produces SVG/PNG/GIF cards or a self-contained HTML file. orca mcp serves six tools over stdio; orca attach records agents in sandboxes or on remote machines; --tls-intercept captures agents that read no base-URL variable at all, such as a Codex CLI signed in with a ChatGPT subscription.
- Debugging an overnight failure: your agent broke something at 2am — replay it at 9am offline and byte-for-byte, then use orca show and orca graph to find which tool call and which shell command (exit 1) deleted the file
- Model selection: fork the same run from the same checkpoint with orca compare --models claude-opus-5,claude-haiku-4-5 --verify "npm test" and grade each model on cost, with identical files and conversation prefix
- CI and scripted verification: every command answers with --, and the programmatic Orca API exposes replay() and show() in code — replay is free and offline
- Letting an agent self-audit: expose orca_replay, orca_graph and other tools via orca mcp so an agent can replay its own last run and report divergences
- Capturing unredirectable agents: a bot with a hardcoded API URL, or a Codex CLI on ChatGPT subscription login, recorded at the socket with orca record exec --tls-intercept
- Recording agents in sandboxes or remote environments: attach with orca attach to a dev container, VPS or CI machine
What are this agent's strengths and limitations?
- No agent modification: capture happens via two env vars and a local proxy, not an SDK wrapper or code changes
- Deterministic offline replay: egress=blocked guarantees no model call, no tokens, no variance — the run reproduces byte-for-byte
- Five capture layers reach what SDK wrappers structurally cannot: shell exit codes, file writes, MCP calls, even agents with no redirectable endpoint via --tls-intercept
- Controlled-variable model comparison: same files, same conversation prefix, only the model changes — the documented example shows both models passing at a 15× cost difference
- Substantial engineering evidence: 1,393 tests, CI integration checks across eight frameworks, and a separately published CC BY 4.0 trace spec with an existing Python reader
- Privacy-by-design: local traces (0600), secrets redacted in the write path, auth headers never written, and orca scrub for post-hoc cleanup with an integrity digest
- The project is early (v0); formats and commands may still change
- Replay is not a sandbox: recorded tool calls execute for real, and tools that open their own sockets (curl, MCP servers) are outside the guarantee by default — sandboxing is your responsibility
- Replaying a hand-driven interactive session is approximate, not byte-exact: harness-issued calls (quota probes, session naming) are skipped, and interactive-only tools like AskUserQuestion are absent in terminal-less replay
- orca scrub cannot rewrite filesystem snapshots (content-addressed git objects); it can only search and report, or you can use --drop-fs and lose the ability to fork the run
- Documented adapter limitations: goose's shell calls lack real exit codes and stdout/stderr splits (use --no-shell), and browser-use coverage is LLM-layer only — the browser is not driven
- Requires Node 20+; on Windows, shell capture depends on a POSIX shell like Git for Windows, otherwise you must record with --no-shell
How do you install or deploy this agent?
npm i -g orcareplay
orca doctorRequires Node 20 or newer, with no native dependencies. From source: git clone https://github.com/Continuum-AI-Corp/OrcaReplay && cd OrcaReplay && npm ci && npm run build && npm install -g ./packages/cli. To try it with no agent, key or network: orca quickstart ships a real recording and the buggy project it was made against, and replays one over the other offline.
How do you use this agent?
Three steps: orca record claude (record your agent unmodified); orca replay last (replay offline — no network, no tokens, no charge); orca replay last --from 4 --model claude-haiku-4-5 --ui (fork at checkpoint 4 onto a new model and open the timeline UI). Compare models: orca compare last --from 4 --models claude-sonnet-5,claude-haiku-4-5 --verify "npm test". Configure a gateway: orca setup (defaults to OrcaRouter; override with --gateway <url> for any OpenAI-compatible gateway). Inspect runs: orca list, orca show last, orca graph last, orca export last -o bug.html. For agents: add {"mcpServers":{"orca":{"command":"orca","args":["mcp"]}}} to your MCP config. Runs live in .orca/runs/ inside your project; traces are local with mode 0600 and API keys never reach a trace.
How does this agent compare with similar options?
The README explicitly contrasts OrcaReplay with observability tools: those tell you a run cost $4.12 and used 61k tokens, while OrcaReplay tells you which tool call deleted the file, reruns the agent with the same answer from the recording byte-for-byte, lets you change the model and re-run from step 4, and works after you close the terminal — with only two env vars instead of an SDK wrapper.