Kitaru

What does this agent do, and when should you use it?

Kitaru is an Apache-2.0, self-hosted replay-evaluation system for AI agents from the team behind ZenML, built on the premise that a trace should be runnable, not merely readable. It records or imports production runs — Langfuse, LangSmith, Braintrust, Logfire, Arize Phoenix, MLflow, or a custom importer — as sessions, then replays them by actually re-executing your agent code while tool calls are answered from the recording, so side effects like refunds are not repeated. Evaluation starts from human judgment: a coding assistant reviews the sessions that matter, interviews you against the evidence, pins your answers to exact trace locations, drafts evaluators, freezes cohorts, and runs experiments on your change before release. A single server (FastAPI + Postgres, with Docker image and Helm chart) never executes user code; replays, imports and evaluations run on workers inside your own environment and credentials. Python and TypeScript SDKs share one server, with adapters for PydanticAI, LangGraph, OpenAI Agents SDK, Claude Agent SDK, Mastra and Vercel AI SDK.

Kitaru turns each agent run into a session, either by wrapping the agent once with a record adapter in production or by importing existing traces from Langfuse, LangSmith, Braintrust, Logfire, Arize Phoenix, MLflow, or a documented custom importer. On replay, the real agent code runs again on a local worker while tool calls are answered from the recording: an unchanged replay yields a faithful baseline, and a forked replay isolates the effect of one change. Built-in deterministic evaluators sweep sessions first; a coding assistant then discusses the ones that matter with you, pins your judgments to exact trace locations, and drafts an evaluator calibrated by them. Cohorts freeze the population and experiments replay that cohort against your change, reporting what improved and what regressed. Operation runs through kitaru login (managed cloud or --local, which provisions a FastAPI + Postgres server in Docker or Podman) and kitaru setup (installs skills into ~/.agents/skills and registers kitaru-mcp with Claude Code, Codex, Cursor and Windsurf); other MCP clients connect via a small JSON block.

1. Teams already shipping PydanticAI, LangGraph, or OpenAI Agents SDK agents who want to regression-test a model or prompt change against real historical traffic.
2. Teams storing traces in Langfuse, LangSmith, Braintrust, Logfire, Arize Phoenix or MLflow that want those read-only records converted into executable replay cases.
3. Engineering teams building a regression suite that replays production traffic against a candidate change and gates it in CI.
4. Developers who want Claude Code, Codex, Cursor or Windsurf to drive the investigation loop instead of hand-writing evaluation scripts.
5. Maintainers of agents with external side effects, such as ticket refunds, who need tool calls served from the recording rather than executed again.
6. Organizations with data-residency or compliance constraints that require self-hosting with traces never leaving their systems.

How do you install or deploy this agent?

From a terminal in your agent's repository, run the one-line installer; it adds Kitaru to that project's environment with uv, installs the coding-agent skills, and registers the MCP server with Claude Code, Codex, Cursor and Windsurf:

curl -fsSL https://kitaru.ai/install | bash

If the repository is already open in Claude Code, Codex or Cursor, paste this instead:

Set up Kitaru in this repository by following https://kitaru.ai/install.md. Use the one-line installer and tell me what it did.

To do it by hand, add the package to the project environment, then pick a server:

uv add "kitaru[cli,worker,mcp]" kitaru-pydantic-ai
uv run kitaru login          # managed cloud; 14-day trial, no credit card required
uv run kitaru login --local  # local server in Docker or Podman
uv run kitaru login <your-team-url>

Re-run setup for any editor installed later:

uv run kitaru setup            # or: kitaru setup --mode read-only, --no-skills, --no-mcp

Other MCP clients can paste this configuration:

{
  "mcpServers": {
    "kitaru": {
      "command": "uv",
      "args": ["run", "kitaru-mcp", "--server", "http://localhost:8000", "--mode", "standard"]
    }
  }
}

How do you use this agent?

Start with the bundled example agent, which ships with checked-in Langfuse traces:

git clone https://github.com/zenml-io/kitaru.git
cd kitaru/examples/python/pydantic_ai_ticket_resolver

Open that directory in Claude Code, Codex or Cursor and ask for the guided tour:

Use kitaru-guided-tour to walk me through Kitaru on the PydanticAI returns agent example. I am new; explain each step as we go, and ask before anything paid or live.

For your own agent, run the investigation skill:

Use kitaru-investigation to investigate this agent and help me test one meaningful improvement. Show me the recorded evidence before asking for a judgment, and ask before creating resources or starting paid replay.

The assistant sweeps sessions with built-in deterministic evaluators, interviews you on the ones that matter, drafts the evaluator, runs the experiment, and reports what improved and what regressed; you make the judgment calls. For an existing production agent, record with an adapter or import traces from Langfuse, LangSmith, Braintrust, Logfire, Arize Phoenix or MLflow.

What are this agent's strengths and limitations?

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

How does FollowAgents rate this agent?

FAQ