TypeSafe Evaluate

What does this agent do, and when should you use it?

TypeSafe Evaluate is an MCP server distributed as a static Go binary and exposing one read-only tool named evaluate. It accepts text or JSON state plus noul, choice, or score questions, then uses TypeSafe's Jev model to produce structured answers with probabilities. Requests can go directly to the TypeSafe API or through OpenRouter's Decisions endpoint, but both routes depend on Jev rather than providing a general model adapter. evaluate setup mcp registers the server with Claude Code, Claude Desktop, and Codex when detected, while pi connects through a separately installed native extension. It fits programmable routing and classification workflows that need machine-readable confidence, but it is not a general text generator and cannot perform evaluations without network access and suitable API credentials.

The evaluate MCP tool reads a state value containing plain text or a JSON object or array with named fields, followed by a questions map keyed by caller-defined question IDs. Each question uses the noul, choice, or score type and may include instructions and the applicable criteria. The server sends this material to the TypeSafe API at POST /v1/systemone, or to POST /decisions when configured for OpenRouter, and returns the raw structured JSON with answers under the original IDs. Multiple independent questions about the same state can be submitted in one call and run in parallel. HTTP 429 and 529 responses are retried with exponential backoff, while other API failures become tool errors the agent can inspect. Malformed criteria are rejected locally with the submitted field path; requests have a 60-second timeout, and responses larger than 16 MiB are rejected rather than truncated.

1. A support-platform developer needs to judge whether a ticket is urgent and route it among billing, technical, and sales teams from the ticket text.
2. An agent-workflow author needs a probabilistic yes/no judgment so application code can choose a branch based on confidence.
3. A backend engineer wants to evaluate several independent questions against the same event state and receive all answers in one structured call.
4. A Claude Code, Claude Desktop, or Codex user wants to avoid maintaining rigid prose prompts and fragile output parsers for recurring decisions.
5. A pi user wants to add typed judgments to an automation through the supplied native extension and its evaluate mcp backend.

How do you install or deploy this agent?

On macOS or Linux running amd64 or arm64, use the installer; it places the binary in ~/.local/bin:

curl -fsSL https://raw.githubusercontent.com/itsmostafa/typesafe-mcp/main/install.sh | sh

If that directory is not on PATH, add it:

export PATH="$HOME/.local/bin:$PATH"

Go users can install it directly instead:

go install github.com/itsmostafa/typesafe-mcp/cmd/evaluate@latest

After obtaining a TypeSafe API key, register the MCP server with detected Claude Code, Claude Desktop, and Codex installations:

TYPESAFE_API_KEY=your-key evaluate setup mcp

To access Jev through OpenRouter, use:

OPENROUTER_API_KEY=your-key evaluate setup mcp

TYPESAFE_API_KEY takes precedence when both variables are set. Because pi has no MCP client, install the supplied extension separately:

evaluate setup pi

This writes ~/.pi/agent/extensions/evaluate.ts; run /reload in pi afterward. Upgrade an installed binary in place with:

evaluate update

How do you use this agent?

Complete client registration and make a valid TYPESAFE_API_KEY or OPENROUTER_API_KEY available to the shell that launches the agent. Ask the agent to use evaluate for a specific judgment, such as deciding whether a ticket is urgent and which team should own it. The corresponding tool input can be:

{
  "state": "Help! My payouts have been failing for 3 days.",
  "questions": {
    "is_urgent": {
      "type": "noul",
      "instructions": "Does this convey urgency?"
    },
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "billing": "Payments, refunds",
        "technical": "Bugs, outages",
        "sales": "Pricing"
      }
    }
  }
}

The result is raw JSON, with each answer stored under the question ID supplied in the request. An optional model field is supported; it defaults to jev-latest on the TypeSafe route and ~typesafe/jev-latest on OpenRouter.

What are this agent's strengths and limitations?

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

How does FollowAgents rate this agent?

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

1. v0.4.14 2026-11-03
   Release notes excerpt

   [0.4.14](https://github.com/itsmostafa/system-one-connector/compare/v0.4.13...v0.4.14) (2026-10-25) Bug Fixes * eval: make the taxcat eval neutral and score it only on its own dataset ([12212ee](https://github.com/itsmostafa/system-one-connector/commit/12212eefc24a002f5adf34f461b14b8ba2877263)) * eval: show recall per taxcat group when limits differ ([2b0f555](https://github.com/itsmostafa/system-one-connector/commit/2b0f555ececbed1bed2b2ada41586867010b78c2))

   View on GitHub ↗
2. v0.4.13 2026-10-23
   Release notes excerpt

   [0.4.13](https://github.com/itsmostafa/system-one-connector/compare/v0.4.12...v0.4.13) (2026-10-23) Bug Fixes * client: keep /v1/decisions base URLs as given ([39dcd64](https://github.com/itsmostafa/system-one-connector/commit/39dcd64a8b8a7e31a4ba53a7351fb5ef8821b368)) * client: keep /v1/decisions base URLs as given ([702423f](https://github.com/itsmostafa/system-one-connector/commit/702423f1c6111301c05c65c4d2ade05dbd5d95de)) …

   View on GitHub ↗
3. v0.4.12 2026-10-09
   Release notes excerpt

   [0.4.12](https://github.com/itsmostafa/system-one-connector/compare/v0.4.11...v0.4.12) (2026-10-08) Features * client: send extra request headers from TYPESAFE_HEADERS ([e2fcc02](https://github.com/itsmostafa/system-one-connector/commit/e2fcc02c39015daf7ef9cd6854daa5fd10edb481)) * client: send extra request headers from TYPESAFE_HEADERS ([adca31b](https://github.com/itsmostafa/system-one-connector/commit/adca31b71e48d90ac0d08b05df0ae4c752560591)), closes [#53](https://github.com/itsmostafa/system-one-connector/issues/53)

   View on GitHub ↗

FAQ