MetaHarness
Turn a repository into a branded, governed, publishable AI agent harness.
Per-dimension scores and reasoning
The evidence shows a strong secure-by-default posture: MCP denies network, shell, and writes by default; dangerous actions require approval and are bounded by timeouts, call budgets, and auditing; CI uses read-only permissions; publishing uses short-lived WIF credentials; API keys are hashed, scoped, expiry-checked, and protected against enumeration. Publishing/IPFS operations have confirmation gates, while Darwin/AVO describe isolation, rollback, and signed receipts. Deductions apply because the supplied files do not map every end-to-end data path for remote MCP, model providers, memory, and telemetry; rollback is described in several subsystems but not as one product-wide recovery procedure; and Cargo metadata still attributes the project to an older repository name.
The CI definition covers Rust, WASM, Node, multiple operating systems, package-install smoke tests, and native lifecycle checks. Tests directly cover authentication, reserve-and-commit budgeting, failover, actionable status codes, and non-disclosing errors. Deductions reflect internal README count conflicts, such as 21 versus 17 subcommands and 19 versus 18 example packages, plus tension between statements that some packages are private experiments and that many packages are published. Installation also falls back from npm ci to npm install, and runtime availability can depend on registries, model providers, and external services.
Nineteen vertical templates, ten host targets, browser/CLI/local-repository entry points, replaceable models, removable components, and configurable governance address a broad set of audiences and environments. Experimental features, host limitations, deny-list behavior, storage adapters, and authentication prerequisites establish unusually clear boundaries. Trigger precision is deducted because the supplied implementation excerpts do not fully expose the selection rules for template recommendations, automatic routing, upgrade overlays, or evolutionary changes; some behavior remains documented rather than inspectable here.
The README has strong navigation, architecture and command tables, host/template matrices, ADR pointers, and quick-start examples. It states the Node 20 requirement, npx installation path, wizard, diagnostics, security limitations, and beta status. MIT metadata and the full license text agree. Deductions apply for substantial naming drift among MetaHarness, metaharness, agent-harness-generator, create-agent-harness, and the obsolete Cargo repository URL; no CHANGELOG content is included beyond version and release-process summaries; and although a maintainer email and security-response targets are provided, broader governance and succession are not shown and publisher identity remains unverified rather than established.
The output is concretely described as a publishable ZIP with a branded npx CLI, host configuration, MCP, memory, and governance artifacts, supported by practical doctor, validate, compare, audit, SBOM, and signing commands. It offers clear marginal value over manually assembling numerous host adapters. Cost-benefit is deducted because cost estimation, budget gates, and cheap-model routing are present, but strong claims such as frontier quality at one-tenth the cost and completion within 60 seconds are not corroborated by included benchmark results and methodology; some advanced paths also require GPUs, external models, or hosted services.
Many claims point to named ADRs, workflows, packages, and tests, while supplied tests directly exercise authentication, budgeting, error redaction, routing, and failover. The README carefully labels experimental findings, non-eligible smoke results, deferred claims, and unmet gates, providing strong fact/inference separation. Cross-source corroboration is deducted because the referenced ADR bodies, lockfiles, publishing/security workflows, actual CI records, benchmark reports, and signed artifacts are absent; test counts, green status, SLSA level, performance, and publication claims therefore remain only partially checkable from these excerpts.
- This is a low-confidence static review; no code, tests, installation, release, or signature verification was executed.
- Before adoption, inspect lockfiles, complete security and publishing workflows, actual CI runs, npm artifacts, and witness signatures, and determine whether the stale Cargo repository URL is only legacy metadata.
- Remote MCP, model-provider, memory-storage, and experimental AVO/ARC/field-memory features expand data and execution boundaries; review network, secret, storage, sandbox, and confirmation policies individually before deployment.
- Do not treat README claims about test counts, performance, cost, or SLSA as independently verified; the corresponding run records and artifacts were not supplied.
What does this agent do, and when should you use it?
MetaHarness is a repository-oriented harness generator rather than a fixed agent framework. Its browser Studio and `npx metaharness` CLI analyze a GitHub repository or start from a blank slate, then produce an npm-publishable package with its own branded CLI, project-scoped memory, skills, commands, governance policy, and optional MCP server. Generated harnesses share `@metaharness/kernel`, a Rust kernel built for WASM and NAPI-RS. The documented host set covers Claude Code, Codex, pi.dev, Hermes, OpenClaw, RVM, GitHub Copilot, OpenCode, Prime Agent, and GitHub Actions, using MCP where supported and native extension or skill formats elsewhere. Security and release features include default-deny tool policy, static MCP scanning, audit limits, Ed25519 witness signing, SPDX SBOM generation, and CI-backed release gates. It is a strong fit for teams that want repository knowledge and operational policy packaged as a versioned product, although several advanced capabilities remain experimental or gated on future evaluations.
A typical flow starts with npx metaharness --wizard, npx metaharness my-bot --template vertical:coding --host claude-code, or harness analyze-repo . --scaffold my-bot for an existing checkout. Repository analysis is deterministic and static: it does not execute repository code, and inferred build or test commands are emitted as trust: inferred · execution: disabled. The generator creates an npm-ready harness containing package.json, an executable CLI, recommended agents, skills, slash commands, project-scoped memory, governance configuration, and host-specific files. MCP can be disabled, run locally over stdio, or exposed remotely over authenticated HTTPS; the scaffold emits src/mcp/{server,tools,resources,prompts,policy,audit}.ts and .harness/mcp-policy.json. The generated harness CLI supports operations including doctor, validate, diag, score, genome, mcp-scan, threat-model, compare, audit, sbom, upgrade, sign, verify, and publish --confirm. Optional Darwin Mode can inspect and mutate harness configuration, execute tests in a sandbox, evaluate changes, and retain only measured improvements, while the optional router selects the cheapest model predicted to meet a quality threshold from evaluation logs.
- A platform engineering team wants to package an internal repository's conventions, memory namespace, permissions, and release checks as a shared organization-level
npxpackage. - An open-source maintainer needs a repo-aware maintenance harness but does not want an unfamiliar repository executed during the analysis stage.
- A team using several coding-agent hosts wants one harness definition that can emit integrations for Claude Code, Codex, OpenCode, Copilot, or GitHub Actions.
- A security reviewer needs a static report on MCP shell and network grants, audit settings, timeouts, wildcard permissions, secret handling, and dependency pinning, with high-severity findings producing a failing exit code.
- An engineering organization wants to reduce model spend using
@metaharness/routeror a cheap-to-frontier cascade trained or evaluated on its own workload. - A release team needs Ed25519-signed witness manifests, an SPDX-2.3 SBOM, and CI gates for tampering and software-supply-chain checks.
What are this agent's strengths and limitations?
- The result is an owned, renameable npm package with its own CLI and branding, rather than configuration that only runs inside the generator.
- One generation model targets ten named hosts, including native output forms for pi.dev and Prime Agent where MCP is not used.
- Repository inspection does not execute target code, inferred commands remain disabled, and MCP starts with network, shell, and file writes denied.
- The release design includes Ed25519 witness signatures, tamper tests, SPDX-2.3 SBOM output, npm provenance, CodeQL, and a multi-platform CI matrix.
- It offers browser-only generation, a local CLI, host-specific npm wrappers, and a shared Rust/WASM/NAPI-RS kernel.
- The project identifies itself as a v0.1.x beta and says credibility and documentation reconciliation work is still in progress.
- The stronger AVO capability claim remains blocked on a preregistered 100-task SWE-bench gate; the ARC-AGI-3 packages are private and experimental with no official performance claim.
- Host capabilities are uneven: Codex has no hooks, pi.dev does not use MCP, and Prime Agent cannot enforce a deny-list by itself and may require a separate sandbox.
- Field-memory deployment requires an absolute storage path, a compatible adapter, an authentication-backed principal verifier, and a persistent deployment-secret key of at least 32 bytes; multi-process use also requires distributed writer configuration.
- Developing the complete repository spans Node.js, Rust, WASM, and NAPI-RS, creating more maintenance surface than a single-language agent template.
- Some published benchmark economics are estimates or deliberately limited evidence: SWE-bench Verified per-instance cost was not captured directly, and the 100-problem LiveCodeBench result is described as directional.
How do you install or deploy this agent?
The CLI path requires Node.js and npm. Run npx metaharness --wizard without a global installation, or clone the source and run npm install. The browser alternative is https://ruvnet.github.io/metaharness/; the Studio runs client-side and requires no MetaHarness account, hosted backend, or API key. Source development additionally uses the Rust toolchain and documents verification with cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings, npm run build:wasm, npm test, and node scripts/healthcheck.mjs.
How do you use this agent?
For a first scaffold, run npx metaharness my-bot --template vertical:coding --host claude-code, followed by cd my-bot && npx . --help. For an existing checkout, use harness analyze-repo . to inspect deterministic recommendations or harness analyze-repo . --scaffold my-bot to materialize them. In the generated project, run harness doctor for a smoke check and harness validate for all release gates; use harness mcp-scan <path> to inspect an MCP configuration. Select integrations with --host, disable the scaffolded Darwin feature with --no-darwin, or opt into experimental field memory with --field-memory. Credentials for remote MCP, publishing, IPFS, or external model providers depend on the chosen deployment, while the local Studio and static repository analysis do not themselves require an API key.
How does this agent compare with similar options?
Unlike a conventional agent framework, which supplies a fixed environment in which developers build agents, MetaHarness is positioned as a factory that lets repositories publish their own harnesses. Its Prime Agent adapter also illustrates a concrete runtime tradeoff: Prime Agent has no MCP, so MetaHarness emits Python-backed skills under .prime/agent/skills/ and produces SANDBOX-REQUIRED.md when a deny-list cannot be enforced. Compared with sending every task to a frontier model, @metaharness/router and the documented cheap-to-frontier cascade aim to select the least expensive model that can clear a quality threshold; the repository also discloses estimation and sample-size caveats for its benchmark results.
FAQ
Will it execute an unfamiliar repository while analyzing it?
analyze-repo and genome are described as deterministic static analysis, and inferred build or test commands are marked as disabled.Does it require a hosted service or an OpenAI API key?
OPENAI_API_KEY; external model providers or remote MCP deployments may have their own authentication requirements.Is MCP mandatory?
off, local stdio, and authenticated remote HTTPS modes, and the pi.dev and Prime Agent integrations use non-MCP mechanisms.