Pi Agent Harness

Evidence shows: README explicitly states Pi has no built-in permission system, runs with the permissions of the launching user, and recommends containerization or sandboxing. This is an honest disclosure of permission boundaries, but no least-privilege mechanism or user confirmation flow is provided. Data flow transparency: README mentions telemetry packages but does not detail data flows. Sensitive data handling is not mentioned. Dependency security: README describes supply-chain hardening measures in detail, such as exact version pinning, npm audit, shrinkwrap, etc., with strong evidence. External effects: no mention of external side effects or network behavior. Rollback: no rollback mechanism mentioned. Source attribution: README and LICENSE provide author information, but publisher is unverified. Therefore, except for dependency security, most criteria score low.

Evidence shows: Test files (agent-loop.test.ts, agent.test.ts, e2e.test.ts) demonstrate rich unit and integration tests covering agent loop, tool calls, error handling, etc., indicating good internal consistency. Dependency availability: README mentions npm and Bun, but no detailed offline installation instructions, only build:offline. Failure messages: tests verify error message generation, e.g., 'provider exploded', indicating clear error handling. But no detailed documentation of runtime failures.

Evidence shows: README describes multiple use cases, such as coding agent CLI, unified LLM API, TUI, etc., and provides containerization patterns (Gondolin, Docker, OpenShell), indicating consideration for various environments. Capability boundaries: README explicitly states no built-in permission system and recommends sandboxing, defining boundaries. Trigger precision: no detailed description of tool call trigger conditions, but tests show precise handling of tool calls. Environment fit: supports multiple platforms (darwin, linux, windows) and multiple LLM providers, indicating good adaptability.

Evidence shows: README provides clear information architecture, listing all packages and descriptions. Install notes are detailed, including npm install, build, test, etc. Naming stability: package names use @earendil-works/ prefix, but no naming convention documentation. Examples and FAQ: README provides usage examples and links to docs, but no FAQ. Known limitations: README and SECURITY.md clearly state security boundaries and limitations. License is MIT, clear. Versioning: package.json has version number, but no CHANGELOG. Maintenance responsibility: README provides contribution guidelines and contact info, indicating maintainers.

Evidence shows: Output usability: agent produces structured messages and events, tests verify output format. Marginal value: provides unified LLM API and agent framework, potentially reducing integration costs. Cost-benefit: no performance or cost data, but open-source MIT license reduces cost.

Evidence shows: Claims in README (e.g., supply-chain hardening) are supported by specific implementations (e.g., check scripts), but no independent verification. Cross-source corroboration: test files provide internal validation, but no cross-validation with external sources. Fact-inference separation: README distinguishes facts (e.g., feature descriptions) from inferences (e.g., recommendations), but not explicitly labeled.