Jev Guard

Least privilege scores 2: assessment is limited to calls crossing host hooks or the ACP proxy and read-only tools are skipped, but installation modifies several user-level agent configurations and the product reads sessions, transcripts, and instruction files. User confirmation scores 2: deny/ask/allow policy, ACP permission requests, and host adaptations are covered by code tests, but hosts such as Codex and Gemini cannot provide native ask behavior and fall back to warnings. Data-flow transparency scores 3: the README identifies transmitted fields, both endpoints, local session/cache paths, credential precedence, and uncovered paths. Sensitive-data handling scores 2: the key file is mode 0600 and is not meant to reach the agent, but arbitrary sensitive tool results may be sent to third parties; requested zero retention is not proof of server-side enforcement. Dependency security scores 2: zero direct runtime dependencies and a Node floor reduce supply-chain exposure, but the optional peer dependency is unpinned and core decisions depend on a remote model/API. External effects score 2: dangerous actions can be blocked or confirmed and installation has idempotency tests; however, failure is open by default and an explicit user request can promote a medium-risk ask to allow. Rollback scores 1: reversibility informs risk classification, but no uninstall command, configuration backup, or recovery procedure is documented. Source attribution scores 2: repository, author, license, Jev, and pricing sources are identified, but publisher identity is unverified and maintenance ownership is represented only by an account.

Self-consistency scores 3: documented thresholds, host differences, context rules, injection classification, and installation behavior align closely with the supplied tests. Dependency availability scores 2: API requests have three attempts, an overall timeout, two backends, and configurable fail-closed behavior; nevertheless, default fail-open removes protection during outages and the service remains externally dependent. Failure messages score 3: deny, ask, injection, instruction-file findings, HTTP failures, and timeouts have explicit messages or exit semantics with corresponding static tests; this does not imply the tests were executed in this review.

Audience and scenarios score 3: eight agent/protocol families, CI checks, skill scanning, GUI hosts, and varied risk cases receive specific coverage. Capability boundaries score 3: the product explicitly says it is a guardrail rather than a sandbox, sees only routed paths, can be wrong, and should coexist with other controls. Trigger precision scores 2: tiered thresholds, user-request and untrusted-source context, discussion exemptions, and extensive tests are present, but results under 200 characters, local edit/search results, and configured skip lists create blind spots, while probabilistic thresholds retain false-positive and false-negative risk. Environment fit scores 3: host-specific response dialects, absolute Node paths, GUI credential loading, ACP operation, and extensive environment tuning are concretely addressed and statically tested.

Information architecture scores 3: installation, decisions, tuning, CLI, development, security, and source layout are clearly organized. Install notes score 3: every host receives commands, behavior differences, credential guidance, and a generic CLI path. Naming stability scores 2: commands and environment variables are systematic, but the project is at version 0.3.1 and package description coverage lags the broader README host list, suggesting continued interface evolution. Examples and FAQ score 3: dangerous-call examples, ACP configuration, CI uses, threshold tables, and calibration cases cover common questions despite no standalone FAQ. Known limitations score 3: fail-open behavior, hook bypasses, ACP visibility, short-result skipping, and non-sandbox status are explicit. License scores 3: MIT package metadata and the full license agree. Versioning and changelog score 1: a package version exists, but no changelog, release history, or upgrade compatibility guidance is supplied. Maintenance responsibility scores 2: author, issue tracker, and private security-reporting route are given, but there is no named maintenance team, response target, or support policy, and publisher identity remains unknown.

Output usability scores 3: decisions are translated into host-consumable deny, ask, warning, flag, exception, and exit-code forms with reasons and probability details. Marginal value scores 3: one cross-host policy combines pre-action risk assessment, result injection detection, session context, and instruction-file scanning, providing a clear addition to native controls. Cost-benefit scores 2: pricing, latency, and caching are quantified and zero direct dependencies simplify deployment; deductions reflect the small 21-call timing sample, reliance on external performance and price claims, network latency, privacy exposure, and the protection gap created by default fail-open behavior.

Claim traceability scores 2: core policy maps to named source modules, variables, thresholds, and tests, while external pricing and privacy assertions identify sources; however, the core implementation files, host manifests, and raw calibration records are not included in the supplied evidence. Cross-source corroboration scores 2: README, SECURITY, package metadata, LICENSE, and broad tests mutually support thresholds, installation, key permissions, and host dialects, but live API performance, the 662-skill scan, and end-to-end outcomes remain repository-authored claims without independent support here. Fact-inference separation scores 2: the documentation often labels claimed, measured, tested, and limited behavior, but promotional statements such as calling the design an exact equivalent or imperceptibly fast are mixed with evidence, and static test code is presented alongside assertions of actual passing and live measurement.