agent-qa

Code shows path-resolved workspace file operations, atomic replace with symlink resolution, single-writer locks, and CI running with minimal GitHub permissions (contents: read); pnpm overrides pin numerous dependencies and onlyBuiltDependencies restricts build scripts — dependency_security earns 2. But for an agent that drives browsers/mobile and calls models, the repo documents no permission model, user confirmation flows, data flow to models, secret handling, or rollback of agent actions; the remaining trust criteria sit at 1. least_privilege: file managers appear workspace-scoped but no documented capability/permission surface for the agent itself (1). user_confirmation: no evidence of confirmation gates in provided files (1). data_flow_transparency: memory-as-reviewable-files is stated but what leaves the machine is not documented (1). sensitive_data_handling: a deleted test named 'declared-secret-masking-deleted' hints at masking but no substantive evidence (1). external_effects: browser/mobile automation with no consent/scope documentation (1). rollback: atomic writes are robust for files but nothing covers undoing agent-driven external effects (1).

CI is unusually thorough (sharded matrix, macOS legs, MCP conformance gating on its own exit code, documented timeout rationale), enforces --frozen-lockfile and pinned pnpm/node — dependency_availability 2. Validation returns structured errors with line/column/suggestion — failure_messages 2. The CI file's own comments admit that until 2026-08-02 every workflow was workflow_dispatch-only and the test-bearing Release workflow had never run, with green claims resting on local runs — a self-acknowledged consistency gap that caps self_consistency at 2.

Audience (QA teams), scenarios (web/Android/iOS via CLI, dashboard, MCP/skills) and environment (Node 24+, pnpm 10.6.1, platform-specific macOS CI for BSD/proc differences) are clearly stated — audience_and_scenarios and environment_fit 2. Deducted: capability boundaries reduced to one line ('Early stage. Expect bugs') with no statement of where natural-language tests break down (capability_boundaries 1); trigger precision of the MCP/skills interfaces is undocumented (trigger_precision 1).

Monorepo architecture is coherent, install steps are concrete, @vostride naming is consistent, and LICENSE.md is a complete FSL-1.1-ALv2 text matching the README badge — information_architecture, install_notes, naming_stability, license each 2. Deducted for: no in-repo CHANGELOG and no version field (versioning_changelog 1), no FAQ (examples_and_faq 1), known limitations limited to one sentence (known_limitations 1), no CONTRIBUTING/ownership files beyond a copyright line and issue link (maintenance_responsibility 1). Note the registry license metadata is NOASSERTION and FSL is source-available, not OSI open source.

README shows real CLI output, failure evidence side-by-side with screenshots, and a knowledge-traceability panel — output_usability 2. Memory + validated self-improvement for QA is a differentiated angle — marginal_value 2. Deducted: nothing on model costs, run/token overhead, or ROI versus existing E2E tooling — cost_benefit 1.

Benchmark claims (AndroidWorld 100% pass@1, AppControlBench 99.6%, '#1' ranks, gpt-6-sol) resolve only to the publisher's own blog, with no run evidence, reproduction scripts, or raw results inside the repository — claim_traceability and cross_source_corroboration 1. Claims and facts are not separated in the README; only the CI comments carry traceable issue-numbered factual context — fact_inference_separation 1.