Agent Audit

SECURITY.md explicitly states pure static analysis, no execution of scanned code, no telemetry, no network requests, and no privileged execution requiring only read access, which supports least_privilege and external_effects at 2. However, the inspect subcommand launches MCP server processes (e.g. npx -y ...), an external side effect documented only as 'read-only, never calls tools' with no discussion of the risk of spawning third-party processes, so external_effects is not full marks. user_confirmation: CLI offers --fail-on/--baseline gating but no interactive confirmation, and inspect spawns external processes without a prompt, so 1. sensitive_data_handling: semantic secret detection and masked output (sk-a***) exist, but findings may embed secret fragments and no explicit output-redaction guarantee is shown, so 2. dependency_security: only a verbal 'minimal dependencies' claim, no lockfile or dependency-audit evidence, and CI installs bandit/semgrep best-effort, so 1. rollback: only a baseline mechanism for incremental comparison; no rollback story for scanned targets or config changes, so 1. source_attribution: OWASP/CWE references and a citation exist, but publisher identity is unverified and reports mix the names Argus and agent-audit, muddying attribution, so 1.

self_consistency is contradicted by the evidence: README says 66 rules, the description says 51, the spec is v0.14.1, SECURITY.md supports only 0.15.x while README is at v0.20.0; test counts appear as both 1924 and 1873; the checked-in result file is layer1_v0.19.0.json yet paired with v0.20.0 numbers, so 1. dependency_availability: PyPI badge and poetry install instructions exist, but no lockfile evidence and CI relies on third-party actions and optional tools, so 1. failure_messages: CI comments document an exit-code contract (0/1/2), empty-SARIF detection, and an import-path regression guard, showing attention to diagnosable failure, so 2.

audience_and_scenarios: clearly targets agent developers, security engineers, and MCP teams with CI integration scenarios, so 2. capability_boundaries: 'Current Scope' and spec section 6 clearly list static-only analysis, intra-procedural taint, and Python focus, so 2. trigger_precision: relies on heuristics and confidence tiers, but the OpenClaw report self-reports 615 of 680 (90.4%) likely false positives and README admits FP 90 with unlabeled cases, so 1. environment_fit: Python 3.9-3.12 matrix, offline operation, and SARIF/JSON/markdown outputs fit common CI environments, so 2.

information_architecture: layered docs (README, spec, rules, architecture, CI integration, benchmarks) are well structured, so 2. install_notes: pip install, poetry dev setup, and a GitHub Action input table are provided, so 2. naming_stability: dual naming (Argus vs agent-audit), rule renames (AGENT-040 to 050), and cross-document version drift, so 1. examples_and_faq: quick start, sample report, collapsible Action example, and an intentionally vulnerable sample file, so 2. known_limitations: dedicated sections in both README and spec, so 2. license: full MIT text, so 3. versioning_changelog: no CHANGELOG file, only scattered in-document version notes, so 1. maintenance_responsibility: SECURITY.md response timelines and a CONTRIBUTING reference exist, but the publisher is unverified and there is no maintainer roster or release process, so 1.

output_usability: terminal report, JSON, SARIF, markdown, baseline, and exit-code gating make output directly CI-usable, so 2. marginal_value: MCP config auditing and tool-boundary taint target categories general SAST misses, a genuine differentiator, so 2. cost_benefit: lightweight install, but the high false-positive rate (90.4% likely FP on OpenClaw) implies heavy manual triage, and benchmark figures include a self-admitted non-reproducible adjusted value, so 1.

claim_traceability: some claims point to scripts and result files (precision_recall.py, results/layer1_v0.19.0.json), but version mismatch breaks the chain and 'first and only' style assertions lack independent support, so 1. cross_source_corroboration: numbers conflict across documents (rule counts 51/63/66, test counts, versions), so 1. fact_inference_separation: reports interleave inference about the Claude Code leak and OpenClaw trust model with measured findings, and admit scanning a 'Python port/reconstruction' rather than original source, blurring fact and inference, so 1.