Bug Hunter
An adversarial code-audit skill for AI coding agents: a multi-role pipeline finds security vulnerabilities, logic errors, and runtime bugs — scan-only by default, with fixing gated behind explicit permission.
Least privilege and confirmation design is well evidenced: scan-only default, explicit approval for edits/commits, warnings against --autonomous/--auto-commit, fail-closed boundaries, and SECURITY.md corroboration of Fixer scope/lock/worktree design justify full marks on external effects and least privilege. However, dependency security rests only on two devDependencies and CI description — no lockfile content, CVE audit, or supply-chain verification is in evidence (deducted to 1); sensitive-data handling is only indirect via SECURITY.md principles with no implementation shown; rollback is mentioned as a fix-report field but its mechanism is undocumented; publisher is unverified so attribution rests on consistent but unverified URLs (deducted to 2).
Self-consistency is strong: README, package. (3.2.0), and CI workflows (Node 22/24 matrix, tests, benchmark gate, preflight) agree, with fail-closed and schema validation described throughout. Dependency availability shows Node>=22, pinned pnpm, frozen lockfile, but the doctor self-check and runtime dependency resolution are described, not source-shown. Failure messaging appears only as policy language ('a failed phase, not a clean scan'); concrete error messages and recovery guidance are absent from the evidence, deducted to 2.
Audience, capability boundaries, and trigger precision are well evidenced: style issues explicitly out of scope, unsettled results routed to MANUAL_REVIEW/unreviewed, a multi-agent install target table, one-to-one flag-to-workflow mapping, and clear mode separation earn full marks. Environment fit lists eight agents and Node versions, but per-environment differences (context budgets, permission modes) are only summarized without per-environment verification detail, deducted to 2.
Information architecture and install notes are excellent: layered tables, doc links, dual npx/npm install paths, and explicit --agent advice. MIT license text is present (full marks), and known-limitations statements are honest (bundled-fixture-only benchmark, no zero-FP promise). But naming stability is questionable — the README describes an evolving interface across versions (version-pinned old-commit hero image, 'closest flag-based mode' phrasing); examples/FAQ are links to files not provided (usage-guide, troubleshooting absent from evidence); CHANGELOG.md is listed in the package inventory but its content is not shown; maintenance responsibility is only indirect via SECURITY.md response timelines and CI gates. These are deducted to 2.
Output usability is strong: canonical JSON artifacts plus report.md, four-way result separation (confirmed/dismissed/manual/unreviewed), and count/ID join checks serve CI/CD directly (full marks). Marginal value: the Hunter/Skeptic/Referee adversarial structure and FP-penalty incentive design are distinctive, but the only quantitative evidence is a self-reported bundled-fixture metric (precision 1.00 etc.), explicitly declared non-independent (deducted to 2). Cost-benefit gives median tokens and p95 latency but no comparison against alternatives (deducted to 2).
Fact/inference separation is exemplary: the README distinguishes bundled-fixture validation from independent benchmarking, forbids invented guarantees when docs are unavailable, and refuses to dress MANUAL_REVIEW as clean (full marks). Claim traceability: stage-to-schema-artifact mapping is clear, but referenced docs (world-class-protocol.md, SKILL.md) and actual script sources are not in evidence, so core mechanisms cannot be statically verified (deducted to 2). Cross-source corroboration is weak: all performance/precision figures originate from the project's own files and CI gates, with no third-party audit or independent benchmark (deducted to 1).
- This is a static review with no executed scans or installs; all performance figures (precision 1.00, etc.) are self-reported bundled-fixture results, not independent benchmarks.
- Core mechanisms (SKILL.md, prompts, script sources, CHANGELOG, usage-guide) are absent from the evidence; fail-closed, Fixer-scope, and rollback claims cannot be verified from source here.
- Dependency security evidence is thin: no lockfile content or CVE audit results are provided; prefer the --path single-repo install and inspect installed content in an isolated environment first.
- Do not use --autonomous or --auto-commit unless you intend to grant unattended write permissions; run the documented scan-only mode first.
- Publisher is unverified and the npm release may lag GitHub main (per the README); pin a specific commit and diff before production use.
What does this agent do, and when should you use it?
Bug Hunter is an installable code-audit skill for file-based AI coding agents including Claude Code, Codex, Cursor, GitHub Copilot, Kiro, Windsurf, OpenCode, and Factory Droid CLI. Its pipeline combines deterministic risk triage, architecture recon, and hypothesis-driven retrieval with three adversarial roles: a Hunter makes evidence-backed bug claims, a Skeptic tries to disprove each one, and a Referee issues REAL_BUG, NOT_A_BUG, or MANUAL_REVIEW verdicts. The default run is scan-only and single-pass; source edits, autonomous fixing, and commits each require separate explicit permission. Every phase writes schema-validated JSON artifacts under .bug-hunter/, consumable by CI/CD, security dashboards, and PR gates. v3.2.0 adds a measurable quality gate covering precision, recall, F1, calibration, and repeat stability, plus fast/balanced/assurance adaptive execution profiles. The fix workflow uses canary-first rollout, inter-batch verification, a failure-rate circuit breaker, and rollback on a dedicated fix branch.
The installed skill is invoked through your coding agent (e.g. sending /bug-hunter src/auth), not by running a scan in a shell. First scripts/triage.cjs performs a deterministic, model-free risk triage that selects scan scope and order. Recon maps the stack, architecture, and trust boundaries; the Hunter reads production code for reachable bugs — wrong conditions, off-by-one errors, races, injection, XSS, SSRF, path traversal, resource leaks — and every claim must carry a source location, evidence, and a runtime trigger. The Skeptic re-traces code with 14 hard exclusions for recurring non-bugs; the Referee independently re-verifies and delivers verdicts, enriching Critical/High security findings with STRIDE category, CWE identifiers, and CVSS 3.1 vectors. Hybrid verification runs argv-only tests, type checks, static checks, and fuzz checks, failing closed when required checks fail. Optional flags extend scope: --deps scans npm/pnpm/Yarn/Bun lockfiles for CVEs with reachability classification, --threat-model generates a STRIDE threat model, and --pr/--pr-security audit pull requests and changed code. Confirmed findings are classified in fix-strategy. as safe-autofix, manual-review, etc.; --plan produces a fix plan without edits, while --fix --approve executes canary fixes with batch verification and rollback on a dedicated branch, recording results in fix-report..
- A development team wanting a read-only security and logic audit of an entire repository before release without letting the AI touch code — just run /bug-hunter.
- An engineer reviewing pull requests who uses --pr or --pr-security to narrow the audit to the diff about to merge, with STRIDE and dependency context.
- A JavaScript/TypeScript team using --deps to scan npm, pnpm, Yarn, or Bun lockfiles for high/critical CVEs and classify reachability.
- A security lead who needs a STRIDE threat model before launch, generated via --threat-model into .bug-hunter/threat-model.md with trust boundaries, assets, and threats.
- A developer who wants reviewed, automated fixes for confirmed bugs using --fix --approve, getting a verified canary rollout with rollback on a fix branch.
- A team wiring audit results into CI/CD gates or security dashboards by consuming machine-readable artifacts like .bug-hunter/scan-report..
What are this agent's strengths and limitations?
- The adversarial Hunter/Skeptic/Referee structure, plus hard evidence requirements and the Skeptic's 14 exclusions, systematically cuts false positives; unsettled results surface explicitly as MANUAL_REVIEW instead of looking clean.
- Scan-only by default with fixing and committing as separate, explicitly granted permissions; the fix path adds a dedicated branch, canary rollout, batch verification, a circuit breaker, and rollback — fail-closed throughout.
- Every phase emits schema-validated JSON artifacts (scan-report., fix-plan., etc.) that plug directly into CI/CD, PR gates, and security dashboards.
- One skill covers source auditing across 14 languages plus STRIDE threat modeling, npm/pnpm/Yarn/Bun dependency CVE scanning, and PR/change review, installable into eight or more coding agents.
- Ships a deterministic regression benchmark fixture and quality gate (precision, recall, F1, calibration, stability) verifiable via pnpm quality:world-class.
- It depends on an already-installed file-based coding agent as the runtime; it is not a standalone CLI scanner, and scans start through the agent rather than a bug-hunter scan command.
- Dependency auditing only supports JavaScript/TypeScript (npm, pnpm, Yarn, Bun lockfiles); Python, Go, and Rust return scanner-unsupported and are never reported clean.
- Requires Node.js 22+, and larger scans consume significant model tokens (the benchmark fixture records a median of ~12,090 tokens per true positive), so heavy use has real cost.
- Automatic rollback requires commit-backed checkpoints; runs without commit permission may need manual restoration, and line-range, symlink, and submodule compliance is not independently verified after a patch.
- Autonomous-fix flags (--autonomous, --auto-commit) carry sensitive authority boundaries; the project advises reading SECURITY.md before autonomous fixing in a sensitive repository, and misuse can broaden write access.
How do you install or deploy this agent?
Requires Node.js 22 or newer and a supported file-based coding agent. Install from the current GitHub source (matches the docs): npx --yes https://github.com/codexstar69/bug-hunter/archive/refs/heads/main.tar.gz install --agent codex, replacing --agent with claude-code, codex, cursor, copilot, kiro, windsurf, opencode, droid, or agents. Then verify: npx --yes https://github.com/codexstar69/bug-hunter/archive/refs/heads/main.tar.gz doctor --agent codex. For the latest published npm release (may lag GitHub main): npm exec --yes --package=@codexstar/bug-hunter@latest -- bug-hunter install --agent codex. Always pass --agent explicitly when multiple coding agents are installed. Restart the agent if it was open during installation. Removal is manual.
How do you use this agent?
From the repository you want to audit, send a natural-language prompt through your coding agent, e.g.: "Use the bug-hunter skill to scan this repository. Do not edit files. Return the final report and call out every item that needs manual review." That is the recommended first run — scan-only and single-pass. Slash-command syntax also works: /bug-hunter (full repo, one pass), /bug-hunter src/auth (specific path), --staged, --pr, --pr-security, --deps, --threat-model, --plan (plan without edits), --fix --approve (reviewed fixes), --loop (complete queued coverage). No flags means scan-only; --autonomous and --auto-commit grant unattended fixing and commit permission respectively and should be used deliberately. Results are written to .bug-hunter/ (add it to .gitignore), with scan-report. and report.md as the primary outputs.
How does this agent compare with similar options?
The README contrasts Bug Hunter with generic AI code-review tools and security scanners that produce long lists of candidate issues for developers to triage themselves: its differentiator is that every finding must survive a Skeptic challenge and an independent Referee verdict, with uncertain results shown explicitly rather than presented as clean. No specific competitors are named.