Numbat
Endpoint visibility into AI agent activity: local detection, optional pre-action blocking, and forensic reconstruction after the fact.
Evidence shows read-only defaults, monitor-only default with opt-in enforce limited to rules explicitly marked, and outbound limited to configured HTTP sinks with HTTPS by default per SECURITY.md. Deductions: hook uninstall/rollback semantics are only named in the command list without full recovery detail for state and hook config; publisher identity is unverified and no maintainer attribution appears in evidence, so least_privilege/rollback/source_attribution do not earn full marks.
CI covers three-OS tests with race, fuzz targets, and go mod tidy consistency gates; SECURITY.md itemizes failure behavior (decode errors suppress deny, 4 MiB hook limit, hook status caveats), giving documented failure semantics. Not a 3 because actual error message samples and recovery details beyond configuration verification are absent from this evidence set.
The Scope section and referenced coverage matrix document per-agent support boundaries, known gaps, and WSL/Windows path differences, and state numbat never executes artifact commands — exceptionally complete boundary declaration (3). Audience fit and trigger precision are only inferable from rule structure; no false-positive data or environment-specific validation is shown, so those stay at 2.
Documentation architecture (coverage, event model, CLI, deployment, enforcement, rules, rule catalog, schemas) is thorough and install notes cover download+checksums, go install, and source builds (3). But no CHANGELOG and no maintainer list/CODEOWNERS appear in evidence; the only maintenance statement is 'only the most recent minor release receives security fixes', so versioning_changelog and maintenance_responsibility score 1.
Versioned NDJSON records, JSON Schemas, and case bundles with SHA-256 manifests make outputs directly consumable (3). However, this evidence set cannot substantiate actual detection quality or false-positive rates; marginal_value and cost_benefit rest on claimed features without benchmarks, so they stay at 2.
Records carry evidence_refs, cited_event_ids, schema_version, and rule versions, and the docs explicitly state findings are rule matches, not proof of compromise — excellent fact/inference separation (3). Many README-linked docs are not in this evidence set, so cross-source corroboration is not possible, and all examples are described as controlled replays rather than executed runs, so claim_traceability and cross_source_corroboration are deducted accordingly.
- Publisher identity is unverified; independently confirm provenance and supply chain before enterprise adoption.
- collect binds to 127.0.0.1:4318 with no client authentication; rebinding to another interface provides no TLS/auth — isolate at the network layer.
- Redaction is pattern masking only and project-path hashes are unsalted stable join keys; manually review records and case bundles before sharing.
- In enforce mode, decode/evaluation errors suppress the deny; the agent host remains the enforcement point — do not treat numbat as a fail-closed control.
- This is a static review with nothing executed; detection quality, false-positive rates, and actual hook loading must be validated before deployment.
What does this agent do, and when should you use it?
Numbat is an open-source endpoint security tool from Perplexity that gives visibility into AI agent activity on developer machines. It observes supported desktop, CLI, IDE, and gateway agents through local hooks and plugins, OTLP/HTTP logs, and on-disk session artifacts. Live and at-rest activity are normalized into a single event model and evaluated by the same CEL rule engine, with all detection running locally. Records are emitted as versioned NDJSON to stdout, a local file, or optionally an HTTP sink, with events and findings retaining source references. Blocking is off by default and restricted to supported synchronous pre-action hooks; every shipped rule is monitor-only. It ships as a single cgo-free binary for macOS, Linux, and Windows on amd64 and arm64.
Numbat collects data through three paths: installing local hooks/plugins for supported agents (e.g., Codex and Claude Code pre-action callbacks), receiving OTLP/HTTP logs, and scanning on-disk session artifacts left by agents even without prior numbat instrumentation. Normalized activity is evaluated locally against built-in CEL rules, multi-step sequence rules, and custom YAML rules, producing five record types as versioned NDJSON: events, findings, enforcement decisions, indicators, and scan summaries. The CLI spans numbat agents (read-only discovery), numbat scan (artifact scanning), numbat hook install/status/uninstall and numbat collect (live capture), numbat ship (record delivery), numbat rules check/list/test (rule development), numbat timeline (per-session timelines), and numbat case build/verify (portable case bundles with SHA-256 manifests). Scanning is read-only with secret redaction. When enforcement is enabled, a pre-action call matching a rule marked enforce: true receives an agent-specific deny response.
- Security teams monitoring developer workstations want real-time detection of high-risk agent network behavior, such as requests to the cloud metadata endpoint (169.254.169.254).
- Incident responders need to reconstruct multi-step attack chains (e.g., secret-file read followed by data egress) from session artifacts on machines where monitoring was never deployed.
- Platform engineering teams roll out a set of custom YAML policies across research endpoints via
--rules-dirfor consistent agent behavior rules. - Compliance auditors use the read-only
scanandtimelinecommands to produce case bundles with SHA-256 manifests for record-keeping and review. - Administrators who want to block specific actions (e.g., writing SSH authorized_keys) copy a shipped rule as an
enforce: truereplacement and install it on a supported pre-action hook. - Engineers building agent-security pipelines feed existing agent logs over OTLP/HTTP into the same detection engine.
What are this agent's strengths and limitations?
- Detection runs entirely on-device, and one CEL rule engine plus one event model serve both live hooks and at-rest artifact scans — the same rule fires in both modes.
- Multi-step sequence rules (e.g., chain.secret_read_then_egress) produce findings citing concrete event IDs with ATT&CK tags, giving an auditable evidence chain.
- Blocking is off by default and every shipped rule is monitor-only, making rollout low-risk; enforcement semantics are narrowly scoped to supported synchronous pre-action hooks.
- Single-binary distribution across macOS, Linux, and Windows on amd64/arm64, built without cgo, suits fleet-wide endpoint deployment.
- Records follow versioned JSON Schemas (v0.3.0) and retain source references, easing downstream SIEM integration.
- Blocking only works on supported synchronous pre-action hooks and requires manual rule overrides (same id, enforce: true, version bump) plus hook reinstallation; no out-of-the-box shipped rule blocks anything.
- At-rest reconstruction cannot recover activity an agent did not persist and is not disk or memory acquisition; findings are rule matches, not proof of compromise.
- Agent coverage is governed by the coverage matrix with documented deferred stores, fidelity limits, and root overrides; Windows native versus WSL path handling also differs.
- Hook trust requirements vary by agent and scope (e.g., Codex user hooks need manual trust, --managed hooks are trusted by policy), and hook status verifies configuration but not actual execution or delivery.
- Records can retain sensitive endpoint and agent context even after redaction; unsigned case bundles do not prove source authenticity or completeness.
How do you install or deploy this agent?
Option 1: download a release for macOS, Linux, or Windows (amd64/arm64) from https://github.com/perplexityai/numbat/releases; each release includes SHA-256 checksums. Option 2: install with Go 1.26.6 or newer: go install github.com/perplexityai/numbat/cmd/numbat@latest. Option 3: build a static binary from a checkout — macOS/Linux: CGO_ENABLED=0 go build -trimpath -o numbat ./cmd/numbat; Windows PowerShell: $env:CGO_ENABLED = "0"; go build -trimpath -o numbat.exe ./cmd/numbat. No credentials or cloud account required.
How do you use this agent?
- Read-only inventory and scanning (installs no hooks, changes no agent configuration): run
numbat agentsto discover agents,numbat scanto scan all discovered parser-backed agents, ornumbat scan --agent codexto limit discovery. 2. Live monitoring (Codex example):numbat hook install --agent codex --emit allinstalls hooks in monitor-only mode, writing events, findings, indicators, and enforcement decisions to~/.numbat/records.nd; verify withnumbat hook status --agent codex. Note you must review and trust the hook definition in the agent's hook settings (e.g., Codex CLI /hooks or Settings > Hooks). 3. Enable enforcement: copy the complete shipped YAML from rules/ into an operator directory, keep the same id, addenforce: true, bump the version, validate withnumbat rules check --rules-dir ./numbat-policy, then install withnumbat hook install --agent codex --emit all --rules-dir ./numbat-policy --enforce. 4. Also available:numbat timelinefor session timelines,numbat shipfor record delivery,numbat case build/case verifyfor case bundles. Runnumbat --helpfor the full command list.
FAQ
Will it block or interfere with normal agent operations?
enforce: true, installed with --enforce, are denied.Does it require network access or a cloud service?
Which agents and operating systems are supported?
Can it investigate machines where monitoring was never deployed?
numbat scan and timeline perform read-only forensic reconstruction from supported on-disk session artifacts without prior instrumentation; however, this is not disk/memory acquisition and cannot recover activity the agent never persisted.How do I add my own detection rules?
--rules-dir flags to scan, collect, hook, and rules check commands. Use --no-builtin-rules for an operator-only catalog and numbat rules test for testing.