AuditPilot
A traceable enterprise audit workflow for evidence retrieval, governed execution, review, remediation, and delivery.
What does this agent do, and when should you use it?
AuditPilot is a FastAPI workbench for enterprise audit delivery, covering audit projects, control matrices, procedures, sampling, findings, remediation, and delivery packages. Its runtime coordinates Planner, Evidence, Control, Risk, Compliance, Remediation, Verification, and Delivery roles through a bounded, dependency-aware Plan / Execute / Reflect loop. Evidence work is supported by multi-path retrieval, metadata filtering, source locations, and an Evidence Graph linking tasks, steps, tool runs, and artifacts. The repository also includes a layered Evaluation Harness, release gating, human-review exits, RBAC plus project ABAC, and local SQLite WAL persistence. It fits self-hosted prototypes or internal tools that need governed audit traces, but it does not establish business accuracy, horizontal scale, or a complete enterprise identity stack.
A user creates audit scope in the web workbench and enters an audit subject, standard, or risk scenario in Agent Collaboration. The Hybrid Intent Router uses Working, Episodic, and Profile Memory to route work; the role chain plans, retrieves evidence, performs control and risk checks, and produces remediation and delivery results in a bounded dependency-aware loop. Agentic RAG uses TF-IDF, keyword retrieval, fusion reranking, and metadata filtering, returns page or section sources, and flags conflicting or missing evidence; semantic embeddings are optional. Skills / MCP-style Tools apply RBAC, tenant isolation, caching, and circuit-breaking before execution, while recording call logs and metrics. The Evaluation Harness assesses task results, trajectories, tool calls, evidence grounding, security permissions, context, and robustness; failed critical assertions block release, while audit, evaluation, task, and memory records persist in tenant-scoped SQLite WAL storage.
- An internal audit team needs one project workspace to manage scope, control testing, evidence requests, findings, and remediation status.
- An audit lead needs to inspect the page or section sources behind a conclusion and identify evidence gaps or contradictions.
- A platform team needs pre-release regression evaluation across task outcomes, trajectories, tools, security, and robustness, with critical assertions acting as release gates.
- A multi-tenant audit environment needs RBAC, project-membership checks, and traceable tool use at HTTP and Skill boundaries.
- A delivery team needs high-risk or low-confidence outputs to collect more evidence and receive human review before remediation and delivery packaging.
What are this agent's strengths and limitations?
- Combines the audit workbench, dependency-aware runtime, evidence retrieval, human review, remediation, and delivery packaging in one traceable workflow.
- Its Evaluation Harness assesses more than final outputs: it covers trajectories, tools, evidence, security, context, and robustness, with critical assertions able to block releases.
- Hybrid RAG offers source locations, contradictory-evidence detection, missing-evidence prompts, and fail-closed filtering without requiring a local embedding model.
- It documents concrete authorization and storage controls: RBAC at HTTP and Skill boundaries, project ABAC, tenant isolation, signed checkpoints, and SQLite WAL optimistic concurrency.
- The default persistence layer is single-node SQLite WAL, and RAG chunks remain a single-node snapshot/index, so it should not be assumed to be a multi-node production architecture.
- The built-in evaluation suite is a regression and smoke baseline; independent held-out data and domain-expert review are still needed for business-accuracy or release-grade claims.
- Its bearer-token RBAC is only a deployable baseline; enterprise OIDC/SSO, external KMS, object storage, and multi-node databases require additional integration.
- Local semantic embeddings are not installed by default; enabling them requires requirements-embeddings.txt and RAG_ENABLE_EMBEDDINGS=1.
How do you install or deploy this agent?
Python 3.10 or later is required. On macOS/Linux:
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp config.env.example config.env
python start.py
On Windows PowerShell:
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
Copy-Item config.env.example config.env
python start.py
Open the address printed by the terminal. Without a model key, the application runs in deterministic fallback mode. For LLM enhancement, set DEEPSEEK_API_KEY or a compatible-provider variable in the local config.env. Shared or network deployments require SECURITY_MODE=enforced, AUDITPILOT_API_TOKENS_JSON, and AUDIT_LOG_SIGNING_KEY.
How do you use this agent?
After starting the server, open the address printed in the terminal. In Agent Collaboration, enter an audit subject, standard, or risk scenario to observe planning, evidence, control, risk, and remediation collaboration. In Audit Projects, define scope and run control tests, Deep Research, evidence requests, and the remediation loop. Maintain searchable audit knowledge in Knowledge Base, run layered assessments and inspect release gates in Agent Evaluation, and review tool schemas, permission declarations, call records, and runtime status in Skills / MCP.