Skill Scanner - Security Scanner for Agent Skills
Scan AI agent skills for prompt injection, data exfiltration, and malicious code.
Evidence shows: scanner defaults to static analysis only, no automatic file uploads; LLM and VirusTotal external services require explicit opt-in and API keys; docs emphasize human review. Deductions: no explicit user confirmation mechanism (e.g., pre-scan prompt), and external service calls may send data but no detailed data flow transparency is provided.
Evidence shows: comprehensive test suite and CI configuration, dependencies locked in uv.lock, and error messages provided on failure. Deductions: no detailed failure recovery mechanisms or error handling documentation.
Evidence shows: supports multiple skill formats (OpenAI Codex, Cursor, Claude Code), provides lenient mode and non-standard format scanning, and supports custom policies and rules. Deductions: no explicit adaptation details for different environments (e.g., CI, local).
Evidence shows: clear documentation structure including installation, quick start, CLI options, example output; Apache 2.0 license; version tags and changelog links. Deductions: no detailed changelog content, and maintenance responsibility not explicitly assigned.
Evidence shows: multiple output formats (SARIF, HTML, JSON) suitable for CI/CD integration; scanner has clear value for Agent Skills security detection. Deductions: no performance benchmarks or comparison with other tools.
Evidence shows: README clearly distinguishes detection results from guarantees, and provides architecture docs and threat taxonomy. Deductions: no independent third-party verification or cross-source corroboration.
- External services (LLM, VirusTotal, AI Defense) may send skill content to third parties; configure API keys and network policies carefully.
- Scan results do not guarantee security; manual review is required, especially in high-risk scenarios.
- No rollback mechanism provided; false positives may cause CI failures, so configure failure policies appropriately.
What does this agent do, and when should you use it?
Skill Scanner is a best-effort security scanner for AI Agent Skills, developed by the Cisco AI Defense team. It combines pattern-based detection (YAML + YARA), LLM-as-a-judge, and behavioral dataflow analysis to maximize detection coverage while minimizing false positives. The tool supports OpenAI Codex Skills and Cursor Agent Skills formats, and with --lenient mode can scan non-standard formats like Claude Code commands. It offers a CLI, Python SDK, and GitHub Actions integration with SARIF output for CI/CD. The project is open-source under Apache 2.0 and requires Python 3.10+.
Skill Scanner reads a skill directory, scans metadata files and scripts, and detects malicious patterns. It includes multiple analyzer engines: static analysis using YAML and YARA rules, bytecode analysis for .pyc integrity, pipeline analysis for command taint in shell pipelines, behavioral analysis via AST dataflow, LLM analyzer for semantic evaluation through provider APIs, a meta-analyzer for false-positive filtering, plus VirusTotal and Cisco AI Defense cloud integrations. Users trigger scans via CLI commands like skill-scanner scan or scan-all, or programmatically through the Python SDK. Output supports summary, json, markdown, table, sarif, and html formats, with options like --fail-on-severity for CI failure.
- A developer scanning a skill directory locally before committing it to a repository.
- A CI/CD engineer integrating the reusable GitHub Actions workflow to scan skills on every pull request, with inline annotations via GitHub Code Scanning.
- A security team running recursive scans across a skills repository, using scan-all --recursive --check-overlap to detect cross-skill overlap threats.
- A team using --use-llm and --enable-meta to perform deep semantic analysis and suppress false positives.
- An organization enforcing a security baseline with preset policies like strict, balanced, or permissive, or a custom policy file via --policy.
What are this agent's strengths and limitations?
- Multi-engine detection combining static, behavioral, LLM, and cloud services for broader coverage.
- Provides pre-commit hook and GitHub Actions integration for automated scanning in development workflows.
- Supports multiple output formats including SARIF, enabling direct integration with GitHub Code Scanning.
- Requires external API keys for full functionality (e.g., LLM analyzer needs SKILL_SCANNER_LLM_API_KEY).
- Scanner can produce false positives, requiring human review and tuning; it should not be the sole security measure.
- Support for non-standard skill formats requires --lenient mode, which may lower accuracy.
How do you install or deploy this agent?
Installation requires Python 3.10+ and uv (recommended) or pip. Using uv: uv pip install cisco-ai-skill-scanner. Using pip: pip install cisco-ai-skill-scanner. Optional cloud provider extras: pip install cisco-ai-skill-scanner[bedrock], [google], [vertex], [azure], or [all].
How do you use this agent?
For a guided start, run skill-scanner with no arguments to launch the interactive wizard. Or use the CLI directly: skill-scanner scan /path/to/skill --use-llm --enable-meta. Set environment variables SKILL_SCANNER_LLM_API_KEY and SKILL_SCANNER_LLM_MODEL to enable the LLM analyzer. Use scan-all --recursive to scan multiple skills recursively, and scan-repo owner/repo to scan a GitHub repository. To generate SARIF output for GitHub Code Scanning: skill-scanner scan-all ./skills --fail-on-severity high --format sarif --output results.sarif. Python SDK example: from skill_scanner import SkillScanner; from skill_scanner.core.analyzers import BehavioralAnalyzer; scanner = SkillScanner(analyzers=[BehavioralAnalyzer()]); result = scanner.scan_skill("/path/to/skill").