Agentic Radar
A CLI security scanner that maps agentic workflows, their tools, MCP servers, and selected runtime risks.
Per-dimension scores and reasoning
Evidence shows: The tool primarily runs locally, but advanced features (e.g., prompt hardening, testing) call LLM APIs, potentially sending code or prompts externally. No explicit user confirmation mechanism or least-privilege design was found. Data flow transparency is explained in README (local run, optional LLM), but no detailed data flow diagram or logs are provided. Sensitive data handling is not explicitly stated. Dependency security: pyproject.toml lists dependencies but no vulnerability scanning or lock files are provided. External effects: the tool generates report files, but impact on the system is not described. Rollback mechanism is not mentioned. Source attribution: author info is in pyproject.toml, but publisher is unverified. Deductions: lack of user confirmation, least privilege, and sensitive data handling evidence; dependency security not fully demonstrated.
Evidence shows: README and pyproject.toml descriptions are consistent; feature list matches documentation. Dependency availability: dependencies are listed, but no lock files or version compatibility notes are provided. Failure messages: no examples of error handling or user prompts are provided. Deductions: dependency availability not fully verified; failure messages not demonstrated.
Evidence shows: README clearly identifies target audience (developers, researchers, security professionals) and use cases (scan, test, CI/CD). Capability boundaries are defined via framework support matrix (e.g., OpenAI Agents, CrewAI, etc.). Trigger precision: commands and options are clear (e.g., scan, test). Environment fit: supports Python 3.9+, with extra installation options. Deductions: some features (e.g., test) only support specific frameworks, but documentation explains this.
Evidence shows: README is well-structured with installation, usage, advanced features, FAQ. Installation notes are detailed, including pip and extra dependencies. Naming is stable; commands and options are consistent. Examples and FAQ provide usage examples and common questions. Known limitations: README mentions some features only support specific frameworks, but not all limitations are listed. License is Apache-2.0, standard. Versioning and changelog: version number in pyproject.toml, but no CHANGELOG provided. Maintenance responsibility: author info in pyproject.toml, but no maintenance plan. Deductions: known limitations incomplete, missing changelog, maintenance responsibility unclear.
Evidence shows: Output is an HTML report with visualization, tool identification, vulnerability mapping, etc., which is useful. Marginal value: provides security scanning and testing features, differentiating from existing tools. Cost-benefit: tool is free and open-source, but advanced features require API keys, potentially incurring costs. Deductions: cost-benefit not fully assessed; no performance or resource consumption data.
Evidence shows: Claims in README (e.g., feature support) are partially consistent with code and docs, but no independent verification sources are provided. Cross-source corroboration: blog and tutorial links provide additional info, but not verified within the repo. Fact-inference separation: README distinguishes feature descriptions from future plans, but does not explicitly mark inferences. Deductions: claims lack traceability, cross-source verification insufficient, fact-inference separation unclear.
- Advanced features (e.g., prompt hardening, testing) call LLM APIs, potentially sending code or prompts externally; verify data privacy policies before use.
- Dependencies lack lock files; consider using virtual environments and regularly checking for vulnerabilities.
- Testing feature currently supports only OpenAI Agents; support for other frameworks is limited.
What does this agent do, and when should you use it?
Agentic Radar is a Python command-line security scanner for agentic workflows. It scans LangGraph, CrewAI, n8n, OpenAI Agents, and Autogen code and produces a shareable HTML report. The report covers workflow visualization, external and custom tools, MCP server detection, and mappings from identified tools to known vulnerabilities. It can optionally harden detected system prompts, and it provides adversarial runtime testing for OpenAI Agents. Static workflow analysis and vulnerability mapping run locally, while optional advanced features such as prompt hardening may send detected system prompts to an LLM.
The command agentic-radar scan [OPTIONS] FRAMEWORK:{langgraph|crewai|n8n|openai-agents|autogen} reads a target workflow codebase; for example, agentic-radar scan langgraph -i path/to/langgraph/example/folder -o report.html. It identifies workflow structure, external and custom tools, and MCP servers, then writes an HTML report that maps tools to known vulnerabilities using the OWASP Top 10 LLM Applications and OWASP Agentic AI threats and mitigations references. With --harden-prompts, it improves detected system prompts and displays the hardened prompts in the report. agentic-radar test openai-agents "path/to/workflow.py" injects itself into the supplied workflow, runs tests for prompt injection, PII leakage, harmful content generation, and fake-news generation, and prints a terminal table with agent name, injected input, output, pass/fail result, and explanation. A YAML file passed through --config can define custom tests and choose whether built-in tests are included.
- A security engineer scans a LangGraph repository before merge and shares the generated HTML workflow and tool-risk report with reviewers.
- A CrewAI team inventories predefined and custom tools, then checks their documented vulnerability mappings.
- A developer maintaining n8n workflows needs a report of MCP servers used in those workflows.
- An OpenAI Agents team wants to simulate prompt injection, PII leakage, or harmful-output attacks against an executable workflow.
- A team operating agent code in CI/CD wants scans to run after pushes and the generated report uploaded as a GitHub Actions artifact.
- A team using OpenAI Agents, CrewAI, or Autogen wants to inspect automatically hardened versions of detected system prompts.
What are this agent's strengths and limitations?
- A single scan combines workflow visualization, tool inventory, MCP-server detection, and vulnerability mapping instead of producing only one of those views.
- Its core static analysis and vulnerability mapping run locally, which suits teams that want those analyses to remain on their machine.
- Scanning is documented for five frameworks: OpenAI Agents, CrewAI, n8n, LangGraph, and Autogen.
- OpenAI Agents runtime testing supports built-in adversarial scenarios as well as YAML-defined custom inputs and success conditions.
- A provided GitHub Actions workflow example can run scans on pushes and upload the report as an artifact.
- Runtime vulnerability testing is currently documented only for OpenAI Agents; the roadmap marks it unavailable for CrewAI, n8n, LangGraph, and Autogen.
- Prompt hardening is documented for OpenAI Agents, CrewAI, and Autogen, but not for n8n or LangGraph.
- Runtime tests require
OPENAI_API_KEYorAZURE_OPENAI_API_KEY, and prompt hardening requiresOPENAI_API_KEY. - CrewAI predefined-tool descriptions and some features need an extra install;
crewai-toolsis limited to Python >= 3.10 and < 3.13. - When advanced LLM features are used, detected system prompts may be sent to an LLM.
How do you install or deploy this agent?
Install Python and pip, then run:
pip install agentic-radarVerify the installation:
agentic-radar --versionFor CrewAI runtime testing or predefined-tool descriptions, install pip install "agentic-radar[crewai]"; its crewai-tools dependency supports Python >= 3.10 and < 3.13 only. For OpenAI Agents runtime testing, install pip install "agentic-radar[openai-agents]". Runtime testing requires OPENAI_API_KEY or AZURE_OPENAI_API_KEY.
How do you use this agent?
For a first scan, run:
agentic-radar scan langgraph -i path/to/langgraph/example/folder -o report.htmlOpen report.html to inspect the workflow, tools, MCP servers, and vulnerability mappings. Before testing an OpenAI Agents workflow, set OPENAI_API_KEY or AZURE_OPENAI_API_KEY, then run:
agentic-radar test openai-agents "examples/openai-agents/basic/lifecycle_example.py"For prompt hardening, set OPENAI_API_KEY and run:agentic-radar scan openai-agents --harden-prompts -i examples/openai-agents/basic/lifecycle_example -o report.html.
How does this agent compare with similar options?
For visualization alone, the repository points users to the separate Agentic Visualizer. Agentic Radar has a broader documented scope: it adds tool identification, MCP-server detection, and vulnerability mapping, plus selected prompt-hardening and runtime-testing capabilities.
FAQ
Does scanning send my source code elsewhere?
What credentials do runtime tests need?
OPENAI_API_KEY or AZURE_OPENAI_API_KEY, either as an environment variable or in a .env file.Which frameworks support the test command?
Can I add my own attack cases?
tests with name, input, and success_condition through --config custom_tests.yaml; set include_default_tests: false to exclude built-in tests.