DeepTeam - LLM Red Teaming Framework
An open-source framework for pen-testing LLMs and AI agents with simulated attacks like jailbreaking and prompt injection, plus production guardrails.
Evidence shows no documentation on permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust criteria are unmet, hence score 0.
Self-consistency: Version numbers in README and pyproject.toml are consistent (1.0.8), but no evidence of internal consistency like API docs matching code. Dependency availability: Dependencies are listed, but no lock file or version range verification. Failure messages: No documentation on error handling or failure messages.
Audience and scenarios: README clearly targets LLM security engineers with various vulnerability and attack scenarios. Capability boundaries: No explicit limitations or boundaries. Trigger precision: No detailed configuration options or trigger conditions. Environment fit: Supports Python 3.9-3.14, but no OS or hardware requirements.
Information architecture: README is well-structured with quickstart and feature lists. Install notes: pip install command provided. Naming stability: No naming conventions or API stability notes. Examples and FAQ: Multiple code examples but no FAQ. Known limitations: Not mentioned. License: Apache-2.0 license file complete. Versioning and changelog: Version in pyproject.toml but no changelog. Maintenance responsibility: Author info clear but no maintenance policy.
Output usability: Provides risk assessment results in dataframes and JSON. Marginal value: 50+ vulnerabilities and 20+ attacks offer clear value. Cost-benefit: No performance or cost data.
Claim traceability: Feature claims in README lack test or evidence support. Cross-source corroboration: No external validation. Fact-inference separation: No clear distinction between facts and inferences.
- No permission management or user confirmation mechanisms, may execute high-privilege operations by default.
- Dependencies are not pinned, posing supply chain risks.
- No documentation on sensitive data handling, potential data leakage.
- No rollback mechanism, errors may be hard to recover.
What does this agent do, and when should you use it?
DeepTeam is an open-source red teaming framework for LLM systems, AI agents, RAG pipelines, and chatbots. It simulates 50+ out-of-the-box vulnerabilities (e.g., bias, PII leakage, SQL injection) and 20+ research-backed adversarial attacks (e.g., prompt injection, Crescendo jailbreak) to uncover weaknesses. All evaluations run locally on your machine, using LLM-as-a-Judge metrics to produce binary pass/fail scores with reasoning. It also offers 7 production-ready guardrails for real-time input/output protection, and maps to established safety frameworks like OWASP, NIST, and MITRE. DeepTeam can be used via CLI or Python API, built on DeepEval, and integrates with the Confident AI platform for managing and monitoring risks.
DeepTeam accepts a model_callback function that wraps the LLM system under test. It dynamically generates adversarial inputs based on specified vulnerabilities (like Bias, Toxicity) and attacks (like PromptInjection, Crescendo Jailbreak), then evaluates the model's outputs using LLM-as-a-Judge metrics (e.g., BiasMetric) to assign binary scores (0 or 1) and calculate pass rates. It operates entirely locally without needing a pre-built dataset. Users can run red teaming from the CLI (with YAML configs) or programmatically in Python. It also provides a Guardrails class for input and output guards, such as PromptInjectionGuard and PrivacyGuard. Finally, it generates risk assessments that can be saved as JSON and optionally uploaded to the Confident AI platform for centralized management and monitoring.
- Security engineers evaluating an internal LLM app against prompt injection and jailbreaking attacks, using CLI or Python API for quick scans.
- AI developers red-teaming RAG pipelines before deployment to detect privacy risks like PII leakage and prompt leakage.
- Enterprise security teams aligning with OWASP Top 10 or NIST AI RMF to ensure compliance with industry standards.
- Teams requiring real-time production protection, using Guardrails API to classify and block risky LLM inputs and outputs dynamically.
- Agent developers testing multi-agent systems for vulnerabilities like Goal Theft and Excessive Agency using agentic attack categories.
- Organizations wanting to centralize red teaming results and monitor production vulnerabilities by integrating the Confident AI platform.
What are this agent's strengths and limitations?
- Runs locally, preserving data privacy and not requiring external services
- Comprehensive vulnerability library (50+) and attack methods (20+)
- Direct mapping to security frameworks like OWASP and NIST
- Production-ready guardrails for real-time threat mitigation
- Flexible integration via CLI and Python API
- Relies on LLMs as judges, so accuracy depends on their quality
- Requires writing a model_callback to integrate your system; not plug-and-play
- Multi-turn attack methods may be resource-intensive and time-consuming
- Extended features through the Confident AI platform may require registration and payment
How do you install or deploy this agent?
Install with pip: pip install -U deepteam. You need to set the OPENAI_API_KEY environment variable (or use a custom model supported by DeepEval).
How do you use this agent?
- Import modules:
from deepteam import red_team; from deepteam.vulnerabilities import Bias; from deepteam.attacks.single_turn import PromptInjection. 2. Define an async model_callback function that takes a string and returns a string. 3. Callred_team(model_callback=..., vulnerabilities=[Bias(types=["race"])], attacks=[PromptInjection()]). 4. Run the script to generate a local risk assessment.