DeepTeam - LLM Red Teaming Framework
An open-source framework for pen-testing LLMs and AI agents with simulated attacks like jailbreaking and prompt injection, plus production guardrails.
- Source repo
- confident-ai/deepteam
- Stars
- ★ 3k
- Last updated
- 2d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 29/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- Security engineers evaluating an internal LLM app against prompt injection and jailbreaking attacks, using CLI or Python API for quick scans.
- Not a fit if
- Users wanting plug-and-play without writing a model_callback
- Teams unwilling to sign up or pay for the Confident AI platform
- Source review
- 29/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
DeepTeam is an open-source red teaming framework for LLM systems, AI agents, RAG pipelines, and chatbots. It simulates 50+ out-of-the-box vulnerabilities (e.g., bias, PII leakage, SQL injection) and 20+ research-backed adversarial attacks (e.g., prompt injection, Crescendo jailbreak) to uncover weaknesses. All evaluations run locally on your machine, using LLM-as-a-Judge metrics to produce binary pass/fail scores with reasoning. It also offers 7 production-ready guardrails for real-time input/output protection, and maps to established safety frameworks like OWASP, NIST, and MITRE. DeepTeam can be used via CLI or Python API, built on DeepEval, and integrates with the Confident AI platform for managing and monitoring risks.
DeepTeam accepts a model_callback function that wraps the LLM system under test. It dynamically generates adversarial inputs based on specified vulnerabilities (like Bias, Toxicity) and attacks (like PromptInjection, Crescendo Jailbreak), then evaluates the model's outputs using LLM-as-a-Judge metrics (e.g., BiasMetric) to assign binary scores (0 or 1) and calculate pass rates. It operates entirely locally without needing a pre-built dataset. Users can run red teaming from the CLI (with YAML configs) or programmatically in Python. It also provides a Guardrails class for input and output guards, such as PromptInjectionGuard and PrivacyGuard. Finally, it generates risk assessments that can be saved as JSON and optionally uploaded to the Confident AI platform for centralized management and monitoring.
- Security engineers evaluating an internal LLM app against prompt injection and jailbreaking attacks, using CLI or Python API for quick scans.
- AI developers red-teaming RAG pipelines before deployment to detect privacy risks like PII leakage and prompt leakage.
- Enterprise security teams aligning with OWASP Top 10 or NIST AI RMF to ensure compliance with industry standards.
- Teams requiring real-time production protection, using Guardrails API to classify and block risky LLM inputs and outputs dynamically.
- Agent developers testing multi-agent systems for vulnerabilities like Goal Theft and Excessive Agency using agentic attack categories.
- Organizations wanting to centralize red teaming results and monitor production vulnerabilities by integrating the Confident AI platform.
How do you install or deploy this agent?
Install with pip: pip install -U deepteam. You need to set the OPENAI_API_KEY environment variable (or use a custom model supported by DeepEval).
How do you use this agent?
- Import modules:
from deepteam import red_team; from deepteam.vulnerabilities import Bias; from deepteam.attacks.single_turn import PromptInjection. 2. Define an async model_callback function that takes a string and returns a string. 3. Callred_team(model_callback=..., vulnerabilities=[Bias(types=["race"])], attacks=[PromptInjection()]). 4. Run the script to generate a local risk assessment.
What are this agent's strengths and limitations?
- Runs locally, preserving data privacy and not requiring external services
- Comprehensive vulnerability library (50+) and attack methods (20+)
- Direct mapping to security frameworks like OWASP and NIST
- Production-ready guardrails for real-time threat mitigation
- Flexible integration via CLI and Python API
- Relies on LLMs as judges, so accuracy depends on their quality
- Requires writing a model_callback to integrate your system; not plug-and-play
- Multi-turn attack methods may be resource-intensive and time-consuming
- Extended features through the Confident AI platform may require registration and payment
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| DeepTeam - LLM Red Teaming Framework This agent | 29 · Major gaps | Library / SDKFree + model costs | ★ 3k | 2d ago | Python | — |
| AI-Infra-Guard (A.I.G) – AI Red Teaming Platform by Tencent Zhuque Lab | 48 · Major gaps | Self-hosted serviceFreemium | ★ 6.9k | today | Python | — |
| Agentic Security | 45 · Major gaps | CLIFree + model costs | ★ 2k | today | Python | OpenAI API |
| PentestCode | 55 · Major gaps | CLIFree + model costs | ★ 852 | 16d ago | TypeScript | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows no documentation on permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust criteria are unmet, hence score 0.
Self-consistency: Version numbers in README and pyproject.toml are consistent (1.0.8), but no evidence of internal consistency like API docs matching code. Dependency availability: Dependencies are listed, but no lock file or version range verification. Failure messages: No documentation on error handling or failure messages.
Audience and scenarios: README clearly targets LLM security engineers with various vulnerability and attack scenarios. Capability boundaries: No explicit limitations or boundaries. Trigger precision: No detailed configuration options or trigger conditions. Environment fit: Supports Python 3.9-3.14, but no OS or hardware requirements.
Information architecture: README is well-structured with quickstart and feature lists. Install notes: pip install command provided. Naming stability: No naming conventions or API stability notes. Examples and FAQ: Multiple code examples but no FAQ. Known limitations: Not mentioned. License: Apache-2.0 license file complete. Versioning and changelog: Version in pyproject.toml but no changelog. Maintenance responsibility: Author info clear but no maintenance policy.
Output usability: Provides risk assessment results in dataframes and JSON. Marginal value: 50+ vulnerabilities and 20+ attacks offer clear value. Cost-benefit: No performance or cost data.
Claim traceability: Feature claims in README lack test or evidence support. Cross-source corroboration: No external validation. Fact-inference separation: No clear distinction between facts and inferences.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- No permission management or user confirmation mechanisms, may execute high-privilege operations by default.
- Dependencies are not pinned, posing supply chain risks.
- No documentation on sensitive data handling, potential data leakage.
- No rollback mechanism, errors may be hard to recover.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v1.0.9 First Stable Release 🎉
Release notes excerpt
We're excited to announce the stable release of DeepTeam v1.0.0 - a Python framework that orchestrates LLM red teaming. From simulating adversarial, malicious inputs to evaluating how your LLM app responds in those situations, DeepTeam is all you need to uncover vulnerabilities across your AI stack. We've built DeepTeam with modularization in mind - making it easy to build custom LLM security pipelines, combining 20+ single and multi-turn attacks to expose 50+ vulnerabilities across different risk categories. --- Why DeepTeam? 🧩 Modular Architecture …View on GitHub ↗ -
v0.1.9 🎉 New CLI tool, Agentic Red Teaming
Release notes excerpt
🚀 DeepTeam CLI Release We’re excited to release the first version of the DeepTeam CLI – a powerful command-line tool for red teaming and evaluating LLM applications with DeepEval. ✨ Features - Red Team Simulation - Easily specify simulator and evaluation models (gpt-3.5-turbo-0125, gpt-4o, etc.) - Attack LLM systems with predefined vulnerability categories (e.g., Bias, Toxicity) - Target System Configuration - Test both foundational models (like gpt-3.5-turbo) and full LLM applications via custom Python wrappers …View on GitHub ↗ -
v0.1.4 First Stable Release 🎉
Release notes excerpt
DeepTeam v0.1.0 – First Release 🎉 We’re excited to launch the first public release of DeepTeam, the open-source framework for LLM red teaming. 🧠 DeepTeam enables you to simulate real-world attacks on language models, test for failure modes like jailbreaks, and uncover model vulnerabilities using structured, reproducible evaluation. 🚀 Features - ✅ Built-in adversarial attack strategies (jailbreaks, refusal bypasses, prompt injections) - ✅ Automatic generation of adversarial test cases - ✅ Multi-metric evaluation (pass/fail, toxicity, relevance, etc.) …View on GitHub ↗