Argus Agent Red Team

Attack your AI agent before someone else does with 266 bundled red-team probes over tools, content and requests.

Stars
★ 216
Last updated
today
License
Apache-2.0
Primary language
Python

At a glance

How it runs
CLI
Works with
Universal · cross-platformOpenAI API · Claude API
Cost
Free software; you pay for model usage
Setup effort
Medium · a few setup steps
You'll need
Python 3.11+argus-probe (PyPI)Docker (scan-skill only)ANTHROPIC_API_KEY or OPENAI_API_KEY (for judged probes)Playwright (browser probes)Shell / CLINetwork accessLocal filesystemMCP Server
Typical use
A security team screening a chatbot or tool-using agent before release and uploading the SARIF into GitHub Code Scanning.
Not a fit if
  • Teams that want every verdict decided without any cloud LLM judge
  • Users validating macOS-only skill behaviour (scan-skill runs in a Linux container)
  • Teams wanting an inline runtime guard for production traffic
Maintenance
Commits in the last 30 dayslast commit 2026-11-20latest release v1.19.0 (2026-11-19)+1.0 ★/day over 7 days
Source review
74/100 · Some gaps

What does this agent do, and when should you use it?

Argus is an open-source red-team testing framework for AI agents, distributed on PyPI as argus-probe, with a bundled library of 266 adversarial probes mapped to OWASP LLM Top 10, MITRE ATLAS and NIST AI RMF. It treats the target agent as a black box and reaches it through 15 target adapters such as openai_compat, anthropic_native, mcp_agent, browser_use, a2a_agent and vllm_server. Verdicts come either from an LLM judge with per-probe rubrics or from deterministic checks on recorded browser requests, MCP tool calls and mock x402 payments. Beyond the probe library it ships three focused subcommands: mcp-trap for running your agent against a hostile MCP server, scan-mcp for auditing an MCP server's tools, and scan-skill for sandboxing a Claude Code skill before installation. Findings are emitted as SARIF 2.1.0, JUnit XML or HTML, ready for GitHub Code Scanning or a CI gate via the argus-probe-action GitHub Action. Local mode runs everything in-process, so the CLI does not need a self-hosted server.

Argus loads YAML probes from argus/probes/, each with prompts, an optional conversation flag, severity, mappings (atlas, owasp_llm, owasp_agentic, nist_ai_rmf, eu_ai_act) and a judge.rubric_path. At run time it picks an adapter based on the target JSON's kind, sends the attack prompts, and judges replies against rubrics in argus/rubrics/ using a fast model (Anthropic Haiku 4.5 by default) with a strong-model re-judge (Sonnet 4.6) when the first pass reports high/critical with confidence ≥ 0.7; OpenAI, litellm-backed providers and TypeSafe Jev are also supported. The mcp_agent probes, most payment probes and several garak wrappers skip the judge and assert against recorded MCP interactions or payment testbed results. Reports come out as SARIF 2.1.0, JUnit XML or HTML with per-standard coverage and, for probes with captured evidence, an evidence section next to the verdict. Companion commands include mcp-trap (hostile MCP server scenarios), scan-mcp (read-only tool calls against an MCP server), scan-skill (runs a Claude Code skill in a throwaway container and records file reads, commands and outbound requests), and draft-target (drafts a target spec from a local OpenAPI file).

  1. A security team screening a chatbot or tool-using agent before release and uploading the SARIF into GitHub Code Scanning.
  2. Developers building on MCP who want to check whether their agent falls for tool poisoning, rug pulls, tool shadowing or credential lures with mcp-trap.
  3. Payment-agent teams using the payment_agent adapter to see whether the agent pays a swapped payee, splits payments under an approval limit or pays twice on retry.
  4. Anyone about to install a third-party Claude Code skill who wants scan-skill to observe decoy-secret reads, suspicious commands and outbound requests in a sandbox.
  5. DevSecOps teams wiring argus-probe-action into CI so every commit triggers a local scan and fails the job on critical findings or errored probes.
  6. Security researchers who need findings classified against OWASP LLM Top 10, MITRE ATLAS and NIST AI RMF for audit records.

Key setup facts

Pulled from the install and usage notes below. The project README has the current values.

Install / run
  • pipx install argus-probe
  • pip install argus-probe
  • pip install
Environment variables
ANTHROPIC_API_KEY OPENAI_API_KEY AGENT_API_KEY ARGUS_API_URL ARGUS_API_TOKEN
Config files
argus-target.json
Runtime
Python 3.11+
Install notes cover
Docker

How do you install or deploy this agent?

Requires Python 3.11+. Prefer pipx to avoid clashing with the unrelated argus package on PyPI:

pipx install argus-probe
# or
pip install argus-probe

No model key is needed for deterministic probes; set an Anthropic or OpenAI key to run judged probes:

export ANTHROPIC_API_KEY=<your key>
# or
export OPENAI_API_KEY=<your key>

scan-skill additionally needs Docker and ANTHROPIC_API_KEY, and browser probes need Playwright with Chromium. To install from source, clone the repo and run pip install ..

How do you use this agent?

One-minute trial with no API key, using a bundled vulnerable MCP demo agent:

argus-probe quickstart

This writes argus-quickstart.html. For your own agent, write a target JSON such as:

{
  "kind": "openai_compat",
  "endpoint_url": "https://api.your-agent.example.com/v1/chat/completions",
  "model": "your-agent-prod-v3",
  "api_key": "${AGENT_API_KEY}"
}

Run selected probes in-process and emit SARIF:

argus-probe run --local --target argus-target.json \
  --probes owasp_01_prompt_injection_basic,syscard_crescendo \
  --format sarif --out argus.sarif

List probes that fit the target kind:

argus-probe list-probes --target argus-target.json

Point your agent at a hostile MCP server:

argus-probe mcp-trap

Scan a skill before installing it:

argus-probe scan-skill ./path/to/skill

Wire it into GitHub Actions with the official Action and a fail-on threshold:

- uses: actions/checkout@v4
- uses: gy15901580825/argus-probe-action@v2
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
  with:
    target: argus-target.json
    fail-on: critical

Without --local the CLI is a client of a server implementing /api/v1/redteam and needs ARGUS_API_URL and ARGUS_API_TOKEN; the full self-hosted stack lives at the v0.3-full tag.

What are this agent's strengths and limitations?

Pros
  • 266 bundled probes with OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF and OWASP Agentic Top 10 mappings, so findings trace back to specific framework entries.
  • MCP, payment and browser probes prefer deterministic verdicts from recorded calls and requests rather than a model reading the reply, reducing judging noise.
  • Local --local mode needs no server, and quickstart runs end-to-end without an API key, so first contact is cheap.
  • Native SARIF 2.1.0 and JUnit XML output plus a GitHub Action with Code Scanning upload fit naturally into devsecops pipelines.
  • Three focused subcommands, mcp-trap, scan-mcp and scan-skill, cover MCP agents, MCP servers and Claude Code skills respectively.
Limitations
  • Judged probes depend on cloud models and incur spend; daily and per-run caps exist but you still need Anthropic, OpenAI or a litellm-compatible key.
  • scan-skill runs only inside a throwaway Linux container, so macOS-only skills do nothing there, and a clean report is not proof of safety.
  • The full self-hosted server stack (REST API, web UI, database migrations, Helm charts) has been removed from the current repo and is only available at the v0.3-full tag.
  • mcp_agent probes default to a local testbed and need extra configuration for real agents, and scanning a stdio MCP server requires --on-host.
  • An unrelated argus package on PyPI installs the same top-level name, so pipx or a dedicated virtualenv is advised.

How does this agent compare with similar options?

The README names NVIDIA's garak directly: Argus bundles 99 wrappers restating garak probe prompts, and garak is not a dependency. It also recommends running NVIDIA SkillSpector or Cisco skill-scanner alongside scan-skill for static review of skill text.

Key facts side by side with the most closely related agents.

Agent Source review Form / cost Stars Updated Language Full support on
Argus Agent Red Team This agent 74 · Some gaps CLIFree + model costs ★ 216 today Python OpenAI API · Claude API
Agent Audit 60 · Some gaps CLIFree ★ 246 2mo ago Python —
Superagent SDK 46 · Major gaps Library / SDKFreemium ★ 6.9k 2mo ago TypeScript Claude Code · OpenAI API · Claude API
SkillSpector Security Scanner 75 · Good CLIFree + model costs ★ 27k today Python Codex · Claude Code · OpenAI API · Claude API

How does FollowAgents rate this agent?

FollowAgents source review · FARS-2.1
Some gaps
74/ 100 5-point scale 3.7 / 5
This review covers the source as of Nov 11, 2026. The project has shipped 14 stable release(s) since, which have not been re-reviewed. The score and verdict describe the reviewed version only. Recent releases ↓
Trust 21/29
Reliability 9/14
Adaptability 15/18
Convention 14/18
Effectiveness 10/13
Verifiability 5/8
Why each dimension lost points
Trust21 / 29 · 3.6/5

Evidence: SECURITY.md states credentials are read only from the operator's environment and used only against the operator's own target, and treats credential leakage as a vulnerability; README discloses that a cloud judge sends attack prompts and target replies to a third party, with cost caps and predictive abort; pyproject pins litellm to a narrow reviewed range with a documented 2026-10-21 malicious-release incident, plus CVE floors for jinja2/filelock — solid dependency security. Deductions: scan-skill installs and executes untrusted skills in a throwaway container and scan-mcp calls tools by default (stdio needs --on-host); these external effects are documented but lack a unified confirmation/rollback path. No rollback or undo guidance exists beyond checking out the v0.3-full tag, so rollback scores 1. Attribution names garak and the Semia paper but no NOTICE content or per-item provenance list is shown, so 2.

Reliability9 / 14 · 3.2/5

Evidence: CLAUDE.md documents an import-linter boundary contract; CI covers tests, install, each testbed, image builds and probe schema validation; failure semantics are concrete in README (401/timeout becomes error not pass, empty reply is error, unreachable registry is error). Deductions: README and quickstart are in tension about server mode (the repo no longer ships a server; use the v0.3-full tag), and client_agent must match the server release or every browser probe errors, showing brittle version coupling; degradation paths for optional deps (mcp, playwright, Docker) are documented but not evidenced in code, so no criterion earns full marks.

Adaptability15 / 18 · 4.2/5

Evidence: the README Scope section enumerates 15 target adapters and per-category counts of 266 probes, and has an explicit 'What Argus does not do' list (no model training, no patching, not a production runtime gate), so capability boundaries are well handled; audience and scenarios (security teams, CI, payment/MCP/browser agents) are broad. Deductions: trigger precision relies on probe names matching target kind, the README admits judge misjudgment (small models miss jailbreaks, Azure content filters block judge calls) and deep-mode traffic is not fully predictable; environment fit requires the target to reach 127.0.0.1 or a self-deployed hosted trap for browser probes, with extra config for containers/remote hosts.

Convention14 / 18 · 3.9/5

Evidence: install notes are thorough (pipx/pip, Python 3.11+, warning about the unrelated PyPI 'argus' package, optional extras); examples and FAQ are rich (quickstart, target JSON, GitHub Action YAML, scan-skill samples); known limitations are unusually specific (a clean scan is not proof of safety, macOS-only skills do nothing in the Linux container, judge-accuracy doc); LICENSE is full Apache-2.0 with license-files declared. Deductions: information architecture is strained — the README is very long and duplicates docs/, and CLAUDE.md and README disagree on server availability; naming is confusable (argus-probe dist vs top-level argus package vs separate argus-probe-action repo); versioning has only a link to GitHub releases with no in-repo CHANGELOG; maintenance responsibility scores 1 because the publisher is unverified and only a contact email is given.

Effectiveness10 / 13 · 3.8/5

Evidence: output usability is strong — SARIF 2.1.0 uploads to GitHub Code Scanning, JUnit gates CI, HTML for humans, with per-standard coverage and evidence sections; the CI workflow shows the fail-on gate tripping with exit code 3. Deductions: marginal value is partly overlapping — 99 of 266 probes are garak wrappers and many verdicts come from an LLM judge, so it overlaps existing garak/LLM-eval tooling, and the README itself says it does not replace an eval harness; cost-benefit is bounded but not cheap (--probes all estimated ~$5.32 for a chat target, scan-skill caps of $1–4.75 per task) and depends on a third-party judge API.

Verifiability5 / 8 · 3.1/5

Evidence: claim traceability is decent — the probe mapping table is generated rather than hand-maintained, SECURITY.md records the exact litellm review method and what it did not verify (publisher identity, no PEP 740 attestations), and RESULTS.md and judge-accuracy.md provide measured data. Deductions: cross-source corroboration requires the reader to sum the Scope category counts to reach 266, and the 12 quickstart probes only indirectly match the 12 SARIF results asserted in CI; fact/inference separation is muddied because the README mixes shipped, in-engine-but-unexposed, and no-longer-shipped states, leaving the reader to sort fact from plan.

Risks and how to mitigate them
  • scan-skill installs and executes untrusted skills in a container, and scan-mcp calls target tools by default (stdio needs --on-host); obtain written authorization before running against production or third-party systems.
  • A cloud judge sends attack prompts and target replies to a third-party LLM provider; do not scan systems whose replies contain data you may not send out.
  • The repo no longer ships the server stack; server mode requires deploying the v0.3-full tag yourself, and client_agent must match the server release or every browser probe errors.
  • The publisher is not verified by the FollowAgents registry; maintenance responsibility and update path rest on a single contact email, so assess supply chain before production adoption.
  • litellm is pinned to 1.108.3–1.108.4, and SECURITY.md itself says publisher identity and PEP 740 attestations were not verified, so dependency safety depends on continued manual review.
Evidence confidence: Low Reviewed Nov 11, 2026 Reviewed revision 9faea47f0eef New commits since this review; the score may not cover them
See the full review method →
Maintain this project? Add the FARS badge to your README

The badge shows the current review score and verdict and links back to this page. It refreshes about once an hour after a re-review.

FARS

[![FARS review](https://followagents.com/badge/argus.svg)](https://followagents.com/en/agents/argus)
<a href="https://followagents.com/en/agents/argus"><img src="https://followagents.com/badge/argus.svg" alt="FARS review"></a>

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

  1. v1.19.0 argus-probe 1.19.0
    Release notes excerpt
    argus-probe 1.19.0 and argus-drivers-google 1.19.0. 320 probes, up from 313. Added: MCP gateway and proxy testing (#344-#361) A new mcp_gateway target tests the gateway or proxy that sits between MCP clients and upstream MCP servers. Argus hosts passive upstream fixtures: - a hostile upstream MCP server; - an honest authorization server; - a foreign origin. …
    View on GitHub ↗
  2. v1.18.0 argus-probe 1.18.0
    Release notes excerpt
    argus-probe 1.18.0 and argus-drivers-google 1.18.0. 313 probes (was 295). Eighteen new probes cover attack scenarios from 2025-2026 security research. Each one is scored from what a trap, testbed or tool ledger records, with no LLM judge. Each one has a control that an honest agent passes, and is skipped when the agent never reached the attack. Added: browser probes without instruction text …
    View on GitHub ↗
  3. v1.17.1 argus-probe 1.17.1
    Release notes excerpt
    argus-probe 1.17.1 and argus-drivers-google 1.17.1. 295 probes (unchanged). Fixed: Gmail seed insertion (#330) - With seed_mode: insert, the Google driver posted the seed message to a path the Gmail API does not have, so every run failed with HTTP 404 and the finding became error ("scenario dependency failed"). It now posts to users/me/messages, the documented insert endpoint. The request body and internalDateSource=receivedTime are unchanged. …
    View on GitHub ↗

FAQ

Do I need a model API key to run anything?
Not for everything. mcp_agent probes, most payment probes and many garak wrappers use deterministic assertions, but judged probes need a key for Anthropic, OpenAI, a litellm-compatible endpoint or Jev. Without one, run --local refuses to start unless you pass --skip-judged.
What does a run cost?
Static runs default to a $0.50 per-run cap and deep runs to $5, tunable with --per-run-cap plus a daily cap. scan-skill has a --per-task-cap, typically $1 to $4.75 depending on model, and a clean Sonnet 4.6 scan costs about $0.30.
Does Argus patch or protect my agent?
No. It attacks the agent as a black box through its own interface. It does not modify the target or act as a production runtime gate; the bundled PromptGuard is only for before/after defense experiments.
Where does my data go?
With a cloud judge, attack prompts and your agent's replies are sent to that provider. Quickstart, the local demo and deterministic probes make no external model calls. scan-skill's sandbox proxy holds the real key and forwards calls to Anthropic.
Can it run in GitHub Actions?
Yes. The official action gy15901580825/argus-probe-action@v2 installs the CLI, runs it in local mode, uploads SARIF, and fails the job when a finding reaches fail-on or a probe errors. argus-probe init --workflow writes a complete workflow file.
Is Argus Agent Red Team open source?
Yes. The source code is on GitHub (gy15901580825/Argus) under the Apache-2.0 licence.
Is Argus Agent Red Team safe to use?
FollowAgents' FARS review (Nov 11, 2026) scored Argus Agent Red Team 21/29 on trust and safety. None of the eight safety controls the review checks for was missing outright. The repository has changed since that review.
Is Argus Agent Red Team still maintained?
Yes. The latest commit on GitHub is from 2026-11-20, within the last 30 days. The latest release is v1.19.0 (2026-11-19).
Who makes Argus Agent Red Team?
Argus Agent Red Team is published on GitHub by the account gy15901580825. FollowAgents has not verified which person or company is behind that account.
What is Argus Agent Red Team's official website?
The project's GitHub repository lists https://gy15901580825.github.io/Argus/ as its website. That link is set by the repository owner; the source code itself is on GitHub (gy15901580825/Argus).
Was this page helpful?
View on GitHub ↗ Install ↓

Compare agents like this one

The same FARS review applied across the shortlist this agent qualifies for.

Related concepts