diff --git a/capabilities/ai-red-teaming/agents/ai-red-teaming-agent.md b/capabilities/ai-red-teaming/agents/ai-red-teaming-agent.md index 1b3af6f4..cb50d7fa 100644 --- a/capabilities/ai-red-teaming/agents/ai-red-teaming-agent.md +++ b/capabilities/ai-red-teaming/agents/ai-red-teaming-agent.md @@ -5,7 +5,7 @@ description: > Unified AI Red Teaming agent for probing security and safety vulnerabilities in LLMs, agentic systems, MCP servers, multi-agent architectures, ML classifiers, and custom AI endpoints. Orchestrates 70+ attack strategies covering traditional ML and generative AI, 590+ transforms, - 140+ scorers, and 260 bundled harm goals across OWASP LLM Top 10, + 140+ scorers, and 279 bundled harm goals across OWASP LLM Top 10, OWASP ASI01-ASI10, and MITRE ATLAS frameworks. --- @@ -379,7 +379,7 @@ Three tiers with sub-categories: | agentic_cascading_failure | Agentic Cascading Failure | 10 | ASI08 | | agentic_trust_exploitation | Agentic Trust Exploitation | 10 | ASI09 | -**Total: 25 sub-categories, 260 bundled goals across 3 tiers.** +**Total: 25 sub-categories, 279 bundled goals across 3 tiers.** ## Agentic Red Teaming diff --git a/capabilities/ai-red-teaming/capability.yaml b/capabilities/ai-red-teaming/capability.yaml index fa77e755..6b89a079 100644 --- a/capabilities/ai-red-teaming/capability.yaml +++ b/capabilities/ai-red-teaming/capability.yaml @@ -11,7 +11,7 @@ description: > and NIST AI RMF compliance frameworks. Also probes traditional black-box ML classifiers: model extraction (model stealing), membership inference (training-data leakage), and model evasion (adversarial examples) across tabular, image, and text. - 590+ transforms, 140+ scorers, and 260 bundled harm goals across 25 + 590+ transforms, 140+ scorers, and 279 bundled harm goals across 25 sub-categories in safety, security, and agentic tiers. agents: diff --git a/capabilities/ai-red-teaming/skills/compliance-mapping/SKILL.md b/capabilities/ai-red-teaming/skills/compliance-mapping/SKILL.md index f1780e62..511d27cf 100644 --- a/capabilities/ai-red-teaming/skills/compliance-mapping/SKILL.md +++ b/capabilities/ai-red-teaming/skills/compliance-mapping/SKILL.md @@ -96,7 +96,7 @@ Maps AIRT attacks, transforms, and scorers to major AI security compliance frame - MCP security — 12 transforms, 7 scorers - Multi-agent security — 10 transforms, 6 scorers - Exfiltration — 8 transforms, 5 scorers -- Content policy — 260 bundled goals across 25 sub-categories +- Content policy — 279 bundled goals across 25 sub-categories - System prompt extraction — 6 transforms - RAG poisoning — 7 transforms diff --git a/capabilities/ai-red-teaming/skills/scorer-reference/SKILL.md b/capabilities/ai-red-teaming/skills/scorer-reference/SKILL.md index f17e8d77..5cdb88d6 100644 --- a/capabilities/ai-red-teaming/skills/scorer-reference/SKILL.md +++ b/capabilities/ai-red-teaming/skills/scorer-reference/SKILL.md @@ -1,12 +1,12 @@ --- name: scorer-reference -description: Complete catalog of all 84 scorers with exact names, categories, and when to use each +description: Catalog of AIRT scorers - 146 user-selectable (+ agentic *_detected scorers auto-wired per OWASP-ASI category); documents the core set with exact names, categories, and when to use each allowed-tools: generate_attack generate_agentic_attack --- # Scorer Reference -All 84 scorers available in the AIRT SDK. Use these EXACT names in the `scorers` parameter. +The AIRT SDK exposes **146 user-selectable scorers** (plus agentic `*_detected` scorers that the agentic suite auto-wires per OWASP-ASI category). The tables below document the core set - use these EXACT names in the `scorers` parameter, or ask `"Show me the available scorers"` for the full list. ## Default Scorer diff --git a/capabilities/ai-red-teaming/skills/transform-reference/SKILL.md b/capabilities/ai-red-teaming/skills/transform-reference/SKILL.md index e931ea13..875ba200 100644 --- a/capabilities/ai-red-teaming/skills/transform-reference/SKILL.md +++ b/capabilities/ai-red-teaming/skills/transform-reference/SKILL.md @@ -1,12 +1,12 @@ --- name: transform-reference -description: Complete catalog of all 183 transforms with exact names, parameters, categories, and LLM requirements +description: Catalog of AIRT transforms - 429 text transforms (596 incl. multimodal/variants); documents the core set with exact names, parameters, categories, and LLM requirements allowed-tools: generate_attack generate_agentic_attack --- # Transform Reference -All 183 transforms available in the AIRT SDK. Use these EXACT names in the `transforms` parameter. +The AIRT SDK resolves **429 text transforms** (596 incl. multimodal + parameterized variants). The tables below document the core set - use these EXACT names in the `transforms` parameter, or ask `"Show me all available transforms"` for the full list. ## Encoding (17 transforms) diff --git a/capabilities/ai-red-teaming/tools/attacks.py b/capabilities/ai-red-teaming/tools/attacks.py index 0308b045..94b4caa3 100644 --- a/capabilities/ai-red-teaming/tools/attacks.py +++ b/capabilities/ai-red-teaming/tools/attacks.py @@ -224,7 +224,7 @@ def generate_category_attack( """Generate a multi-goal attack across bundled harm categories. Sweeps attacks across entire goal sub-categories from the bundled - dataset (260 goals across 25 sub-categories in safety/security/agentic + dataset (279 goals across 25 sub-categories in safety/security/agentic tiers). Goal text is embedded in the generated script — never returned to the agent.