Repository navigation
feat(ai-red-teaming): generalize attack registry + unified white-box support (v1.20.0) - #193
Merged
Merged
Conversation
Adds autodan_attack to the attack_runner registry - AutoDAN (Liu et al. ICLR 2024), a closed-box genetic-algorithm readable jailbreak that runs against hosted dn/ models like any other jailbreak. The other new white-box attacks (GCG family, embedding, activation) are intentionally SDK-only: they need dreadnode[whitebox] + a local open-weight surrogate for gradients, which the hosted inference sandbox does not provide. Paired with dreadnode-tiger SDK PR #2857.
…ry (v1.19.0) Replaces hand-maintained attack wiring with auto-discovery: _discover_attack_defs() introspects dreadnode.airt for standard-signature (goal/target/evaluator_model) *_attack functions and derives module/function/has_attacker/iterations/defaults from each signature. New SDK attacks now flow into the capability with no manual entry. The manual table is kept as tuned overrides (merged on top). White-box gradient attacks are gated out (_WHITEBOX_LOCAL_ONLY) since they need a local surrogate the hosted sandbox can't provide. Verified: 41 attacks auto-discovered from the installed SDK, white-box correctly excluded, manual overrides preserved.
rdheekonda
changed the title
…rrogate skill (v1.20.0) White-box attacks are no longer gated out of discovery; they are included and flagged requires_surrogate so SDK and TUI share one surface. The user supplies a local open-weight surrogate as attacker_model (the SDK already errors clearly if a dn/* model is passed - no gradients over the proxy). Adds the whitebox-local-surrogate skill: how to run them, local-runtime/GPU + dreadnode[whitebox] requirements, trust_remote_code + resource guardrails, and when to prefer closed-box instead.
rdheekonda
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two things, giving unified SDK + TUI support for the full attack set with no hand-wiring:
1. Generalized attack registry.
_discover_attack_defs()introspectsdreadnode.airtfor standard-signature (goal/target/evaluator_model)*_attackfunctions and derivesmodule/function/has_attacker/default_iterations/extra_defaultsfrom each signature._ATTACK_DEFS = {**discovered, **manual}- auto-discovery is the base; the hand-tuned manual table overrides where specific defaults matter. New SDK attacks now flow in automatically (verified: 70 discovered against the SDK).2. Unified white-box support. White-box attacks (GCG family, embedding, generator, activation) are no longer gated out - they're discoverable and flagged
requires_surrogate: True. The user supplies a local open-weight surrogate asattacker_model(the SDK already raises a clear error if adn/*model is passed - no gradients over the proxy, so no silent crash). Newwhitebox-local-surrogateskill teaches the agent how to run them: local-runtime/GPU +dreadnode[whitebox]requirements,trust_remote_code+ resource guardrails, and when to prefer closed-box instead.Non-text attacks (evasion/extraction/inversion/image) are naturally excluded by the signature gate. Graceful fallback: if the SDK import fails, discovery returns
{}and the manual table still works. Capability1.18.8 -> 1.20.0.Verification
requires_surrogate=True; closed-box (autodan/tap)False; defs well-formed (the keys codegen reads). No bugs.Test plan