Skip to content

FEAT: Extract attack factory constructor compatibility rules (#2764) - #2795

Merged
Roman Lutz (romanlutz) merged 8 commits into
microsoft:mainfrom
Kaap10:feature/extract-attack-factory-rules-2764
Sep 28, 2026
Merged

Roman Lutz (romanlutz) merged 8 commits into
microsoft:mainfrom
Kaap10:feature/extract-attack-factory-rules-2764

Conversation

@Kaap10

Copy link
Copy Markdown
Contributor

Description

This PR extracts constructor-contract inspection and scoring configuration compatibility logic out of AttackTechniqueFactory into a dedicated, private helper class _ConstructorCompatibilityHelper located in pyrit.scenario.core._attack_constructor_compatibility.

Key changes:

  • Decouples constructor parameter reflection and type introspection from factory orchestration.
  • Caches discovered constructor parameters (accepted_params) and scoring config type requirements (scoring_config_type) once at helper instantiation time.
  • Migrates the 5 internal methods (_get_accepted_params, _should_apply_scoring_config, _apply_scorer_policy, _get_scoring_config_type, _unwrap_optional) into _ConstructorCompatibilityHelper.
  • Re-exports ScorerOverridePolicy from attack_technique_factory.py to maintain 100% public API backwards compatibility.
  • Updates converter-composition validation, keyword argument checks, and converter-appending capability checks to leverage the helper.

Closes #2764

Tests and Documentation

  • Added unit tests in tests/unit/scenario/core/test_attack_constructor_compatibility.py exercising _ConstructorCompatibilityHelper directly across parameter discovery, optional type unwrapping, and ScorerOverridePolicy behaviors (SKIP, WARN, RAISE).
  • Maintained factory integration and creation tests in tests/unit/scenario/core/test_attack_technique_factory.py.
  • Ran full pre-commit checks locally (uv run pre-commit run --all-files), all 17 hooks passed.
  • Ran static type analysis (uv run -m ty check pyrit/ tests/unit/scenario/core/), 0 errors.
  • Ran full scenario unit test suite (uv run -m pytest tests/unit/scenario/), all 1,142 tests passed.
  • Verified test coverage: 100% statement coverage on _attack_constructor_compatibility.py.

Comment thread pyrit/scenario/core/_attack_constructor_compatibility.py Outdated
@Kaap10

Copy link
Copy Markdown
Contributor Author

Thanks for catching this!

I have updated _ConstructorCompatibilityHelper to resolve scoring_config_type dynamically on demand via a @property instead of eagerly evaluating and permanently storing None in __init__. This ensures that forward-referenced config annotations defined after factory construction (or during module loading) resolve correctly at create() time.

Added unit and regression tests:

  • In test_attack_constructor_compatibility.py: verified deferred annotation resolution once the forward-referenced class is defined.
  • In test_attack_technique_factory.py: added regression tests for factory.create() with forward-referenced config types defined after factory construction across RAISE, WARN, and SKIP policies.

Preserve the benchmark prompt-prefix copy helper and use the extracted constructor contract for its capability check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ack-factory-rules-2764

# Conflicts:
#	pyrit/scenario/core/attack_technique_factory.py
Integrate the author branch at b3ecf22, including main at 5503ecb, while preserving the earlier local conflict resolution.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Declare deliberately late-bound configuration names only for static analysis, preserving the runtime forward-reference regression coverage without suppressing type errors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Kaap10

Copy link
Copy Markdown
Contributor Author

Hi Roman Lutz (@romanlutz), I updated the branch with the latest main. Could you please approve the workflow run when you have a moment? Thanks!

@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Sep 28, 2026
Merged via the queue into microsoft:main with commit 9c26cd7 Sep 28, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MAINT Extract attack factory constructor compatibility rules

2 participants