Skip to content

[Issue #149] Deterministic tier floors replace the size-only QA trigger (v2.1.3 P2) - #151

Merged
KHAEntertainment merged 3 commits into
mainfrom
feat/149-risk-tiers
Oct 25, 2026
Merged

KHAEntertainment merged 3 commits into
mainfrom
feat/149-risk-tiers

Conversation

@KHAEntertainment

Copy link
Copy Markdown
Owner

Closes #149
Closes #107

Rationale

Phase 3.5 decided QA by diff size (lines, file count, interface/schema/auth change) while the Phase 0 router listed 3.5 for some paths and not others (#107). This PR replaces the size trigger with deterministic risk-tier floors, stated once in phases/phase3.5.md (Risk Tiers). The router marks every path 3.5 (per risk tier) and mirrors each tier's Phase 3.5 value; scripts/validate_skill.py now fails when the two disagree. Emergency Hotfix is not exempt; the tier rule is the single rule for every path.

T1 interim: until P3 lands, T1 keeps today's lanes (QA + independent review), so QA is no longer size-gated and small T1 changes get QA. No lane is removed. P3 makes T1 QA conditional on a declared acceptance check. T2/T3 protections are unchanged.

Acceptance criteria

  • AC-1 docs-only diff outside behavior paths → T0, Phase 3.5 skip, no reviewer or acceptance lane: ✓ (tier table; pinned by test_tier_table_states_each_floor_once)
  • AC-2 change under a declared T2 path → T2, run, current T2/T3 protections unchanged: ✓ (T2 floor = plan-approval list; showrunner T2 paths declared in PROJECT_CONTEXT.md)
  • AC-3 router and trigger state one rule; validator fails on disagreement: ✓ (check_risk_tiers; 14 mutation cases plus a CLI exit-code test)
  • AC-4 size trigger removed, pins replaced: ✓ (the old trigger had no validator/test pins; assertNotIn pins for every retired threshold in phase3.5.md, SKILL.md and README, plus validator RETIRED_TRIGGER_TOKENS)
  • AC-5 qa-agent.md step 6 names only step-10 evaluator types: ✓ (test_qa_step_6_names_only_step_10_evaluator_types)
  • SOP inconsistency: the Phase 0 router omits Phase 3.5 for small changes, but the QA trigger has no path exemption #107: README path table updated in the same PR (3 lines, trigger/hotfix statements only): ✓

Focused checks (owner directive; CI Verification Gate owns the rest)

$ python3 scripts/validate_skill.py
OK: validated .../skills/dev                                  exit 0
$ python3 -m unittest tests.test_backend_contract
Ran 123 tests ... OK                                          exit 0
$ python3 -m unittest tests.test_validate_skill
Ran 50 tests ... OK                                           exit 0
$ python3 -m py_compile scripts/validate_skill.py tests/test_backend_contract.py   exit 0

scripts/verify.sh, the full suite and installer suites were not run locally.

Coverage-path audit

Router paths (6): New Project, New Feature, Small Change, Hotfix, Architectural Change, Refactoring: each states Phase 3.5 once as 3.5 (per risk tier) (validator-enforced). Tier rows T0-T3 in both documents (validator-enforced for name, value and pinned policy). Referrers of the old trigger: SKILL.md core principle, README lines 51/86/90 (updated); CHANGELOG.md line 52 is a historical entry (left).

Caller impact

Documentation and validator only. New check_risk_tiers runs in main() of validate_skill.py; no CLI change. PROJECT_CONTEXT.md gains a ## Risk Tiers section; the template gains an optional one. Rollback: revert the PR.

🤖 Generated with Claude Code

KHAEntertainment and others added 2 commits October 24, 2026 21:21
… tiers

Phase 3.5 now states one rule: tier floors T0-T3, computed from the diff's
paths. The Phase 0 router marks every path `3.5 (per risk tier)` and mirrors
each tier's Phase 3.5 value, which resolves the router/trigger disagreement
in #107 (no hotfix exemption). phase3.md names its plan-approval list as the
T2 floor, the project-context template and PROJECT_CONTEXT.md declare behavior
and T2/T3 paths, qa-agent step 6 no longer names code analysis as an
evaluator, and README matches the tier rule.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
check_risk_tiers compares the Phase 0 tier mirror and every router path with
the Phase 3.5 tier table and the pinned policy, and rejects the retired size
trigger. Tier pins replace the size-trigger pins; mutation tests prove each
drift fails.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e8622f55-1bf8-4884-8e45-530828258f2a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…er decides

Line 51 still described the retired 80-point pass gate. It now matches the
P4 outcome model, and the README retired-token pin covers the old wording.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
@kilo-code-bot

kilo-code-bot Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (9 files)
  • PROJECT_CONTEXT.md - 0 issues
  • README.md - 0 issues
  • scripts/validate_skill.py - 0 issues
  • skills/dev/SKILL.md - 0 issues
  • skills/dev/agents/qa-agent.md - 0 issues
  • skills/dev/phases/phase3.5.md - 0 issues
  • skills/dev/phases/phase3.md - 0 issues
  • skills/dev/templates/PROJECT_CONTEXT_TEMPLATE.md - 0 issues
  • tests/test_backend_contract.py - 0 issues

Reviewed by nemotron-3.5-super-vl:free · Input: 198.1K · Output: 30.4K · Cached: 260.2K

Review guidance: REVIEW.md from base branch main

@KHAEntertainment
KHAEntertainment merged commit 5f01a9d into main Oct 25, 2026
8 checks passed
@KHAEntertainment
KHAEntertainment deleted the feat/149-risk-tiers branch October 25, 2026 04:31
KHAEntertainment added a commit that referenced this pull request Oct 25, 2026
Keep both sides: check_risk_tiers and RiskTierTests from P2, and
check_bounded_remediation and BoundedRemediationTests from P5.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant