[damaz91]

Description

Escalates governance approval requirements to at least Level 3 (Technical Council) whenever a Pull Request adds, removes, or modifies normative RFC 2119 clauses (MUST, MUST NOT, SHALL, SHALL NOT, SHOULD, SHOULD NOT, MAY, or prose REQUIRED, RECOMMENDED, OPTIONAL) in a Markdown document (*.md, *.markdown) matched by a governance rule that opts in via normative_escalation: true, even if that rule would otherwise only require Level 1 (devops-maintainers) or Level 2 (maintainers) approval. In ucp-rules.yml, only documentation_spec (docs/specification/**) opts in.

Note

Rebased onto main following the merge of #76 (org-tools/requirements-extractor v0.4.0). normative_detector.py directly reuses requirements_extractor's Markdown parser, RFC 2119 classifier, exclusion filters, and clause digest normalization without duplicating logic, and isolates Markdown dependencies via PEP 723 inline script metadata (pr_validator.py) and uv run --with (tests.yml) so pyproject.toml and uv.lock remain untouched.

Key Changes

• Normative Clause Change Detector (org-tools/governance/scripts/normative_detector.py):
  • Directly delegates Markdown parsing (parser.parse_text), RFC 2119 classification (classifier.classify), exclusion rules (fenced/indented code blocks, inline code spans, HTML blocks/comments, headings, table headers, RFC 2119 boilerplate definitions citing $\ge 4$ keywords, and schema field optionality annotations), compound obligation splitting, and clause digest normalization (identity.normalize, identity.text_digest) to requirements_extractor (feat(org-tools): add the requirements extractor and a reusable catalog check #76).
  • Compares clauses via normalized text and digests so non-semantic formatting changes (link URL retargeting, { target="_blank" } attr_list suffixes, dash swaps, non-keyword emphasis) do not trigger false positives, while preserving code-span tokens (e.g., `dev.ucp.*`) and leading field/status labels on multi-sentence or colon-split list items.
  • Detects unbolded uppercase RFC 2119 keywords in prose by routing report.candidates_without_emphasis candidates through classifier.classify (preserving inline code spans), and tracks list items under normative colon-terminated stems.
• Opt-in Level 3 Escalation (org-tools/governance/scripts/pr_validator.py, governance_config_parser.py, pr_models.py, rules/ucp-rules.yml):
  • Adds an optional per-rule normative_escalation boolean (default false). Only Markdown files matched by an opted-in rule are checked; general docs, README.md, AGENTS.md, .github/**, fallback files, and repositories whose rules files do not opt in are unaffected. ucp-rules.yml enables it only for docs/specification/**.
  • Fetches diff patches and base/head contents only for opted-in Markdown files.
  • Fails closed: if neither the base/head contents nor the diff patch of an opted-in file can be retrieved (e.g., very large files or diffs), the file is escalated with a reason stating that its clauses could not be analyzed. Partially fetched contents (e.g., a failed base fetch on a modified file) are discarded so they are never mistaken for an added or deleted document.
  • Upgrades any matched requirement with level < 3 on a touched Markdown file to Level 3 while preserving min_approvals (requirements already at Level 3 or Level 4 governance-council remain unchanged).
  • Resolves domain-specific file paths (shopping, food, lodging, payment/payments) to the corresponding Domain Tech Council (shopping-tech-council, food-tech-council, lodging-tech-council, payments-tech-council) and cross-domain documents to any Technical Council (Level 3) or higher.
  • Declares markdown-it-py==3.0.0 and markdown==3.10.2 via PEP 723 # /// script metadata in pr_validator.py and --with flags in .github/workflows/tests.yml.
• Governance Validation Report (org-tools/governance/scripts/validation_logger.py):
  • Clearly states why Level 3 approval is required in both the Global PR Summary (* **Reason:** ⚠️ ...) and the File-by-File Breakdown (* **Escalation:** ⚠️ ...), including the affected RFC 2119 keywords and the original rule tier from which the requirement was escalated.
• Documentation & Tests:
  • Updated org-tools/README.md, org-tools/governance/README.md, and org-tools/governance/docs/validation_report.md.
  • Added comprehensive unit tests in test_normative_detector.py, test_pr_validator.py, test_pr_models.py, test_validation_logger.py, and test_governance_config_parser.py (including opt-in scoping and fail-closed behavior).

Category (Required)

Please select one or more categories that apply to this change.

• Core Protocol: Changes to the base communication layer, global context, or breaking refactors. (Requires Technical Council approval)
• Governance/Contributing: Updates to GOVERNANCE.md, CONTRIBUTING.md, or CODEOWNERS. (Requires Governance Council approval)
• Capability: New schemas (Discovery, Cart, etc.) or extensions. (Requires Maintainer approval)
• Documentation: Updates to README, or documentations regarding schema or capabilities. (Requires Maintainer approval)
• Infrastructure: CI/CD, Linters, or build scripts. (Requires DevOps Maintainer approval)
• Maintenance: Version bumps, lockfile updates, or minor bug fixes. (Requires DevOps Maintainer approval)
• SDK: Language-specific SDK updates and releases. (Requires DevOps Maintainer approval)
• Samples / Conformance: Maintaining samples and the conformance suite. (Requires Maintainer approval)
• UCP Schema: Changes to the ucp-schema tool (resolver, linter, validator). (Requires Maintainer approval)
• Community Health (.github): Updates to templates, workflows, or org-level configs. (Requires DevOps Maintainer approval)

Related Issues

Builds on org-tools/requirements-extractor (#76)

Checklist

• I have followed the Contributing Guide (including Conventional Commits title requirements and ! for breaking changes).
• I have updated the documentation (if applicable).
• My changes pass all local linting and formatting checks.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• (For Core/Capability) I have included/updated the relevant JSON schemas.
• I have regenerated Python Pydantic models by running generate_models.sh under python_sdk.