Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/pr-risk.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,9 @@ name: PR Risk Grade (reusable)
# last-writer-wins with still EXACTLY ONE `risk:*` label — possibly the staler tier, which gates
# nothing meanwhile and which the next grade re-syncs (on the LAST push there is no next grade, so
# re-dispatch on `pr_number` if a final grade looks wrong). It can no longer leave two contradictory
# labels on a PR under one `label_map`; remapping `label_map` orphans the old names, which is a
# one-time repo-side cleanup. What the shape does cost is a narrower residual: the PUT is built from
# labels on a PR under one `label_map`; the first remapped grade retires the default

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit — "the first remapped grade retires the default risk:R0..risk:R3 and risk:ungraded labels" overstates two ways: a remap that keeps a default name as a live target (the 1-indexed scheme documented in this same file keeps unknown=risk:ungraded) does not retire that name, and retirement happens per-PR on that PR's next grade, not globally on the first remapped grade.
Raised by 1 of 6 reviewers (kimi-k3-high edge-case).

# `risk:R0`..`risk:R3` and `risk:ungraded` labels. Custom names from an older map still need cleanup.
# What the shape does cost is a narrower residual: the PUT is built from
# a snapshot read, so a NON-owned label added by someone else in the read→PUT window is dropped
# (`risk-dispute` included) and one removed in it is resurrected. That window opens only on a run
# that actually changes the grade and is roughly one API round-trip — about three on the first
Expand Down
6 changes: 6 additions & 0 deletions docs/callers/pr-risk.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,12 @@ fail the caller's next run at startup.
| `repo_map_path` | `.github/risk.json` | Consumer risk-map override, read from the PR **base ref**. |
| `repo_runbooks_path` | `.github/risk-runbooks.json` | Consumer runbook-registry override, read from the PR **base ref**. |

To show `risk:low` through `risk:xhigh`, pass this `label_map` in the caller:
`R0=risk:low,R1=risk:medium,R2=risk:high,R3=risk:xhigh,unknown=risk:unknown`.
The next grade removes a default `risk:R0`..`risk:R3` or `risk:ungraded` label

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — "The next grade removes a default ... label" reads as one-time migration cleanup, but the code makes those five names grader-owned permanently: after adopting a custom map a caller can never keep risk:R0..risk:R3 or risk:ungraded on a PR for any other purpose, since every grade strips them again. Say that outright here, and in the same-understated wording at .github/workflows/pr-risk.yml:117 and scripts/pr-risk/README.md:201.
Raised by 2 of 6 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

and leaves only the mapped label. Custom names from an older map still need
one-time cleanup.

## Gotchas

**Fork PRs need `pull_request_target`, not `pull_request`.** A fork PR under a
Expand Down
8 changes: 3 additions & 5 deletions scripts/pr-risk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,11 +198,9 @@ Operational caveats for a backfill:
invisible: GitHub records it on the PR timeline as an `unlabeled` event by the
grader token. Dispatch when the queue is quiet, and use `pr_number` when you
want the per-PR group to serialize a re-grade against event runs.
- **Remapping `label_map` orphans the old names.** Ownership is defined by the
*current* map, so labels applied under a previous one are no longer owned:
they ride through every future PUT beside the new target and no re-grade will
clear them. Delete the retired label names repo-side once, as part of the
remap.
- **Remapping retires the default `risk:R0`..`risk:R3` and `risk:ungraded`
labels.** The first re-grade removes the old default and leaves one mapped
label. Custom names from an older map still need one-time repo-side cleanup.
- **The pre-grader reads retry.** Rate limits are global, not per-PR, so the
base-ref and override reads — the first hop for every target — retry a
transient failure with backoff, as the grader already does. Without it one
Expand Down
9 changes: 4 additions & 5 deletions scripts/pr-risk/apply-risk-label.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,9 @@
# before-snapshot and the after-read, so the diff that would have to catch it is empty by
# construction.
#
# ONE MORE LIMIT, on a different axis: ownership is defined by the CURRENT LABEL_MAP. Change a
# caller's `label_map` and labels applied under the old map are, by definition, no longer owned —
# they are carried through every future PUT beside the new target, and nothing here will clean them
# up. Remapping is a one-time repo-side cleanup (delete the retired label names), not something a
# re-grade heals.
# ONE MORE LIMIT, on a different axis: ownership is defined by the CURRENT LABEL_MAP plus the five

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — This rewritten paragraph contradicts the OWNERSHIP CONTRACT at the top of the same file (line 5 still says the script owns "EXACTLY the label names in LABEL_MAP's values", with non-owned labels carried through untouched). That contract is the stated safety justification for the destructive whole-set PUT, so line 5 needs the same "plus the five default names" qualifier or the next maintainer reasons about this write from an invariant that no longer holds.
Raised by 3 of 6 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-high edge-case).

# default labels. A remap retires those defaults on the first re-grade. Custom names from an
# older map are unknowable and still need one-time repo-side cleanup.
#
# The label is applied with the plain GITHUB_TOKEN on purpose: GITHUB_TOKEN-applied labels do
# not fire `labeled` workflow triggers, which makes the shadow check incapable of starting a
Expand Down Expand Up @@ -107,6 +105,7 @@ for t in R0 R1 R2 R3 unknown; do
[ -n "$l" ] || die "LABEL_MAP maps tier '$t' to an empty label"
OWNED+=("$l")
done
OWNED+=("risk:R0" "risk:R1" "risk:R2" "risk:R3" "risk:ungraded")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — The five default names are appended to OWNED unconditionally and permanently, so in a repo enrolled with a custom label_map a risk:R0..risk:R3/risk:ungraded label this grader never applied — a human triage marker, an org escalation convention, a parallel/legacy labeler — is now silently deleted by the full-set PUT, which breaks the script's own promise that a human's own label is never fought. There is no caller opt-out, and if another actor re-adds one the PR can never satisfy the in-sync short-circuit, so every subsequent run flaps and re-opens the destructive read→PUT window the header claims is confined to grade-changing runs. Consider gating retirement behind an explicit one-time migration input instead of claiming unrelated labels by name.
Raised by 5 of 6 reviewers (claude-opus-5-thinking-max adversarial, gpt-5.6-sol-max adversarial, kimi-k3-high adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-high edge-case).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — This list hardcodes the values already held in DEFAULT_MAP (line 79), so the two can silently desync: rename a default (say unknownrisk:unlabeled) and the new name is never retired while the stale one stays permanently owned in every repo, with nothing failing to signal it. Deriving the list from DEFAULT_MAP via the existing label_for logic keeps them identical by construction.
Raised by 2 of 6 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit — The append does not dedup, so under the default map — the common case, where these names are already in OWNED from the map — each is held twice. Set semantics downstream are unaffected, but the in-sync loop spawns up to ten has() jq subprocesses per PR instead of five and the DRY_RUN log prints a visibly duplicated owned set.
Raised by 1 of 6 reviewers (claude-opus-5-thinking-max edge-case).

TARGET="$(label_for "$TIER")"

# Colors keyed by TIER (not label text, which callers may remap): green .. red, gray unknown.
Expand Down
11 changes: 11 additions & 0 deletions scripts/pr-risk/tests/test_apply_risk_label.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ echo "— caller remap (a 1-indexed R1..R4 scheme is one input) —"
MAP='R0=risk:R1,R1=risk:R2,R2=risk:R3,R3=risk:R4,unknown=risk:ungraded'
eq "R0 remaps to risk:R1" "risk:R1" "$(run R0 "$MAP")"
eq "R3 remaps to risk:R4" "risk:R4" "$(run R3 "$MAP")"
NAMED_MAP='R0=risk:low,R1=risk:medium,R2=risk:high,R3=risk:xhigh,unknown=risk:unknown'
eq "unknown remaps to risk:unknown" "risk:unknown" "$(run unknown "$NAMED_MAP")"

echo "— validation refuses bad input before any write —"
run R7 >/dev/null 2>&1; eq "bad tier exits 2" 2 "$?"
Expand Down Expand Up @@ -203,6 +205,15 @@ eq "the extra owned label is squashed down to the one target" \
"api -X PUT repos/test/repo/issues/7/labels -f labels[]=keep-me -f labels[]=risk:R2" \
"$putheal"

echo "— a remap retires the default grade label instead of keeping both —"
: > "$GH_LOG"; printf 'risk:R2\nrisk:ungraded\nkeep-me\n' > "$CURRENT_LABELS"
PATH="$SANDBOX/bin:$PATH" REPO=test/repo PR_NUMBER=7 TIER=R2 LABEL_MAP="$NAMED_MAP" \
bash "$SCRIPT" >/dev/null 2>&1
putremap="$(grep -- '-X PUT repos/test/repo/issues/7/labels ' "$GH_LOG")"
eq "the old defaults are replaced by the mapped label" \
"api -X PUT repos/test/repo/issues/7/labels -f labels[]=keep-me -f labels[]=risk:high" \
"$putremap"

# Already-correct label: no write at all beyond the read.
: > "$GH_LOG"; printf 'risk:R2\nkeep-me\n' > "$CURRENT_LABELS"
outsync="$(PATH="$SANDBOX/bin:$PATH" REPO=test/repo PR_NUMBER=7 TIER=R2 bash "$SCRIPT" 2>/dev/null)"
Expand Down
Loading