Repository navigation
scripts/triage.py: what is blocking the issue and PR queues, in one command - #820
Merged
Merged
Conversation
A 133-issue backlog and a 32-PR queue turned out to be two different failures wearing the same clothes, and neither was visible without cross-referencing by hand. This does the cross-reference in one command, read-only. Three things it reports, each because it was expensive to learn the first time. Issues referenced by a merged PR, as candidates to PROBE and never as a verdict. Fifteen open issues were already fixed on development; the ones that closed cleanly all had a regression test naming the issue, and the ones that did not were references to a NEIGHBOURING defect. So the tool says "probe these" and refuses to rank them any further. PRs bucketed by mergeable x CI, with what each bucket closes, and the count that carry no Closes line at all -- 25 of 32 today, which is the whole reason a merge stopped resolving anything. And the one that cost ten days: a red PR whose failing tests no longer exist on the base. Four PRs sat red on two tests that development had already renamed, with byte-identical numbers across all four. The tool pulls the failing names out of the CI log and classifies each: phantom (the file is on the base, the test is not -- merge the base and it clears), new (the file came with the PR, so the failure is its own and real), or live. That distinction is not cosmetic -- the first version reported #800's own new test_1103_navier_stokes_velocity_transport.py as a phantom, which would have sent someone to merge the base and find nothing fixed. gh's GraphQL endpoint returns a bare 502 often enough that the calls retry; a tool meant to be run several times a session cannot treat one as fatal. Underworld development team with AI support from Claude Code
lmoresi
added a commit
that referenced
this pull request
Oct 6, 2026
The script is in #820; this is the line that sends a session to it. Underworld development team with AI support from Claude Code
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
CI status handling, failure aggregation, and stale-base handling can produce misleading triage results.
Review effort: Balanced
Findings: 3
Open (4)
What changed in this PR
Adds a queue-triage utility for identifying issue backlog and pull-request blockers.
Changes:
- Summarizes open issues, merged-PR references, and missing closure links.
- Groups pull requests by mergeability and CI state.
- Classifies failed tests as live, new, or renamed on
development.
| File | Description |
|---|---|
scripts/triage.py |
Implements GitHub queue and CI-failure triage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+84
to
+89
| outcomes = [c.get("conclusion") for c in (pr.get("statusCheckRollup") or [])] | ||
| if "FAILURE" in outcomes: | ||
| return "red" | ||
| if "SUCCESS" in outcomes: | ||
| return "green" | ||
| return "none" |
Comment on lines
+98
to
+114
| for check in pr.get("statusCheckRollup") or []: | ||
| if check.get("conclusion") != "FAILURE": | ||
| continue | ||
| url = check.get("detailsUrl") or "" | ||
| job = url.rsplit("/job/", 1)[-1] | ||
| if not job.isdigit(): | ||
| continue | ||
| log = sh("gh", "run", "view", "--log-failed", "--job", job, check=False) | ||
| found = set() | ||
| for line in log.splitlines(): | ||
| # The CI log prefixes each line with the step name and a timestamp. | ||
| hit = FAILED.search(line[line.find("FAILED"):]) if "FAILED" in line else None | ||
| if hit: | ||
| found.add((hit.group(1), hit.group(2))) | ||
| if found: | ||
| return found | ||
| return set() |
| if not body: | ||
| return "new" | ||
| leaf = name.split("::")[-1] | ||
| return "live" if f"def {leaf}" in body else "phantom" |
| fixed and renamed. Their failures named tests that no longer existed. That | ||
| is the ``phantom`` column, and it is the one worth looking at first. | ||
|
|
||
| Read-only: it calls ``gh`` and ``git`` and writes nothing. Run it often; the |
This script reports which issues a PR closes by scanning its body. Its own PR pastes its own output, which contains the line "closes #640, #747, #793" as sample text -- so it reported itself as closing seven issues it has nothing to do with. Fenced code blocks are now stripped before the scan. Whether GitHub's own linked-issue parser makes the same mistake is unresolved: `closingIssuesReferences` comes back empty through `gh` even for a PR whose body plainly says "Closes #793", so that field does not answer it either way. The sample output in the PR description has had its hashes removed so the question cannot arise. Underworld development team with AI support from Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Read-only. Calls
ghandgit, writes nothing.That last line is the whole diagnosis of why merging stopped resolving anything.
The part that cost ten days
#785, #789, #795 and #800 had been red for ten days on two tests
developmenthad already renamed, with byte-identical numbers across all four. Nothing said
so anywhere.
The three-way classification is not cosmetic. The first version of this script
reported #800's own new
test_1103_navier_stokes_velocity_transport.pyas aphantom — which would have sent someone to merge
developmentand find nothingfixed. A file absent from the base arrived with the PR, so its failure is the
PR's own; only a test missing from a file that is still there has been renamed
away.
Why it reports probe candidates and not a ranking
Fifteen open issues turned out to be already fixed. The ones that closed cleanly
all had a regression test naming the issue in its docstring; the ones that did
not were merged PRs referencing a neighbouring defect. A reference is a reason
to look, never a verdict, so the tool says "probe these" and stops.
gh's GraphQL endpoint returns a bare 502 often enough that the calls retry —it did so three times while this output was being captured.
Underworld development team with AI support from Claude Code
The issue numbers in the sample output above have had their hashes removed. They are this tool's report, not a declaration — and a closing verb followed by a hash and a number is exactly the shape GitHub's linked-issue parser looks for, so left as written this PR would have asked to close seven issues it has nothing to do with. Whether a code fence protects you is unresolved:
closingIssuesReferencescomes back empty throughgheven for a PR whose body plainly declares one, so that field does not settle it. The script now strips fenced blocks before its own scan, which is what it should have done from the start — it was reporting itself as the biggest issue-closer in the queue.