Skip to content

Phase 1 triage never checks whether the default branch's own CI is green #144

Description

@dmccoystephenson

What is missing

Phase 1 triage inspects open PRs, open issues and git log, but never asks whether the default branch's own CI is currently green. In the generated instance where this was found (medieval-factions-dev-loop, against Dans-Plugins/Medieval-Factions), main had been failing its Build check for five days, since 2026-08-22, and no prior cycle had noticed.

The breakage was discovered only because a baseline compile happened to be run before any edit was made. That baseline run is not prescribed anywhere: Phase 3 says to verify the build is clean after implementing, which is too late to tell a pre-existing breakage apart from one the cycle just introduced.

Why it matters

Three failure modes follow from branching off a red default branch without knowing it is red:

  • Misattribution. A cycle that first runs the anchor after editing sees a failure in a file it did not touch and has to work backwards to establish that the failure predates its work. Phase 4 requires the anchor to be green before the rubric may start, so a cycle can stall entirely on someone else's breakage.
  • A dead anchor. While the default branch is red, every pull request opened against it inherits a red required check, so the Phase 4 external anchor conveys nothing about the changeset under review. The rubric is explicitly grounded on that anchor, so the grounding is silently absent. This is the same shape as the structurally-red-anchor condition PR Require unique scratch filenames and define the structurally-red-anchor path #121 added at Phase 1, but for a transient breakage rather than a job that has never succeeded — the existing entry does not cover it, because the job in question succeeds routinely.
  • Unbounded latency. Nothing else in the loop watches the default branch, so a breakage there persists until a cycle stumbles into it.

Suggested instruction text

Add to the Phase 1 triage block, alongside the existing gh pr list / gh issue list / git log calls:

gh run list --branch {{DEFAULT_BRANCH}} --limit 5 --json name,conclusion,headSha,createdAt

with accompanying text along the lines of:

Check the default branch's own CI before selecting work. If the most recent run of a required check on {{DEFAULT_BRANCH}} failed, that failure is this cycle's work and outranks the open-issue backlog: while it stands, every pull request inherits a red required check, and the Phase 4 external anchor says nothing about the changeset under review. Establish a baseline by running {{COMPILE_CMD}} once on an unmodified {{DEFAULT_BRANCH}} before making any edit — a failure first observed after editing cannot be told apart from one the edit caused, and the cycle will spend turns proving it did not.

The baseline-before-editing half is worth stating even when the default branch is green, since it costs one command and converts an ambiguous later failure into an unambiguous one.

Generality

Nothing about this depends on the project it was found in, its language or its build tool. Any dev loop that opens pull requests against a default branch and treats CI as its external review anchor has the same blind spot and the same remedy. It is filed here rather than implemented in the instance for that reason.

Reported from the generated instance dmccoystephenson/medieval-factions-dev-loop, where it is tracked as issue #51.

This issue was filed during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    template-ruleShould be promoted into create-dev-loop.md

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions