Repository navigation
Report fork CI's tested base and withhold CI passed on an old base - #76837
Conversation
302d22d to
28def61
Compare
|
The re-run of the jobs stopped by EC2 Spot interruptions finished: attempt 2 passed. |
shahar1
left a comment
There was a problem hiding this comment.
The tested-base check is sound and well covered by tests. Two small things to look at when you rebase. The rebase is needed anyway: the PR conflicts with main after #76672 and #76681.
1. The large-PR fallback in compare_with_base does not check that the head is still the same (minor, dev/breeze/src/airflow_breeze/utils/fork_ci_reconcile.py, compare_with_base)
head_files = head_side.get("files") or []
if len(head_files) >= COMPARE_FILES_CAP:
head_files = client.rest_get(f"repos/{repo}/pulls/{pr.number}/files?per_page=100", paginate=True)The compare is pinned to pr.head_sha, but pulls/<n>/files lists the PR's current head. Both existing callers of that endpoint re-read the head after listing and give no answer if it moved: pull_request_changes_ci_files returns None, and GateChangeFinder.find raises PullRequestChangedError. This one does not. If a push lands between the two reads, the overlap for pr.head_sha is computed from another commit's files. That can produce a BaseCheck marked current, which then sits in the marker for that SHA. The window is narrow (300+ files, a push mid-run), so the fix is to follow the same pattern: re-read pulls/<n> after the listing and return None (or raise PullRequestChangedError) when head.sha != pr.head_sha. A test for this would add one parametrize case.
2. green_unverified gets a "Rebase needed" paragraph that contradicts its state text (minor, dev/breeze/src/airflow_breeze/utils/fork_ci_messages.py, state_comment / tested_base_text)
state_comment appends tested_base_text for both green and green_unverified. For a non-current base, the paragraph ends with "The CI passed check is not reported until fork CI is green for a branch rebased on <base>". For green_unverified, fork CI never reports CI passed, rebased or not. The state text above it says a maintainer will run project CI, and project CI then tests the merged result anyway. So the contributor is asked for a rebase that does not unblock anything. The overlap case in test_state_comment_says_which_base_was_tested uses green_unverified and asserts this text, so the behaviour looks deliberate, but the wording is wrong for that state. Options: keep only the first sentence (the tested base) for green_unverified, or skip compare_base() for it altogether. The second option also saves the compare calls for a state where the result is never used for CI passed.
Smaller observation, no change needed: when the head commit already has a pending CI passed, for example from the "no longer trusted" revoke after a compare failure on a marker without base, _set_ci_passed does not rewrite the description to "Rebase on … and push". The state comment still says to rebase, so this only matters if you want the check's description to say why.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The findings
below are observations, not blockers; an Apache Airflow
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.More on how Apache Airflow handles maintainer review:
Contributing guide.
Drafted-by: Claude Code (Opus 5.5); reviewed by shahar1 before posting
Fork CI tests the head commit of a gated pull request, while project CI tests it merged into the target branch, so a pull request could go green on a stale base and merge on that result. When a head commit turns green, the fork CI monitor now compares it with the base branch: the merge base it was tested on, how many commits the base branch has since, and which of the pull request's files the base branch changed since. `CI passed` is reported only when the base is fewer than 200 commits behind and changed none of those files; otherwise it is posted as pending with a rebase hint, and the state comment lists the overlapping files. Every green state comment and the `CI passed` description name the base commit the run tested. Testing the merge commit in the fork itself was rejected: it would have to change every checkout of the fork run, there is no merge ref before the pull request exists, GitHub recomputes it asynchronously, and the monitor would have to trust the run's own report of what it tested. The design document lists the reasons. Generated-by: Claude Opus 5
28def61 to
b52864f
Compare
* [v3-4-test] Sync fork CI changes from main Several fork CI changes merged to main since the last sync never reached v3-4-test, and the fork CI monitor reads the gate configuration of a pull request's base branch, so pull requests to v3-4-test were still closed after 7 days and missed ten pilot participants. The fork CI modules, their tests, the design document and its diagrams, the monitor workflow, the gate configuration checks and the contributor documentation now match main, taking in #75814, #75696, #76272, #76316, #76327, #76510, #76538, #76677, #76681, #76835, #76836, #76837, #76839, #76841, #76866, #76267 and #77100. The pilot participants new to this branch join at 2026-11-16 00:00 UTC rather than at their main dates, as the gate configuration check requires a future instant for an entry that is new on the base branch. Generated-by: Claude Opus 5 * [v3-4-test] Say the fork CI monitor runs every 5 minutes in the CI tasks doc The sentence changed with #76836 on main and was missed by the sync. Generated-by: Claude Opus 5 * [v3-4-test] Bring the fork CI GitHub client tests in line with main The tests for the smaller job listing pages (#76327) were missed by the sync; the client code they cover is already in it. Generated-by: Claude Opus 5
Why
One of the points that cancelled the AIP-120 vote: fork CI tests the tip of the contributor's
branch, while project CI tests the PR merged into the target branch, so a gated PR could go
green on a stale base and merge on that result.
What
branch: the merge base the run tested, how many commits the base branch has since, and which
of the PR's files the base branch changed since.
CI passedis reported only when the base is fewer than 200 commits behind and changed noneof the PR's files. Otherwise it is posted as pending with "Rebase on and push", and the
state comment lists the overlapping files and the rebase commands.
CI passeddescription name the base commit the run wastested on.
fork run would change, there is no merge ref before the PR exists, GitHub recomputes it
asynchronously, and the monitor would have to trust the run's own report); the reasons are in
the design doc's "Known limitations".
pull_requestrun, and no paths areexempt from the overlap check; both can be revisited if rebase churn becomes a problem.
related: https://lists.apache.org/thread/5603zwgpj7pgj6j979hbgpr72xsjbbws
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines
🤖 Generated with Claude Code