Skip to content

Test fork CI pull requests merged with their base branch - #77131

Merged
potiuk merged 6 commits into
mainfrom
fork-ci-test-merge-commit
Nov 15, 2026
Merged

potiuk merged 6 commits into
mainfrom
fork-ci-test-merge-commit

Conversation

@potiuk

@potiuk potiuk commented Nov 15, 2026

Copy link
Copy Markdown
Member

Fork CI tested the pushed commit as it is, while project CI's pull_request run tests the pull request merged into its base branch. The "Tested base" check (#76837) only narrowed that gap: CI passed waited for a base fewer than 200 commits behind with no overlapping files. This was one of the points raised in the cancelled ADIP-12 (AIP-120) vote. With this change, every job of a fork run tests the merge, as project CI does.

How a fork run tests the merge

  • build-info's existing early conflict check (Harden fork CI change detection and conflict handling #76955) already fetches the target branch and finds the merge base. It now also outputs both as merge-with, so the whole run merges with the same commit even if the target branch moves during the run.
  • A new local action, .github/actions/merge-with-base, runs after each job's checkout of the pushed commit:
    • It fetches the base commit and the merge base from apache/airflow at depth 1.
    • It builds the tree with git merge-tree --write-tree --merge-base, which needs no history between the commits.
    • It makes the merge commit with the base as first parent, as in refs/pull/<n>/merge, so <merge>^..<merge> is exactly the pull request's changes.
    • Identity and dates are fixed, so every job makes the same merge commit.
    • Nothing is pushed, and fork runs keep a read-only token.
  • merge-with is threaded through every reusable workflow a fork run calls. It is empty in apache/airflow, where each merge step is skipped. The step's if: matters because GitHub reads a local action from the checkout, which an older release ref or a skipped checkout doesn't have.
  • gate, build-info, notify-slack and summarize-warnings keep the pushed commit, because they read it or only report.
  • The basic static checks now diff <merge>^..<merge>, the whole pull request. Before, they diffed only the last pushed commit.
  • breeze's "how to reproduce locally" steps now recreate the merge: fetch both commits, check out the pushed one, git merge the base.
  • A new prek hook, check-fork-ci-tests-merge, keeps it that way. Every call must pass merge-with, and every job that checks out the pushed commit must run the merge step after its last checkout, guarded by merge-with.

How the monitor uses it

  • build-info names the merge in a step: "Test merged with apache/airflow <branch>@<sha>, merge base <sha>".
  • The reconciler reads that step and checks it against GitHub:
    • the branch must be the pull request's base;
    • the merge base must match what GitHub computes for the base commit and the head;
    • the base commit must be on the base branch.
  • If it checks out, CI passed only needs the merge to be fewer than 200 commits behind; the overlap rule no longer applies. The marker stores "merged": true, and the state comment and CI passed say "merged with".
  • If the reported merge does not check out, CI passed is withheld. That covers a stable-branch PR, whose fork run merges with the test branch that shares its defaults. Such a PR is listed for Slack, and a maintainer can use use project ci.
  • If the run's jobs cannot be read, the monitor retries on the next run.
  • Only a run that names no merge gets the old head-only comparison. That happens when the conflict check could not fetch the target branch or find the merge base.

The design doc (sections "Testing the merged result", "Tested base" and "Known limitations") and the contributor docs are updated.

Tested

  • The merge script ran locally against a real pushed commit and main. The second parent is the head, M^..M equals the PR's 6 changed files, and a second run produced the same merge SHA.
  • All breeze tests pass (2844), as do the new hook's tests (19).
  • Static checks pass, including actionlint, zizmor and the ci-amd/ci-arm sync check.

Not tested yet

  • Nothing has run in a real fork yet. That only happens when a pilot contributor pushes after merge.
  • Two things in particular to watch on the first fork run:
    • each job's "Merge with the base branch" step;
    • whether the jobs API returns build-info's step name with the step outputs filled in, which the monitor relies on.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Claude Opus 5)

Generated-by: Claude Code (Claude Opus 5) following the guidelines

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq

Fork CI tested the pushed commit as it is, while project CI tests the
pull request merged into its base branch, so a branch on an old base
could go green on code that was never tested merged. The tested base
check only bounded that gap. Each job now merges the pushed commit with
the base branch commit build-info fixed for the run, so fork CI tests
what project CI would, and the monitor drops the overlap rule for a run
that reports and verifies its merge.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq
GitHub reads a local action from the checkout before the step's inputs
count, so a matrix entry that skips its checkout, or a constraints run
that builds an older release, failed on the missing action even with
nothing to merge.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq
A run that reported a merge tested that merge, not the head commit as
it is, so comparing the head commit's own base could report CI passed
for code no job ran, such as a pull request to a stable branch whose
fork run merged with the test branch.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq
An unreadable job list looked like a run that merged nothing, so its
head commit could be compared as tested as it is and the result kept
in the marker, although its jobs may have tested a merge.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq
The local reproduction steps checked out the pushed commit, while the
fork run tested it merged with the base branch, so a failure caused by
the merged changes would not reproduce.

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq
The CI image build checks out `checkout-ref` into the workspace, which
can be a ref other than the workflow's own, so running a local action
from it lets code of that ref run in privileged runs such as the
scheduled canary (CodeQL actions/cache-poisoning/poisonable-step).

Generated-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_01NyQq9U7i5HhjDgrQ9qd3gq
@potiuk
potiuk merged commit 52487ed into main Nov 15, 2026
183 checks passed
@potiuk
potiuk deleted the fork-ci-test-merge-commit branch November 15, 2026 18:45
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-4-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
❌ v3-4-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 52487ed v3-4-test

This should apply the commit to the v3-4-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants