Skip to content

release: let the stable PR and tag build pass the stable canary-pin and doc-pin checks - #266

Draft
ErenAri wants to merge 6 commits into
mainfrom
claude/stable-canary-pin-bootstrap
Draft

ErenAri wants to merge 6 commits into
mainfrom
claude/stable-canary-pin-bootstrap

Conversation

@ErenAri

@ErenAri ErenAri commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Draft. Merge only on an exact-head operator decision. This is a release-tooling fix for the stable track. It changes nothing in the rc.5 path, and it publishes, tags or schedules nothing.

Defect (DEF-1)

scripts/check-canary-pins.sh always ran check_pin stable "v${stable_version}", with no bootstrap for the stable pin. For v0.4.0 that makes the documented process impossible:

  • Stable PR: release.yaml with stable 0.4.0 fails required CI, because v0.4.0 is not a tag yet.
  • v0.4.0 tag build: release-artifacts.yml runs check-release-consistency, which runs check-canary-pins. The build fails because the stable pin would have to name the tag's own commit, which no commit can contain.

docs/production-release-process.md already said to repin the stable step once the new stable tag exists. The code contradicted that. The guard (c8ccd2d, 2026-09-09) postdates v0.3.7 (2026-08-26), so no stable release has been built under it.

The release-sequence analysis found the defect, with a reproduction against the real script. It does not affect release_channel: prerelease, so the rc.5 publication path is unaffected.

Fix

The stable bootstrap now mirrors the existing candidate bootstrap:

  • When it applies: only with release_channel: stable, and only while vX.Y.Z positively does not exist or resolves to the commit under check (the tag build).
  • What it accepts: a full-SHA stable pin commented with the most recent published (non-draft, non-prerelease) stable vA.B.C before X.Y.Z, whose tag resolves to that SHA.
  • Fails closed: any lookup or API failure, including a partial page.
  • Later commits are strict again, so the repin after the tag stays enforced. The prerelease channel is unchanged.

The runbook is updated to match. A second commit closes a documentation gap in the same section:

  • the stable tag build also refuses undecided blocks_stable distribution decisions and a validator that differs from the graduated RC's, and a v* tag cannot be reused;
  • the [bpfcompat-solo-promotion:v1] record is an input of the final readiness report, so it is written before that report. The promotion dispatch stays a separate decision.

Sibling defect fixed here (DEF-2, found by the independent review of 462287d)

The release-consistency gate requires the stable PR's docs to name vX.Y.Z (check-release-consistency.sh:193-201). But scripts/check-doc-pins.sh refused any tag-form pin whose tag does not exist. So even with DEF-1 fixed, the stable PR could not pass the required "Test (-race)" job.

Commit f61c291 fixes this:

  • What it accepts: while release_channel: stable, a tag-form doc pin on exactly v<stable_version>, when the remote positively lacks that tag (git ls-remote --exit-code returns 2).
  • What still fails: an unreachable remote, another version, a SHA pin claiming the unreleased version, or the prerelease channel.
  • Tests: the check gains its first regression test, scripts/check-doc-pins_test.sh, run in CI. Its single DEF-2 case fails against the old script.

The runbook now also states the stable order: operator record → final readiness report → stable PR → tag → promotion. It also says no other stable release may be published between the stable PR's CI and the tag build. The stable pin may only name the newest published stable below X.Y.Z, so a maintenance release published in that window would fail the tag build, and the v* tag cannot be reused.

Third sibling defect fixed here (DEF-3, found by the independent review of f61c291)

That review simulated the stable PR against the real remote and releases API; tag-build states used a scratch bare remote. Even after DEF-1 and DEF-2, check-doc-pins.sh's one-version rule refused it. The docs/integrations/*.yml templates are pinned by commit SHA to v0.3.7, which docs/integrations/README.md makes policy, and a SHA pin cannot name v0.4.0 before its tag exists.

Commit 2273204 fixes this:

  • During the stable window (the stable PR while vX.Y.Z is absent, and the tag build where it resolves to HEAD), exactly one other documented version may sit beside v<stable_version>, and only as commit-SHA pins. It is the release the consumer canary's # bpfcompat-pin: stable step names, which check-canary-pins.sh verifies is the newest published stable release. The two checks therefore cannot disagree, and no tag listing is involved. This was refined in b9149e7.
  • Every later commit is strict. The runbook's post-tag step now repins the templates together with the consumer canary.
  • Consistency test: check-release-consistency_test.sh reads stable_version (2273204) and release_channel (b9149e7) from release.yaml instead of hard-coding 0.3.7 and prerelease, so the stable PR need not edit it.
  • Runbook: tag the merge commit as soon as the stable PR merges. From the merge until promotion, main's docs name a release that is not published yet.

Fourth sibling defect fixed here (DEF-4, found by the independent review of 2273204)

That review repeated the full stable-sequence simulation, against the real remote and releases API for the stable PR and against a scratch bare remote for the tag-build states. With DEF-1 to DEF-3 fixed, one line still failed: check-release-consistency_test.sh:57 asserted that the current tree is on the prerelease channel. That fails the stable PR's required "Test (-race)", the push to main, and the v0.4.0 tag build's quality-gates step (release-artifacts.yml:78). It now reads release_channel from release.yaml.

With that line fixed, the review's simulation passes the whole sequence:

  • Stable PR: all 32 CI test-job scripts and the tag build's quality gates.
  • Push to main.
  • Tag build: lightweight or annotated tag.
  • Repin: a later commit fails until both the canary and the templates are repinned, then passes.

b9149e7 also closes the review's S2 (a tag-form previous-release pin was let through) and S3 (the two checks' "previous stable" could disagree for an unpublished stable tag), as described above.

Validation

  • scripts/check-canary-pins_test.sh passes.

    • New sections 29–38 cover: the stable PR before the tag; the tag build where v0.4.0 resolves to HEAD, with the release drafted or published; strict refusal on a later commit; non-published, prerelease or older-than-newest pins; and fail-closed lookups and partial or malformed API answers.
    • Section 27 now asserts that the old pin is still refused once the new tag exists at another commit.
    • Against the unmodified script, 27 of the per-case checks fail with DEF-1's message, and the 6 strict-path cases already held.
  • make check-release-consistency passes on the real release.yaml (prerelease channel, unchanged output), and so do check-release-consistency_test.sh and the freeze check.

  • A read-only live run with stable 0.4.0 metadata against the real pins and API passes as a bootstrap on v0.3.7 179ff612…; the unmodified script fails it.

  • Independent review of 462287d: no blocker. 45 adversarial cases, 0 wrong outcomes. Its two should-fix items (DEF-2, and the publication-window warning) are addressed in f61c291.

  • At f61c291: scripts/check-doc-pins_test.sh is 7/7 (the stable-PR case is red on the old script).

  • Independent review of f61c291: no blocker. It ran 41 adversarial cases on DEF-2 and a full stable-PR simulation, which found DEF-3.

  • At 2273204:

    • check-doc-pins_test.sh is 13/13. The DEF-3 cases (stable PR and tag build with templates on the previous stable) are red on f61c291.
    • The strict cases pass: an older template, three versions, after the tag, and the prerelease channel.
    • check-release-consistency_test.sh, the canary-pin tests, actionlint, docs drift, release consistency and the freeze check pass.
    • A trial merge with release: prepare v0.4.0-rc.5 metadata and an offline, AST-based RC preflight (replaces #264) #265's head e1cbd97 is clean.
  • At b9149e7:

    • check-doc-pins_test.sh is 17/17. Three new strict cases are red on 2273204: a tag-form previous pin, templates disagreeing with the canary, and no canary pin.
  • Independent review of b9149e7: no blocker. Its full stable-sequence simulation now passes:

    • the stable PR (32 scripts plus release consistency);
    • the push to main;
    • the tag build (lightweight or annotated tag);
    • a later commit, which stays red until both the canary and the templates are repinned.

    Its one should-fix: the two checks read different comments on a canary line with two version comments.

  • At 4cb4516: check-doc-pins.sh reads the first # v… comment, as check-canary-pins.sh does. check-doc-pins_test.sh is 18/18, and the new case is red on b9149e7.

Decisions for the operator

  • Approval: whether to approve this bootstrap semantics. It mirrors the candidate rule and the documented repin-after-tag process.
  • Timing: land it before the rc.5 tag, so the rc.5 and v0.4.0 trees differ only in metadata, or only before the stable PR.
  • Which release counts: if v0.3.8 is published as a stable release before the v0.4.0 PR, that PR must pin v0.3.8. "Most recent" means the highest published stable version below X.Y.Z. This is stricter than the candidate rule, which accepts any earlier RC; that is the safe direction for a stable pin. Publishing another stable release between the stable PR's CI and the tag build must be avoided (see the runbook).
  • Token: with this change, the stable check needs an authenticated gh (GH_TOKEN). Test 28 already requires it on every workflow path.

Not changed

  • A commented-out canary pin line is accepted while the live step uses @main. This is pre-existing at 402d7c9 and outside this PR; it is recorded as a follow-up.
  • The comments at consumer-canary.yml:40-42, and the GH_TOKEN comments in ci.yml/release-artifacts.yml, now describe the old behaviour incompletely. The only workflow change here is one new ci.yml step that runs scripts/check-doc-pins_test.sh.
  • No CHANGELOG.md entry, to avoid conflicting with the rc.5 preparation PR release: prepare v0.4.0-rc.5 metadata and an offline, AST-based RC preflight (replaces #264) #265. Whichever lands second can add one line.

…heck

check-canary-pins.sh always ran check_pin stable "v<stable_version>": the
stable pin had to carry that exact version comment and equal the commit the
tag resolves to. With no bootstrap for the stable pin (DEF-1), the stable
release PR (stable_version = release_version = X.Y.Z, release_channel
stable) fails required CI because vX.Y.Z is not a tag yet, and the vX.Y.Z
tag build fails too, because the pin would have to name the tagged commit
itself, which that commit cannot contain. docs/production-release-process.md
already says to repin the stable step once the new stable tag exists; the
check made that sequence impossible to follow.

Mirror the existing candidate bootstrap for the stable pin. With
release_channel stable, while vX.Y.Z positively does not exist, or resolves
to the commit under check (the tag build), the stable pin may instead name
the most recent published stable release before X.Y.Z: the newest vA.B.C,
A.B.C < X.Y.Z, listed by the GitHub releases API as non-draft and
non-prerelease, whose tag resolves to the pinned full SHA. As for the
candidate, a missing, mutable or short pin fails, and every lookup failure
fails closed: an unreachable or malformed tag answer, a pinned tag that
does not resolve, a failed or partial API listing, or a malformed line.
Once vX.Y.Z exists at any other commit the rule is strict exactly as
before, so the first commit after the tag must carry the repin. The
prerelease channel's stable pin stays strict, and the candidate rules are
unchanged.

check-canary-pins_test.sh: section 27 no longer asserts that the stable pin
never bootstraps; it now tags the new stable at another commit and asserts
the previous stable pin is still refused there. Sections 29-38 cover the
stable PR, the tag build (also with the v0.4.0 draft staged and after its
publication), the strict commit after the tag and its repin, pins on the
new or a later version, an older-than-latest stable, an rc, a draft, a
release marked prerelease, a tag without a release, a mismatched SHA, a
missing tag, mutable and short refs, a missing pin, each lookup failure,
and the unchanged prerelease channel. The gh stub also answers the paginated
release listing, for the exact projection the script requests. The new
sections are red against the unmodified script and green here. The
release process doc spells out the stable PR and tag build phase.

Signed-off-by: ErenAri <erenari27@gmail.com>
The Stable Release runbook omitted two gates the stable tag build enforces
(every blocks_stable distribution decision recorded; the stable validator
byte-identical to the graduated RC's), and a v* tag cannot be reused after
a refused build. It also had the operator record the promotion marker after
checking the readiness report, while the report reads that record as an
input; the record now comes first and the promotion dispatch stays a
separate decision.

Signed-off-by: ErenAri <erenari27@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

DEF-2, found by the independent review of DEF-1's fix: the release-
consistency gate requires the stable PR's docs to name vX.Y.Z, while
scripts/check-doc-pins.sh refused any tag-form pin whose tag does not exist,
so the stable PR could not pass required CI even with the canary-pin fix.

While release.yaml says release_channel stable, a tag-form doc pin on
exactly v<stable_version> is accepted when the remote positively has no
such tag (git ls-remote --exit-code returns 2). An unreachable remote,
another version, a SHA pin or the prerelease channel still fail. The check
gains its first regression test, run in CI.

The runbook also states the stable order and that no other stable release
may be published between the stable PR's CI and the tag build, because the
stable pin may only name the newest published stable below X.Y.Z.

Signed-off-by: ErenAri <erenari27@gmail.com>
@ErenAri ErenAri changed the title release: let the stable PR and tag build pass the stable canary-pin check release: let the stable PR and tag build pass the stable canary-pin and doc-pin checks Oct 18, 2026
… stable window

DEF-3, found by the independent review of f61c291's simulated stable PR:
the docs/integrations templates are pinned by commit SHA (their README
makes that policy), so they cannot name vX.Y.Z before its tag exists, and
check-doc-pins.sh's one-version rule then refused the stable PR ("docs pin
more than one bpfcompat version").

During the stable window -- the stable PR while vX.Y.Z is absent, and the
vX.Y.Z tag build itself -- exactly one other documented version may sit
beside v<stable_version>: the newest earlier stable release tag on the
remote. Every later commit is strict, so the templates are repinned with the
consumer canary once the tag exists (now stated in the runbook).

check-release-consistency_test.sh no longer hard-codes 0.3.7 as the stable
channel's version; it reads stable_version from release.yaml, so the stable
PR does not have to edit the test.

Signed-off-by: ErenAri <erenari27@gmail.com>
Review of 2273204 (full stable-sequence simulation):
- DEF-4: check-release-consistency_test.sh still asserted that the current
  tree is on the prerelease channel, so the stable PR's required CI, the
  push to main and the v0.4.0 tag build's quality gates would fail. It now
  reads release_channel from release.yaml.
- The stable window's second documented version is now the release the
  consumer canary's '# bpfcompat-pin: stable' step names, which
  check-canary-pins.sh verifies is the newest published stable release, so
  the two checks cannot disagree (a stable tag without a published release
  no longer qualifies), and no remote tag listing is involved.
- Only commit-SHA pins count toward that second version; a tag-form pin on
  the previous release is refused as before.
- The ci.yml step comment and the runbook describe the window precisely;
  docs name an unreleased version until promotion, not only until the tag.

Signed-off-by: ErenAri <erenari27@gmail.com>
…s does

Review of b9149e7: on a canary line with two version comments
("# v0.3.7 # v0.3.6"), check-canary-pins.sh read the first and
check-doc-pins.sh the last, so the stable window could admit templates on
a different release than the verified canary pin. Both now read the
first, and a test pins it.

Signed-off-by: ErenAri <erenari27@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant