From a077c602bc5d031bf3bccb1c12fa8d84de0afd7a Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 16:27:39 +0300 Subject: [PATCH 01/38] release: prepare v0.4.0-rc.5 metadata (local draft) Set release_version to 0.4.0-rc.5 (stable_version 0.3.7 and the prerelease channel unchanged) and turn the Unreleased CHANGELOG section into the 0.4.0-rc.5 section, dated when the tag is cut. Add the merged changes it did not yet describe (installer verification bound to the requested tag; pinned validator build and stable-must-ship-RC-validator; per-report release gate; candidate-bound readiness evidence; CI token and gate hardening), migration notes and known limitations. The consumer-canary candidate pin stays on the published v0.4.0-rc.4 under check-canary-pins.sh's bootstrap rule until the rc.5 tag exists; it must be repinned in the designation change after verification. Signed-off-by: ErenAri --- CHANGELOG.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++ release.yaml | 2 +- 2 files changed, 59 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bcf9636..1b46cd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,20 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a ## [Unreleased] +## [0.4.0-rc.5] - TBD (date set when the tag is cut) + +Release candidate for v0.4.0. It closes the supported-path findings that made +v0.4.0-rc.4 a no-go, and pins how release binaries are built and what ships +with them. rc.4 qualification evidence does not carry over to this candidate. + ### Security +- `install.sh` binds verification to the requested release: the cosign + certificate identity and the GitHub attestation source ref must name exactly + that tag, so a signature or attestation made for another release (for example + a prerelease) no longer vouches for it. `BPFCOMPAT_VERSION` and the resolved + latest tag must look like `vX.Y.Z` or `vX.Y.Z-rc.N`; the installer body runs + only when the whole script was received; whether sudo is needed is decided per + target directory before anything is installed. - The GitHub Action no longer pastes inputs into its `run:` scripts. Every input reaches the scripts through `env:` and is read as quoted data, so a value containing quotes, `$(...)`, backticks or newlines can no longer run as @@ -66,11 +79,56 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a refuses while a stable-blocking decision is undecided. ### Changed +- Release build: the static validator is built inside a digest-pinned Ubuntu + builder from `validator/build-inputs.lock` (exact package versions from a + fixed snapshot.ubuntu.com state), and the build fails unless the result is + byte-identical to the recorded reproduction of the same validator sources. A + stable release must ship the same validator bytes as the release candidate it + graduates. +- Release gate: each VM gate report is evaluated on its own and the gate fails + closed on a missing or malformed report. +- Production readiness: the Falco, canary, rollback, incident and operator + evidence must be bound to the exact candidate (single-object JSON, exact + candidate-evidence hash in the operator note, rollback alias and digest of the + candidate, Falco started after the candidate evidence was generated). +- CI: top-level workflow tokens are read-only; the hosted compatibility gate is + required for `pkg/` and `scripts/` changes; untrusted-input parsers have fuzz + coverage. - **Breaking:** implicit Docker Hub short refs in tag form, such as `org/name:tag`, are no longer pulled. Spell them with the host, e.g. `docker.io/org/name:tag`; the "no such file" error suggests that spelling. Digest-pinned short refs (`org/name@sha256:…`) are still pulled. +### Migration notes +- Docker Hub images: write `docker.io/org/name:tag` instead of `org/name:tag`. + The "no such file" error suggests that spelling. +- GitHub Action users with workspace artifacts whose first path component + contains `.` or `:` (for example `.output/` from libbpf-bootstrap, or + `out.v2/`): prefix the path with `./`. Without it the Action treats the input + as a remote reference. +- Action inputs containing `$VAR`, backticks or `\` are now passed literally; + workflows that relied on bash expanding them must expand them in the workflow. +- Command mode: a target whose guest kernel release cannot be read is now + `INFRA_ERROR` (and fails a required run) instead of passing or failing. +- Self-hosted runners with a validator in `/usr/local/libexec` or + `/usr/libexec`: a release-pinned Action now runs the validator it downloaded; + set `BPFCOMPAT_VALIDATOR_BIN`/`BPFCOMPAT_VALIDATOR_SHA256` in the workflow to + keep using another one (the run then warns). +- `install.sh` installs the license bundle under `/../share/doc/bpfcompat//` + (`BPFCOMPAT_DOC_DIR` to change it) when the release publishes one. + +### Known limitations +- Distribution decisions recorded in `distribution/policy.json` are all + undecided. This candidate's license bundle and relink kit record facts and + ship texts verbatim; they do not elect a license or state that any license + condition is met. A stable release is refused until the stable-blocking + decisions are recorded. +- The prebuilt static validator is published for x86_64 only. +- See "Known limitations of the supported path" in + `docs/compatibility-contract.md` (artifact-path classification, untrusted + names in reports, classification codes, Action binary precedence, shared + hosts and SSH). + ## [0.4.0-rc.4] - 2026-10-14 ### Added diff --git a/release.yaml b/release.yaml index f61109e..08f0a52 100644 --- a/release.yaml +++ b/release.yaml @@ -1,5 +1,5 @@ stable_version: 0.3.7 -release_version: 0.4.0-rc.4 +release_version: 0.4.0-rc.5 release_channel: prerelease release_operator: ErenAri approval_mode: solo-maintainer From 12fe465dd55cf7ea1b0a2c201d882632be26454e Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 16:27:39 +0300 Subject: [PATCH 02/38] release: add a read-only release-candidate preflight inventory scripts/rc-preflight.sh reports each prerequisite of the RC sequence as VERIFIED, MISSING or PENDING with its evidence: metadata, recorded distribution decisions, tag and approved commit, tag build, staged assets, SHA256SUMS membership, validator identity against the lock, optional assets against the tagged policy, publication, designation on main, canary runs in their 20-30h/68-78h windows, scheduled campaigns, rollback drill, and the operator-held notes. It only issues GETs (plus downloading the SHA256SUMS text), never dispatches, writes or executes anything, aborts on a lookup failure instead of reporting it as pending, and never turns elapsed time into VERIFIED. Candidate binding stays with production-readiness-report.sh. scripts/rc-preflight_test.sh covers it in CI. Signed-off-by: ErenAri --- .github/workflows/ci.yml | 4 + scripts/rc-preflight.sh | 230 +++++++++++++++++++++++++++++++++++ scripts/rc-preflight_test.sh | 184 ++++++++++++++++++++++++++++ 3 files changed, 418 insertions(+) create mode 100755 scripts/rc-preflight.sh create mode 100755 scripts/rc-preflight_test.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3d64282..0183d83 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -108,6 +108,10 @@ jobs: # missing or inconsistent inputs; open distribution decisions withhold # optional assets and block stable builds. run: bash scripts/distribution-bundle_test.sh + - name: RC preflight regression + # The read-only release-candidate inventory reports VERIFIED, MISSING + # or PENDING from observed state only; lookup failures abort it. + run: bash scripts/rc-preflight_test.sh - name: Image prefetch reliability regression # The public matrix lane lost five of seven weekly runs (Aug-Oct 2026) # to a single failed image download aborting the whole prefetch. These diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh new file mode 100755 index 0000000..387f635 --- /dev/null +++ b/scripts/rc-preflight.sh @@ -0,0 +1,230 @@ +#!/usr/bin/env bash +# Read-only inventory of a release candidate's prerequisites and evidence. +# +# rc-preflight.sh --tag vX.Y.Z-rc.N [--commit <40-hex>] [--repo owner/repo] +# +# Prints one line per item: VERIFIED, MISSING or PENDING, the item, and the +# evidence behind the status. It only reads: GitHub REST GETs through `gh api`, +# `gh release download` of the text file SHA256SUMS, and files in this +# checkout. It never dispatches or reruns a workflow, never writes to GitHub, +# never executes a downloaded file and never writes operator evidence. +# +# Status meanings: +# VERIFIED observed directly (a ref, a run, a file, a hash) and as required. +# MISSING observed and wrong, or its window has closed without it. +# PENDING not observable yet (no tag, no run, operator-held evidence, an +# open decision). Elapsed time alone never makes an item VERIFIED. +# Run-level canary/campaign/drill items say only that a run of the right kind +# exists; binding its evidence to the candidate is the job of +# scripts/production-readiness-report.sh, which this does not replace. Canary +# windows (20-30h, 68-78h after publication) are judged from each run's start +# time as a proxy for its plan job, which enforces the exact window itself. +# +# Exit: 0 no MISSING, 1 at least one MISSING, 2 usage, 3 a lookup failed (the +# report is then incomplete and nothing is inferred from the failure). +# Test hook: BPFCOMPAT_PREFLIGHT_NOW= fixes "now". +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +repo="Kernel-Guard/bpfcompat" tag="" want_commit="" +while [[ $# -gt 0 ]]; do + case "$1" in + --tag) tag="${2:-}"; shift 2 ;; + --commit) want_commit="${2:-}"; shift 2 ;; + --repo) repo="${2:-}"; shift 2 ;; + *) echo "usage: $0 --tag vX.Y.Z-rc.N [--commit ] [--repo owner/repo]" >&2; exit 2 ;; + esac +done +[[ "$tag" =~ ^v([0-9]+\.[0-9]+\.[0-9]+)-rc\.[1-9][0-9]*$ ]] || { echo "rc-preflight: --tag must be vX.Y.Z-rc.N" >&2; exit 2; } +[[ -z "$want_commit" || "$want_commit" =~ ^[0-9a-f]{40}$ ]] || { echo "rc-preflight: --commit must be a full SHA" >&2; exit 2; } +[[ "$repo" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]] || { echo "rc-preflight: invalid --repo" >&2; exit 2; } +for t in gh jq git; do command -v "$t" >/dev/null || { echo "rc-preflight: $t is required" >&2; exit 2; }; done +version="${tag#v}" +now="${BPFCOMPAT_PREFLIGHT_NOW:-$(date -u +%s)}" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +missing=0 +say() { # + printf '%-8s %-44s %s\n' "$1" "$2" "$3" + [[ "$1" != MISSING ]] || missing=1 +} +# fetch : GET into . Returns 0, or 4 for HTTP 404; any other +# failure aborts the whole report (exit 3). Runs in the main shell, never in +# $(...), so the abort cannot be mistaken for "not found". +fetch() { + if gh api "$1" >"$2" 2>"$tmp/err"; then return 0; fi + if grep -q 'HTTP 404' "$tmp/err"; then return 4; fi + echo "rc-preflight: lookup failed: GET $1: $(head -1 "$tmp/err")" >&2 + exit 3 +} +# need : like fetch, but a 404 is a lookup failure too. +need() { + fetch "$1" "$2" || { echo "rc-preflight: lookup failed: GET $1: not found" >&2; exit 3; } +} +epoch() { date -u -d "$1" +%s; } + +echo "rc-preflight: ${repo} ${tag} at $(date -u -d "@$now" +%Y-%m-%dT%H:%M:%SZ) (read-only)" + +# ------------------------------------------------------------ source metadata +rv="$(awk -F': ' '$1 == "release_version" { print $2 }' "$ROOT/release.yaml")" +rc="$(awk -F': ' '$1 == "release_channel" { print $2 }' "$ROOT/release.yaml")" +if [[ "$rv" == "$version" && "$rc" == prerelease ]]; then say VERIFIED "release.yaml names the candidate" "release_version=$rv release_channel=$rc" +else say MISSING "release.yaml names the candidate" "release_version=$rv release_channel=$rc (checkout $(git -C "$ROOT" rev-parse --short HEAD))"; fi +if grep -Fq "## [${version}]" "$ROOT/CHANGELOG.md"; then say VERIFIED "CHANGELOG has the candidate heading" "## [${version}]" +else say MISSING "CHANGELOG has the candidate heading" "no '## [${version}]' heading"; fi +expected_validator="$(awk -F= '$1 == "reproduces" { split($2, a, " "); print a[2] }' "$ROOT/validator/build-inputs.lock")" + +policy="$ROOT/distribution/policy.json" +if ! bash "$ROOT/scripts/check-distribution-policy.sh" "$policy" >/dev/null 2>&1; then + say MISSING "distribution policy is well-formed" "scripts/check-distribution-policy.sh refused $policy" +else + open="$(jq -r '[.blocks_stable[] as $k | select(.decisions[$k].status != "decided") | $k] | join(",")' "$policy")" + pre="$(jq -r '.decisions.prerelease_with_open_decisions | "\(.status) \(.value)"' "$policy")" + case "$pre" in + "decided true") say VERIFIED "recorded: prerelease with open decisions" "operator recorded true (a policy value, not a legal finding)" ;; + "decided false") + if [[ -n "$open" ]]; then say MISSING "recorded: prerelease with open decisions" "recorded false while open: $open" + else say VERIFIED "recorded: prerelease with open decisions" "recorded false; no stable-blocking decision is open"; fi ;; + *) say PENDING "recorded: prerelease with open decisions" "undecided (Q17); open stable-blocking decisions: ${open:-none}" ;; + esac + if [[ -z "$open" ]]; then say VERIFIED "recorded: stable-blocking decisions" "all recorded in distribution/policy.json" + else say PENDING "recorded: stable-blocking decisions" "undecided: $open (needed before a stable tag, not before this RC)"; fi +fi + +# ------------------------------------------------------------ tag +if ! fetch "repos/${repo}/git/ref/tags/${tag}" "$tmp/ref.json"; then + say PENDING "tag exists" "$tag not created" + for item in "tag build (release-artifacts)" "release staged with required assets" "SHA256SUMS lists required assets" \ + "validator identity" "optional assets match tagged policy" "RC published as prerelease"; do + say PENDING "$item" "requires $tag" + done + for item in "designated: campaign-candidate.json" "designated: consumer-canary candidate pin" "designated: release-candidate-canary" \ + "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)" "rollback drill run"; do + say PENDING "$item" "requires a published $tag" + done + say PENDING "Falco expanded entry" "operator-held; bound by production-readiness-report.sh" + say PENDING "fail-closed promotion incident note" "operator-held; bound by production-readiness-report.sh" + say PENDING "operator promotion confirmation" "operator-held; never written by tooling" + exit "$missing" +fi +obj_type="$(jq -r .object.type "$tmp/ref.json")" obj_sha="$(jq -r .object.sha "$tmp/ref.json")" +commit="$obj_sha" +if [[ "$obj_type" == tag ]]; then need "repos/${repo}/git/tags/${obj_sha}" "$tmp/tagobj.json"; commit="$(jq -r .object.sha "$tmp/tagobj.json")"; fi +if [[ -z "$want_commit" ]]; then say VERIFIED "tag exists" "$tag -> $commit (no --commit to compare)" +elif [[ "$commit" == "$want_commit" ]]; then say VERIFIED "tag exists" "$tag -> $commit (as approved)" +else say MISSING "tag exists" "$tag -> $commit, approved commit is $want_commit"; fi + +# ------------------------------------------------------------ tag build +need "repos/${repo}/actions/workflows/release-artifacts.yml/runs?event=push&head_sha=${commit}&per_page=50" "$tmp/runs.json" +run="$(jq -c --arg t "$tag" '[.workflow_runs[] | select(.head_branch == $t)] | sort_by(.run_attempt) | last // empty' "$tmp/runs.json")" +if [[ -z "$run" ]]; then say PENDING "tag build (release-artifacts)" "no push run for $tag at $commit yet" +else + rid="$(jq -r .id <<<"$run")" ra="$(jq -r .run_attempt <<<"$run")" st="$(jq -r .status <<<"$run")" co="$(jq -r .conclusion <<<"$run")" + if [[ "$st" != completed ]]; then say PENDING "tag build (release-artifacts)" "run $rid attempt $ra is $st" + elif [[ "$co" == success ]]; then say VERIFIED "tag build (release-artifacts)" "run $rid attempt $ra success" + else say MISSING "tag build (release-artifacts)" "run $rid attempt $ra concluded $co"; fi +fi + +# ------------------------------------------------------------ release +need "repos/${repo}/releases?per_page=100" "$tmp/releases.json" +rel="$(jq -c --arg t "$tag" '[.[] | select(.tag_name == $t)] | first // empty' "$tmp/releases.json")" +required=(bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-validator-static-linux-amd64 + bpfcompat-third-party-licenses.tar.gz bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json) +signed=(SHA256SUMS SHA256SUMS.sig SHA256SUMS.crt bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt release-candidate-evidence.json) +published_at="" +if [[ -z "$rel" ]]; then + for item in "release staged with required assets" "SHA256SUMS lists required assets" "validator identity" \ + "optional assets match tagged policy" "RC published as prerelease"; do say PENDING "$item" "no release for $tag yet"; done +else + names="$(jq -r '.assets[].name' <<<"$rel")" + absent=() + for a in "${required[@]}" "${signed[@]}"; do grep -qxF "$a" <<<"$names" || absent+=("$a"); done + if [[ ${#absent[@]} -eq 0 ]]; then say VERIFIED "release staged with required assets" "$(jq '.assets | length' <<<"$rel") assets" + else say MISSING "release staged with required assets" "absent: ${absent[*]}"; fi + if grep -qxF SHA256SUMS <<<"$names" && gh release download "$tag" --repo "$repo" --pattern SHA256SUMS --dir "$tmp/sums" >/dev/null 2>"$tmp/err"; then + sums="$tmp/sums/SHA256SUMS" + unlisted=() + for a in "${required[@]}"; do awk -v f="$a" '$2 == f && $1 ~ /^[0-9a-f]{64}$/ { n++ } END { exit n != 1 }' "$sums" || unlisted+=("$a"); done + if [[ ${#unlisted[@]} -eq 0 ]]; then say VERIFIED "SHA256SUMS lists required assets" "${#required[@]} required entries, $(grep -c . "$sums") lines" + else say MISSING "SHA256SUMS lists required assets" "missing or duplicated: ${unlisted[*]}"; fi + vs="$(awk '$2 == "bpfcompat-validator-static-linux-amd64" { print $1 }' "$sums")" + if [[ -n "$vs" && "$vs" == "$expected_validator" ]]; then say VERIFIED "validator identity" "$vs (validator/build-inputs.lock reproduces=)" + else say MISSING "validator identity" "SHA256SUMS ${vs:-none}, lock expects $expected_validator"; fi + if fetch "repos/${repo}/contents/distribution/policy.json?ref=${tag}" "$tmp/policy-contents.json" && + jq -r .content "$tmp/policy-contents.json" | base64 -d >"$tmp/policy.json" 2>/dev/null && [[ -s "$tmp/policy.json" ]]; then + bad="" + for pair in "ship_link_inputs_asset:bpfcompat-validator-link-inputs-linux-amd64.tar.gz" "ship_native_sources_asset:bpfcompat-validator-native-sources.tar"; do + d="${pair%%:*}" f="${pair#*:}" + ship="$(bash "$ROOT/scripts/check-distribution-policy.sh" --ship "$d" "$tmp/policy.json" 2>/dev/null || echo error)" + listed=false; awk -v f="$f" '$2 == f { n++ } END { exit !n }' "$sums" && listed=true + [[ "$ship" == "$listed" ]] || bad+="$d=$ship but $f listed=$listed; " + done + if [[ -z "$bad" ]]; then say VERIFIED "optional assets match tagged policy" "policy at $tag agrees with SHA256SUMS" + else say MISSING "optional assets match tagged policy" "$bad"; fi + else + say MISSING "optional assets match tagged policy" "distribution/policy.json absent at $tag" + fi + else + for item in "SHA256SUMS lists required assets" "validator identity" "optional assets match tagged policy"; do + say MISSING "$item" "SHA256SUMS is not downloadable from the $tag release" + done + fi + draft="$(jq -r .draft <<<"$rel")" prerel="$(jq -r .prerelease <<<"$rel")" + if [[ "$draft" == true ]]; then say PENDING "RC published as prerelease" "draft (promotion pending)" + elif [[ "$prerel" == true ]]; then published_at="$(jq -r .published_at <<<"$rel")"; say VERIFIED "RC published as prerelease" "published $published_at" + else say MISSING "RC published as prerelease" "published but not marked prerelease"; fi +fi + +# ------------------------------------------------------------ designation and qualification +if [[ -z "$published_at" ]]; then + for item in "designated: campaign-candidate.json" "designated: consumer-canary candidate pin" "designated: release-candidate-canary" \ + "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)" "rollback drill run"; do + say PENDING "$item" "requires a published $tag" + done +else + main_file() { # + need "repos/${repo}/contents/$1?ref=main" "$tmp/contents.json" + jq -r .content "$tmp/contents.json" | base64 -d >"$2" + } + main_file .github/campaign-candidate.json "$tmp/cc.json"; cc="$(cat "$tmp/cc.json")" + if [[ "$(jq -r .tag <<<"$cc")" == "$tag" && "$(jq -r .commit_sha <<<"$cc")" == "$commit" ]]; then say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit" + else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag <<<"$cc") (retarget after verification)"; fi + main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" + pin="$(grep -A1 'bpfcompat-pin: candidate' "$tmp/consumer.yml" | tail -1)" + if [[ "$pin" == *"@${commit} # ${tag}"* ]]; then say VERIFIED "designated: consumer-canary candidate pin" "pinned to $commit" + else say PENDING "designated: consumer-canary candidate pin" "main pins:${pin##*@}"; fi + main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml"; rcc="$(cat "$tmp/rcc.yml")" + rcc_tag="$(sed -nE 's/^ *CANDIDATE_TAG: *"?([^"]*)"?$/\1/p' <<<"$rcc" | head -1)" + rcc_pub="$(sed -nE 's/^ *CANDIDATE_PUBLISHED_AT: *"?([^"]*)"?$/\1/p' <<<"$rcc" | head -1)" + if [[ "$rcc_tag" == "$tag" && "$rcc_pub" == "$published_at" ]]; then say VERIFIED "designated: release-candidate-canary" "$rcc_tag published $rcc_pub" + else say PENDING "designated: release-candidate-canary" "main has $rcc_tag published ${rcc_pub:-?} (retarget with dated schedules)"; fi + + pub="$(epoch "$published_at")" + need "repos/${repo}/actions/workflows/release-candidate-canary.yml/runs?created=%3E%3D${published_at}&per_page=100" "$tmp/cruns.json" + canary() { # + local hit + hit="$(jq -r --argjson p "$pub" --arg e "$2" --argjson lo "$3" --argjson hi "$4" ' + [.workflow_runs[] | select(.event == $e and .status == "completed" and .conclusion == "success") + | select((((.run_started_at | fromdateiso8601) - $p) / 3600 | floor) as $h | $h >= $lo and $h <= $hi) + | "\(.id) attempt \(.run_attempt) started \(.run_started_at)"] | first // empty' "$tmp/cruns.json")" + if [[ -n "$hit" ]]; then say VERIFIED "canary $1" "successful $2 run $hit (candidate binding: readiness report)" + elif (( now < pub + ($4 + 1) * 3600 )); then say PENDING "canary $1" "window ${3}-${4}h after publication open or not reached" + else say MISSING "canary $1" "no successful $2 run started ${3}-${4}h after publication"; fi + } + canary manual workflow_dispatch 0 100000 + canary t-plus-24h schedule 20 30 + canary t-plus-72h schedule 68 78 + + need "repos/${repo}/actions/workflows/latest-kernel-compatibility.yml/runs?event=schedule&created=%3E%3D${published_at}&per_page=100" "$tmp/camp.json" + n="$(jq '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success")] | length' "$tmp/camp.json")" + if [[ "$n" -ge 4 ]]; then say VERIFIED "scheduled campaigns (4)" "$n successful scheduled runs since publication (spacing and binding: readiness report)" + else say PENDING "scheduled campaigns (4)" "$n of 4 successful scheduled runs since publication"; fi + need "repos/${repo}/actions/workflows/rollback-drill.yml/runs?created=%3E%3D${published_at}&per_page=50" "$tmp/drill.json" + drill="$(jq -r '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success") | .id] | first // empty' "$tmp/drill.json")" + if [[ -n "$drill" ]]; then say VERIFIED "rollback drill run" "successful run $drill since publication (alias/digest binding: readiness report)" + else say PENDING "rollback drill run" "no successful run since publication"; fi +fi +say PENDING "Falco expanded entry" "operator-held; bound by production-readiness-report.sh" +say PENDING "fail-closed promotion incident note" "operator-held; bound by production-readiness-report.sh" +say PENDING "operator promotion confirmation" "operator-held; never written by tooling" +exit "$missing" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh new file mode 100755 index 0000000..4992d1a --- /dev/null +++ b/scripts/rc-preflight_test.sh @@ -0,0 +1,184 @@ +#!/usr/bin/env bash +# Regression cover for scripts/rc-preflight.sh, the read-only RC inventory. +# +# A fake `gh` answers `gh api ` from a route table (glob -> HTTP status -> +# fixture file) and `gh release download` from a fixture SHA256SUMS, and logs +# every call so the test can prove nothing but reads happened. "Now" is fixed +# with BPFCOMPAT_PREFLIGHT_NOW. The checkout's own release.yaml, CHANGELOG, +# lock and distribution policy are read as they are. +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +PRE="$ROOT_DIR/scripts/rc-preflight.sh" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +failures=0 +fail() { echo "[rc-preflight-test] FAIL: $*" >&2; failures=$((failures + 1)); } +ok() { echo "[rc-preflight-test] ok: $*"; } + +version="$(awk -F': ' '$1 == "release_version" { print $2 }' "$ROOT_DIR/release.yaml")" +TAG="v${version}" +[[ "$TAG" =~ -rc\.[0-9]+$ ]] || { echo "[rc-preflight-test] skip: release.yaml is not a prerelease ($version)"; exit 0; } +VSHA="$(awk -F= '$1 == "reproduces" { split($2, a, " "); print a[2] }' "$ROOT_DIR/validator/build-inputs.lock")" +C1="$(printf 'a%.0s' {1..40})" C2="$(printf 'b%.0s' {1..40})" TAGOBJ="$(printf 'c%.0s' {1..40})" +PUB="2026-11-02T10:00:00Z" +PUB_EPOCH="$(date -u -d "$PUB" +%s)" +at() { date -u -d "@$((PUB_EPOCH + $1 * 3600 + 600))" +%Y-%m-%dT%H:%M:%SZ; } # -> ISO, 10 min past the hour + +mkdir -p "$tmp/bin" +cat >"$tmp/bin/gh" <<'EOF' +#!/usr/bin/env bash +printf '%s\n' "$*" >>"$GH_LOG" +if [[ "$1" == api ]]; then + while IFS=$'\t' read -r glob status file; do + [[ -z "$glob" || "$glob" == \#* ]] && continue + # shellcheck disable=SC2053 + if [[ "$2" == $glob ]]; then + case "$status" in + 200) cat "$FIX/$file"; exit 0 ;; + *) echo "gh: HTTP $status (https://api.github.com/$2)" >&2; exit 1 ;; + esac + fi + done <"$FIX/routes" + echo "gh: HTTP 404 (unrouted $2)" >&2; exit 1 +fi +if [[ "$1 $2" == "release download" ]]; then + dir=""; while [[ $# -gt 0 ]]; do [[ "$1" == --dir ]] && dir="$2"; shift; done + mkdir -p "$dir"; cp "$FIX/SHA256SUMS" "$dir/SHA256SUMS"; exit 0 +fi +echo "gh shim: refusing non-read command: $*" >&2; exit 9 +EOF +chmod +x "$tmp/bin/gh" + +b64() { base64 -w0 "$1"; } +contents() { jq -n --arg c "$(b64 "$1")" '{content: $c}'; } + +# scenario : a fixture dir with a published release by default. +scenario() { + F="$tmp/$1"; mkdir -p "$F" + jq -n --arg s "$TAGOBJ" '{object: {type: "tag", sha: $s}}' >"$F/ref.json" + jq -n --arg s "$C1" '{object: {type: "commit", sha: $s}}' >"$F/tagobj.json" + jq -n --arg t "$TAG" '{workflow_runs: [{id: 900, run_attempt: 1, head_branch: $t, status: "completed", conclusion: "success"}]}' >"$F/runs.json" + names=(bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-validator-static-linux-amd64 bpfcompat-third-party-licenses.tar.gz + bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json SHA256SUMS SHA256SUMS.sig SHA256SUMS.crt + bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt release-candidate-evidence.json) + printf '%s\n' "${names[@]}" | jq -R '{name: .}' | jq -s --arg t "$TAG" --arg p "$PUB" \ + '[{tag_name: $t, draft: false, prerelease: true, published_at: $p, assets: .}]' >"$F/releases.json" + { for n in bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-third-party-licenses.tar.gz bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json; do + printf '%064d %s\n' 1 "$n"; done + printf '%s bpfcompat-validator-static-linux-amd64\n' "$VSHA"; } >"$F/SHA256SUMS" + contents "$ROOT_DIR/distribution/policy.json" >"$F/policy.json" + jq -n --arg t "$TAG" --arg c "$C1" '{tag: $t, commit_sha: $c}' >"$F/cc-raw.json"; contents "$F/cc-raw.json" >"$F/cc.json" + printf ' - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" + printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$TAG" "$PUB" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" + jq -n --arg m "$(at 0)" --arg a "$(at 25)" --arg b "$(at 70)" '{workflow_runs: [ + {id: 1, run_attempt: 1, event: "workflow_dispatch", status: "completed", conclusion: "success", run_started_at: $m}, + {id: 2, run_attempt: 1, event: "schedule", status: "completed", conclusion: "success", run_started_at: $a}, + {id: 3, run_attempt: 1, event: "schedule", status: "completed", conclusion: "success", run_started_at: $b}]}' >"$F/cruns.json" + jq -n '{workflow_runs: [range(4) | {status: "completed", conclusion: "success"}]}' >"$F/camp.json" + jq -n '{workflow_runs: [{id: 77, status: "completed", conclusion: "success"}]}' >"$F/drill.json" + cat >"$F/routes" < [args...] + local s="$1" h="$2"; shift 2 + : >"$tmp/gh.log" + set +e + PATH="$tmp/bin:$PATH" FIX="$tmp/$s" GH_LOG="$tmp/gh.log" BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + h * 3600)) \ + bash "$PRE" --tag "$TAG" "$@" >"$tmp/out" 2>&1 + rc=$? + set -e +} +line() { grep -E "^$1 +$2 " "$tmp/out" >/dev/null; } +show() { sed 's/^/ | /' "$tmp/out" >&2; } + +# 1. Before the tag: everything after it is PENDING, nothing is MISSING. +scenario pretag; route pretag $'repos/*/git/ref/tags/*\t404\tnone' +run pretag 0 +if [[ $rc -eq 0 ]] && line PENDING "tag exists" && line PENDING "validator identity" && line PENDING "canary t-plus-72h" && + ! grep -q '^MISSING' "$tmp/out" && ! grep -q '^VERIFIED tag' "$tmp/out"; then ok "no tag: later items PENDING, exit 0" +else fail "no tag (rc=$rc)"; show; fi + +# 2. A lookup failure is not "not found": the report aborts (exit 3). +scenario apierr; route apierr $'repos/*/git/ref/tags/*\t502\tnone' +run apierr 0 +if [[ $rc -eq 3 ]] && ! grep -q 'tag exists' "$tmp/out" && grep -q 'lookup failed' "$tmp/out"; then ok "API failure aborts instead of reporting PENDING" +else fail "API failure (rc=$rc)"; show; fi +scenario apierr2; route apierr2 $'repos/*/releases?per_page=100\t500\tnone' +run apierr2 0 +if [[ $rc -eq 3 ]] && ! grep -q 'release staged' "$tmp/out"; then ok "a later lookup failure aborts too" +else fail "later API failure (rc=$rc)"; show; fi + +# 3. Published release, all evidence observable, inside every window. +scenario full +run full 200 --commit "$C1" +if [[ $rc -eq 0 ]] && line VERIFIED "tag exists" && grep -q "as approved" "$tmp/out" && line VERIFIED "tag build" && + line VERIFIED "release staged" && line VERIFIED "SHA256SUMS lists required assets" && line VERIFIED "validator identity" && + line VERIFIED "optional assets match tagged policy" && line VERIFIED "RC published as prerelease" && + line VERIFIED "designated: campaign-candidate.json" && line VERIFIED "designated: consumer-canary candidate pin" && + line VERIFIED "designated: release-candidate-canary" && line VERIFIED "canary manual" && line VERIFIED "canary t-plus-24h" && + line VERIFIED "canary t-plus-72h" && line VERIFIED "scheduled campaigns" && line VERIFIED "rollback drill run" && + line PENDING "operator promotion confirmation"; then ok "annotated tag dereferenced; every observable item VERIFIED" +else fail "full scenario (rc=$rc)"; show; fi +if awk '!/^api / && !/^release download .*--pattern SHA256SUMS/' "$tmp/gh.log" | grep -q .; then fail "non-read gh call: $(cat "$tmp/gh.log")" +elif grep -qE -- '(-X|--method)' "$tmp/gh.log"; then fail "gh api called with a method" +else ok "only GET api calls and the SHA256SUMS download were made"; fi + +# 4. Wrong identities are MISSING. +run full 200 --commit "$C2" +if [[ $rc -eq 1 ]] && line MISSING "tag exists"; then ok "tag on another commit than approved: MISSING"; else fail "commit mismatch (rc=$rc)"; show; fi +scenario badval; sed -i "s/^$VSHA /$(printf '%064d' 7) /" "$tmp/badval/SHA256SUMS" +run badval 200 +if [[ $rc -eq 1 ]] && line MISSING "validator identity"; then ok "different validator digest: MISSING"; else fail "validator mismatch (rc=$rc)"; show; fi +scenario noasset; jq '.[0].assets |= map(select(.name != "bpfcompat-third-party-licenses.tar.gz"))' "$tmp/noasset/releases.json" >"$tmp/x" && mv "$tmp/x" "$tmp/noasset/releases.json" +run noasset 200 +if [[ $rc -eq 1 ]] && line MISSING "release staged" && grep -q 'absent: bpfcompat-third-party-licenses.tar.gz' "$tmp/out"; then ok "missing release asset: MISSING" +else fail "missing asset (rc=$rc)"; show; fi +scenario optional; printf '%064d bpfcompat-validator-native-sources.tar\n' 5 >>"$tmp/optional/SHA256SUMS" +run optional 200 +if [[ $rc -eq 1 ]] && line MISSING "optional assets match tagged policy"; then ok "optional asset shipped against the tagged (undecided) policy: MISSING" +else fail "optional asset (rc=$rc)"; show; fi + +# 5. Time: windows open are PENDING, closed and empty are MISSING; elapsed time never VERIFIES. +scenario early; jq '.workflow_runs = [.workflow_runs[0]]' "$tmp/early/cruns.json" >"$tmp/x" && mv "$tmp/x" "$tmp/early/cruns.json" +jq '.workflow_runs = []' "$tmp/early/camp.json" >"$tmp/x" && mv "$tmp/x" "$tmp/early/camp.json" +jq '.workflow_runs = []' "$tmp/early/drill.json" >"$tmp/x" && mv "$tmp/x" "$tmp/early/drill.json" +run early 50 +if line PENDING "canary t-plus-72h" && line MISSING "canary t-plus-24h" && line PENDING "scheduled campaigns" && + grep -q '0 of 4' "$tmp/out" && line PENDING "rollback drill run"; then ok "50h: T+24h window closed empty is MISSING, T+72h still PENDING" +else fail "time windows at 50h (rc=$rc)"; show; fi +run early 400 +if line MISSING "canary t-plus-72h" && line PENDING "scheduled campaigns" && ! line VERIFIED "scheduled campaigns"; then + ok "400h with no runs: T+72h MISSING, campaigns still PENDING (never VERIFIED by time)" +else fail "time windows at 400h (rc=$rc)"; show; fi +scenario late; jq --arg l "$(at 35)" '.workflow_runs[1].run_started_at = $l' "$tmp/late/cruns.json" >"$tmp/x" && mv "$tmp/x" "$tmp/late/cruns.json" +run late 200 +if line MISSING "canary t-plus-24h"; then ok "a schedule run outside 20-30h does not count as T+24h"; else fail "out-of-window run counted (rc=$rc)"; show; fi +scenario failedrun; jq '.workflow_runs[2].conclusion = "failure"' "$tmp/failedrun/cruns.json" >"$tmp/x" && mv "$tmp/x" "$tmp/failedrun/cruns.json" +run failedrun 200 +if line MISSING "canary t-plus-72h"; then ok "a failed in-window run does not count"; else fail "failed run counted (rc=$rc)"; show; fi + +# 6. Draft release: published-only items wait. +scenario draft; jq '.[0].draft = true | .[0].published_at = null' "$tmp/draft/releases.json" >"$tmp/x" && mv "$tmp/x" "$tmp/draft/releases.json" +run draft 200 +if [[ $rc -eq 0 ]] && line PENDING "RC published as prerelease" && line PENDING "canary manual" && line VERIFIED "validator identity"; then + ok "draft: verified assets, publication-dependent items PENDING" +else fail "draft (rc=$rc)"; show; fi + +# 7. Usage. +if bash "$PRE" --tag v0.4.0 >/dev/null 2>&1; then fail "stable tag accepted"; else ok "non-RC tag refused"; fi + +if [[ $failures -ne 0 ]]; then echo "[rc-preflight-test] $failures failure(s)" >&2; exit 1; fi +echo "rc-preflight_test: all checks passed" From ba5723c7fe4c1f95dd28854f033111c1a3ad0492 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 16:43:15 +0300 Subject: [PATCH 03/38] release: address review of the rc.5 draft and the preflight inventory CHANGELOG 0.4.0-rc.5: drop the readiness-sounding opening, scope the validator byte-identity rule to the recorded tree, state the readiness bindings as enforced (Falco validator binding reported only; an earlier candidate's incident note accepted), and date the "all undecided" note. rc-preflight.sh: - abort (exit 3) on any non-JSON API body, undecodable file contents or failed asset download instead of reporting a status; - a designated release-candidate canary must name the candidate's tag, commit, image digest (from the attested evidence), publication time and rcN artifact/report names, with no other candidate's names left; - canary and campaign runs count only from the time the designation landed on main (PR merge time), and a canary run only if a job after its plan job succeeded; - a rollback drill run stays PENDING: its candidate is not visible here. Tests cover each case. Signed-off-by: ErenAri --- CHANGELOG.md | 34 +++---- scripts/rc-preflight.sh | 169 ++++++++++++++++++++++++----------- scripts/rc-preflight_test.sh | 70 +++++++++++++-- 3 files changed, 196 insertions(+), 77 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b46cd1..b628cf1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,9 +9,8 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a ## [0.4.0-rc.5] - TBD (date set when the tag is cut) -Release candidate for v0.4.0. It closes the supported-path findings that made -v0.4.0-rc.4 a no-go, and pins how release binaries are built and what ships -with them. rc.4 qualification evidence does not carry over to this candidate. +Release candidate for v0.4.0: the changes merged since v0.4.0-rc.4. rc.4 +qualification evidence does not carry over to this candidate. ### Security - `install.sh` binds verification to the requested release: the cosign @@ -81,16 +80,19 @@ with them. rc.4 qualification evidence does not carry over to this candidate. ### Changed - Release build: the static validator is built inside a digest-pinned Ubuntu builder from `validator/build-inputs.lock` (exact package versions from a - fixed snapshot.ubuntu.com state), and the build fails unless the result is - byte-identical to the recorded reproduction of the same validator sources. A - stable release must ship the same validator bytes as the release candidate it - graduates. + fixed snapshot.ubuntu.com state). While the validator sources are the tree the + lock records, the build fails unless the result is byte-identical to that + recorded reproduction. A stable release must ship the same validator bytes as + the release candidate it graduates. - Release gate: each VM gate report is evaluated on its own and the gate fails closed on a missing or malformed report. -- Production readiness: the Falco, canary, rollback, incident and operator - evidence must be bound to the exact candidate (single-object JSON, exact - candidate-evidence hash in the operator note, rollback alias and digest of the - candidate, Falco started after the candidate evidence was generated). +- Production readiness binds more evidence to the exact candidate: inputs must + be single-object JSON; the rollback drill must use the candidate's alias and + image digest and complete after the candidate was built; the operator note + must contain the exact candidate-evidence hash; the Falco run must start after + the candidate was built (whether it ran the candidate's validator is recorded, + not required); the promotion incident note must link exactly one rejected + run, and an earlier candidate's note is accepted with that fact recorded. - CI: top-level workflow tokens are read-only; the hosted compatibility gate is required for `pkg/` and `scripts/` changes; untrusted-input parsers have fuzz coverage. @@ -118,11 +120,11 @@ with them. rc.4 qualification evidence does not carry over to this candidate. (`BPFCOMPAT_DOC_DIR` to change it) when the release publishes one. ### Known limitations -- Distribution decisions recorded in `distribution/policy.json` are all - undecided. This candidate's license bundle and relink kit record facts and - ship texts verbatim; they do not elect a license or state that any license - condition is met. A stable release is refused until the stable-blocking - decisions are recorded. +- When this section was drafted, every decision in `distribution/policy.json` + was undecided. The license bundle and relink kit record facts and ship texts + verbatim; they do not elect a license or state that any license condition is + met. A stable release is refused until the stable-blocking decisions are + recorded. - The prebuilt static validator is published for x86_64 only. - See "Known limitations of the supported path" in `docs/compatibility-contract.md` (artifact-path classification, untrusted diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index 387f635..a4a81d2 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -5,8 +5,8 @@ # # Prints one line per item: VERIFIED, MISSING or PENDING, the item, and the # evidence behind the status. It only reads: GitHub REST GETs through `gh api`, -# `gh release download` of the text file SHA256SUMS, and files in this -# checkout. It never dispatches or reruns a workflow, never writes to GitHub, +# `gh release download` of two text files (SHA256SUMS and +# release-candidate-evidence.json), and files in this checkout. It never dispatches or reruns a workflow, never writes to GitHub, # never executes a downloaded file and never writes operator evidence. # # Status meanings: @@ -14,14 +14,18 @@ # MISSING observed and wrong, or its window has closed without it. # PENDING not observable yet (no tag, no run, operator-held evidence, an # open decision). Elapsed time alone never makes an item VERIFIED. -# Run-level canary/campaign/drill items say only that a run of the right kind -# exists; binding its evidence to the candidate is the job of +# Canary and campaign runs count only when they started after publication and +# after main was designated to this candidate; a canary run counts only if a +# job after its planning job succeeded. A rollback drill run stays PENDING: +# which candidate it drilled is not visible here. Run-level items say only +# that a run of the right kind exists; binding its evidence to the candidate is the job of # scripts/production-readiness-report.sh, which this does not replace. Canary # windows (20-30h, 68-78h after publication) are judged from each run's start # time as a proxy for its plan job, which enforces the exact window itself. # -# Exit: 0 no MISSING, 1 at least one MISSING, 2 usage, 3 a lookup failed (the -# report is then incomplete and nothing is inferred from the failure). +# Exit: 0 no MISSING, 1 at least one MISSING, 2 usage, 3 a lookup, download, +# parse or decode failed (the report is then incomplete and nothing is inferred +# from the failure). # Test hook: BPFCOMPAT_PREFLIGHT_NOW= fixes "now". set -euo pipefail @@ -48,19 +52,25 @@ say() { # printf '%-8s %-44s %s\n' "$1" "$2" "$3" [[ "$1" != MISSING ]] || missing=1 } -# fetch : GET into . Returns 0, or 4 for HTTP 404; any other -# failure aborts the whole report (exit 3). Runs in the main shell, never in -# $(...), so the abort cannot be mistaken for "not found". +# fetch : GET JSON into . Returns 0, or 4 for HTTP 404; any +# other failure, or a body that is not JSON, aborts the whole report (exit 3). +# Runs in the main shell, never in $(...), so the abort cannot be mistaken for +# "not found". +abort() { echo "rc-preflight: $*; report incomplete" >&2; exit 3; } fetch() { - if gh api "$1" >"$2" 2>"$tmp/err"; then return 0; fi + if gh api "$1" >"$2" 2>"$tmp/err"; then + jq -e . "$2" >/dev/null 2>&1 || abort "GET $1 returned a body that is not JSON" + return 0 + fi if grep -q 'HTTP 404' "$tmp/err"; then return 4; fi - echo "rc-preflight: lookup failed: GET $1: $(head -1 "$tmp/err")" >&2 - exit 3 + abort "lookup failed: GET $1: $(head -1 "$tmp/err")" } # need : like fetch, but a 404 is a lookup failure too. -need() { - fetch "$1" "$2" || { echo "rc-preflight: lookup failed: GET $1: not found" >&2; exit 3; } -} +need() { fetch "$1" "$2" || abort "lookup failed: GET $1: not found"; } +# decode : base64 file contents from the contents API. +decode() { jq -r .content "$1" | base64 -d >"$2" 2>/dev/null && [[ -s "$2" ]] || abort "cannot decode file contents ($1)"; } +# asset : download one release asset (text only, never executed). +asset() { gh release download "$tag" --repo "$repo" --pattern "$1" --dir "$2" >/dev/null 2>"$tmp/err" || abort "cannot download $1 from the $tag release: $(head -1 "$tmp/err")"; } epoch() { date -u -d "$1" +%s; } echo "rc-preflight: ${repo} ${tag} at $(date -u -d "@$now" +%Y-%m-%dT%H:%M:%SZ) (read-only)" @@ -131,7 +141,7 @@ rel="$(jq -c --arg t "$tag" '[.[] | select(.tag_name == $t)] | first // empty' " required=(bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-validator-static-linux-amd64 bpfcompat-third-party-licenses.tar.gz bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json) signed=(SHA256SUMS SHA256SUMS.sig SHA256SUMS.crt bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt release-candidate-evidence.json) -published_at="" +published_at="" digest="" if [[ -z "$rel" ]]; then for item in "release staged with required assets" "SHA256SUMS lists required assets" "validator identity" \ "optional assets match tagged policy" "RC published as prerelease"; do say PENDING "$item" "no release for $tag yet"; done @@ -141,7 +151,8 @@ else for a in "${required[@]}" "${signed[@]}"; do grep -qxF "$a" <<<"$names" || absent+=("$a"); done if [[ ${#absent[@]} -eq 0 ]]; then say VERIFIED "release staged with required assets" "$(jq '.assets | length' <<<"$rel") assets" else say MISSING "release staged with required assets" "absent: ${absent[*]}"; fi - if grep -qxF SHA256SUMS <<<"$names" && gh release download "$tag" --repo "$repo" --pattern SHA256SUMS --dir "$tmp/sums" >/dev/null 2>"$tmp/err"; then + if grep -qxF SHA256SUMS <<<"$names"; then + asset SHA256SUMS "$tmp/sums" sums="$tmp/sums/SHA256SUMS" unlisted=() for a in "${required[@]}"; do awk -v f="$a" '$2 == f && $1 ~ /^[0-9a-f]{64}$/ { n++ } END { exit n != 1 }' "$sums" || unlisted+=("$a"); done @@ -150,8 +161,8 @@ else vs="$(awk '$2 == "bpfcompat-validator-static-linux-amd64" { print $1 }' "$sums")" if [[ -n "$vs" && "$vs" == "$expected_validator" ]]; then say VERIFIED "validator identity" "$vs (validator/build-inputs.lock reproduces=)" else say MISSING "validator identity" "SHA256SUMS ${vs:-none}, lock expects $expected_validator"; fi - if fetch "repos/${repo}/contents/distribution/policy.json?ref=${tag}" "$tmp/policy-contents.json" && - jq -r .content "$tmp/policy-contents.json" | base64 -d >"$tmp/policy.json" 2>/dev/null && [[ -s "$tmp/policy.json" ]]; then + if fetch "repos/${repo}/contents/distribution/policy.json?ref=${tag}" "$tmp/policy-contents.json"; then + decode "$tmp/policy-contents.json" "$tmp/policy.json" bad="" for pair in "ship_link_inputs_asset:bpfcompat-validator-link-inputs-linux-amd64.tar.gz" "ship_native_sources_asset:bpfcompat-validator-native-sources.tar"; do d="${pair%%:*}" f="${pair#*:}" @@ -166,62 +177,114 @@ else fi else for item in "SHA256SUMS lists required assets" "validator identity" "optional assets match tagged policy"; do - say MISSING "$item" "SHA256SUMS is not downloadable from the $tag release" + say MISSING "$item" "the $tag release has no SHA256SUMS asset" done fi draft="$(jq -r .draft <<<"$rel")" prerel="$(jq -r .prerelease <<<"$rel")" if [[ "$draft" == true ]]; then say PENDING "RC published as prerelease" "draft (promotion pending)" - elif [[ "$prerel" == true ]]; then published_at="$(jq -r .published_at <<<"$rel")"; say VERIFIED "RC published as prerelease" "published $published_at" + elif [[ "$prerel" == true ]]; then + published_at="$(jq -r .published_at <<<"$rel")" + say VERIFIED "RC published as prerelease" "published $published_at" + digest="" + if grep -qxF release-candidate-evidence.json <<<"$names"; then + asset release-candidate-evidence.json "$tmp/ev" + digest="$(jq -r '.image // empty' "$tmp/ev/release-candidate-evidence.json" 2>/dev/null | sed -n 's/.*@\(sha256:[0-9a-f]\{64\}\)$/\1/p')" + fi else say MISSING "RC published as prerelease" "published but not marked prerelease"; fi fi # ------------------------------------------------------------ designation and qualification +# Runs only count when they started after publication AND after main was +# designated to this candidate; before that, the workflows still named the +# previous candidate. +rcnum="${tag##*-rc.}" if [[ -z "$published_at" ]]; then for item in "designated: campaign-candidate.json" "designated: consumer-canary candidate pin" "designated: release-candidate-canary" \ "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)" "rollback drill run"; do say PENDING "$item" "requires a published $tag" done else - main_file() { # - need "repos/${repo}/contents/$1?ref=main" "$tmp/contents.json" - jq -r .content "$tmp/contents.json" | base64 -d >"$2" + main_file() { need "repos/${repo}/contents/$1?ref=main" "$tmp/contents.json"; decode "$tmp/contents.json" "$2"; } + # last_change : epoch at which the newest change to landed on + # main: the merge time of the pull request that brought that commit in, or + # the commit's own date when it was pushed directly. + last_change() { + need "repos/${repo}/commits?sha=main&path=$1&per_page=1" "$tmp/commits.json" + local sha d + sha="$(jq -r '.[0].sha // empty' "$tmp/commits.json")" + [[ -n "$sha" ]] || abort "no main commit touches $1" + need "repos/${repo}/commits/${sha}/pulls" "$tmp/pulls.json" + d="$(jq -r '[.[] | select(.merged_at != null and .base.ref == "main") | .merged_at] | max // empty' "$tmp/pulls.json")" + [[ -n "$d" ]] || d="$(jq -r '.[0].commit.committer.date' "$tmp/commits.json")" + epoch "$d" } - main_file .github/campaign-candidate.json "$tmp/cc.json"; cc="$(cat "$tmp/cc.json")" - if [[ "$(jq -r .tag <<<"$cc")" == "$tag" && "$(jq -r .commit_sha <<<"$cc")" == "$commit" ]]; then say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit" - else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag <<<"$cc") (retarget after verification)"; fi + ok_designations=0 + main_file .github/campaign-candidate.json "$tmp/cc.json" + jq -e . "$tmp/cc.json" >/dev/null 2>&1 || abort "campaign-candidate.json on main is not JSON" + if [[ "$(jq -r .tag "$tmp/cc.json")" == "$tag" && "$(jq -r .commit_sha "$tmp/cc.json")" == "$commit" ]]; then + say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag "$tmp/cc.json") (retarget after verification)"; fi main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" pin="$(grep -A1 'bpfcompat-pin: candidate' "$tmp/consumer.yml" | tail -1)" - if [[ "$pin" == *"@${commit} # ${tag}"* ]]; then say VERIFIED "designated: consumer-canary candidate pin" "pinned to $commit" + if [[ "$pin" == *"@${commit} # ${tag}"* ]]; then say VERIFIED "designated: consumer-canary candidate pin" "pinned to $commit"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: consumer-canary candidate pin" "main pins:${pin##*@}"; fi - main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml"; rcc="$(cat "$tmp/rcc.yml")" - rcc_tag="$(sed -nE 's/^ *CANDIDATE_TAG: *"?([^"]*)"?$/\1/p' <<<"$rcc" | head -1)" - rcc_pub="$(sed -nE 's/^ *CANDIDATE_PUBLISHED_AT: *"?([^"]*)"?$/\1/p' <<<"$rcc" | head -1)" - if [[ "$rcc_tag" == "$tag" && "$rcc_pub" == "$published_at" ]]; then say VERIFIED "designated: release-candidate-canary" "$rcc_tag published $rcc_pub" - else say PENDING "designated: release-candidate-canary" "main has $rcc_tag published ${rcc_pub:-?} (retarget with dated schedules)"; fi + main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml" + envv() { sed -nE "s/^ *$1: *\"?([^\"]*)\"?\$/\\1/p" "$tmp/rcc.yml" | head -1; } + problems="" + [[ "$(envv CANDIDATE_TAG)" == "$tag" ]] || problems+="CANDIDATE_TAG=$(envv CANDIDATE_TAG) " + [[ "$(envv CANDIDATE_COMMIT)" == "$commit" ]] || problems+="CANDIDATE_COMMIT=$(envv CANDIDATE_COMMIT) " + [[ "$(envv CANDIDATE_PUBLISHED_AT)" == "$published_at" ]] || problems+="CANDIDATE_PUBLISHED_AT=$(envv CANDIDATE_PUBLISHED_AT) " + if [[ -z "$digest" ]]; then problems+="(release-candidate-evidence.json names no image digest) " + elif [[ "$(envv CANDIDATE_DIGEST)" != "$digest" ]]; then problems+="CANDIDATE_DIGEST=$(envv CANDIDATE_DIGEST) "; fi + grep -q "rc${rcnum}-canary-component-" "$tmp/rcc.yml" || problems+="no rc${rcnum}-canary-component names " + if grep -Eo 'rc[0-9]+-canary-component-|reports/rc[0-9]+-action' "$tmp/rcc.yml" | grep -vqE "^(rc${rcnum}-|reports/rc${rcnum}-)"; then + problems+="other candidates' artifact/report names remain " + fi + if [[ -z "$problems" ]]; then say VERIFIED "designated: release-candidate-canary" "tag, commit, digest $digest, published $published_at, rc${rcnum} names"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: release-candidate-canary" "main still differs: $problems"; fi pub="$(epoch "$published_at")" - need "repos/${repo}/actions/workflows/release-candidate-canary.yml/runs?created=%3E%3D${published_at}&per_page=100" "$tmp/cruns.json" - canary() { # - local hit - hit="$(jq -r --argjson p "$pub" --arg e "$2" --argjson lo "$3" --argjson hi "$4" ' - [.workflow_runs[] | select(.event == $e and .status == "completed" and .conclusion == "success") - | select((((.run_started_at | fromdateiso8601) - $p) / 3600 | floor) as $h | $h >= $lo and $h <= $hi) - | "\(.id) attempt \(.run_attempt) started \(.run_started_at)"] | first // empty' "$tmp/cruns.json")" - if [[ -n "$hit" ]]; then say VERIFIED "canary $1" "successful $2 run $hit (candidate binding: readiness report)" - elif (( now < pub + ($4 + 1) * 3600 )); then say PENDING "canary $1" "window ${3}-${4}h after publication open or not reached" - else say MISSING "canary $1" "no successful $2 run started ${3}-${4}h after publication"; fi - } - canary manual workflow_dispatch 0 100000 - canary t-plus-24h schedule 20 30 - canary t-plus-72h schedule 68 78 - - need "repos/${repo}/actions/workflows/latest-kernel-compatibility.yml/runs?event=schedule&created=%3E%3D${published_at}&per_page=100" "$tmp/camp.json" - n="$(jq '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success")] | length' "$tmp/camp.json")" - if [[ "$n" -ge 4 ]]; then say VERIFIED "scheduled campaigns (4)" "$n successful scheduled runs since publication (spacing and binding: readiness report)" - else say PENDING "scheduled campaigns (4)" "$n of 4 successful scheduled runs since publication"; fi + if [[ "$ok_designations" -ne 3 ]]; then + for item in "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)"; do + say PENDING "$item" "requires main to be designated to $tag" + done + else + since="$pub" + for f in .github/campaign-candidate.json .github/workflows/consumer-canary.yml .github/workflows/release-candidate-canary.yml; do + c="$(last_change "$f")"; (( c > since )) && since="$c" + done + since_iso="$(date -u -d "@$since" +%Y-%m-%dT%H:%M:%SZ)" + need "repos/${repo}/actions/workflows/release-candidate-canary.yml/runs?created=%3E%3D${since_iso}&per_page=100" "$tmp/cruns.json" + # A canary run counts only if a job after the planning job succeeded: + # that proves the plan job accepted its window (out-of-window runs skip + # every later job and still conclude "success"). + executed() { + need "repos/${repo}/actions/runs/$1/jobs?per_page=100" "$tmp/jobs-$1.json" + jq -e '[.jobs[] | select(.name != "Select RC observation window" and .conclusion == "success")] | length > 0' "$tmp/jobs-$1.json" >/dev/null + } + canary() { # + local id st h hit="" + while IFS=$'\t' read -r id st; do + [[ -n "$id" ]] || continue + h=$(( ($(epoch "$st") - pub) / 3600 )) + (( h >= $3 && h <= $4 + 1 )) || continue + if executed "$id"; then hit="$id started $st"; break; fi + done < <(jq -r --arg e "$2" '.workflow_runs[] | select(.event == $e and .status == "completed" and .conclusion == "success") | "\(.id)\t\(.run_started_at)"' "$tmp/cruns.json") + if [[ -n "$hit" ]]; then say VERIFIED "canary $1" "$2 run $hit, past its plan job (candidate binding: readiness report)" + elif (( now < pub + ($4 + 2) * 3600 )); then say PENDING "canary $1" "window ${3}-${4}h after publication open or not reached" + else say MISSING "canary $1" "no executed $2 run ${3}-${4}h after publication"; fi + } + canary manual workflow_dispatch 0 100000 + canary t-plus-24h schedule 20 30 + canary t-plus-72h schedule 68 78 + need "repos/${repo}/actions/workflows/latest-kernel-compatibility.yml/runs?event=schedule&created=%3E%3D${since_iso}&per_page=100" "$tmp/camp.json" + n="$(jq '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success")] | length' "$tmp/camp.json")" + if [[ "$n" -ge 4 ]]; then say VERIFIED "scheduled campaigns (4)" "$n successful scheduled runs since designation (spacing and binding: readiness report)" + else say PENDING "scheduled campaigns (4)" "$n of 4 successful scheduled runs since designation ($since_iso)"; fi + fi need "repos/${repo}/actions/workflows/rollback-drill.yml/runs?created=%3E%3D${published_at}&per_page=50" "$tmp/drill.json" drill="$(jq -r '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success") | .id] | first // empty' "$tmp/drill.json")" - if [[ -n "$drill" ]]; then say VERIFIED "rollback drill run" "successful run $drill since publication (alias/digest binding: readiness report)" + if [[ -n "$drill" ]]; then say PENDING "rollback drill run" "run $drill succeeded since publication; which candidate it drilled is checked only by the readiness report" else say PENDING "rollback drill run" "no successful run since publication"; fi fi say PENDING "Falco expanded entry" "operator-held; bound by production-readiness-report.sh" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index 4992d1a..c206b18 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -21,9 +21,12 @@ TAG="v${version}" [[ "$TAG" =~ -rc\.[0-9]+$ ]] || { echo "[rc-preflight-test] skip: release.yaml is not a prerelease ($version)"; exit 0; } VSHA="$(awk -F= '$1 == "reproduces" { split($2, a, " "); print a[2] }' "$ROOT_DIR/validator/build-inputs.lock")" C1="$(printf 'a%.0s' {1..40})" C2="$(printf 'b%.0s' {1..40})" TAGOBJ="$(printf 'c%.0s' {1..40})" +DIG="sha256:$(printf 'd%.0s' {1..64})" +RCN="${TAG##*-rc.}" PUB="2026-11-02T10:00:00Z" PUB_EPOCH="$(date -u -d "$PUB" +%s)" at() { date -u -d "@$((PUB_EPOCH + $1 * 3600 + 600))" +%Y-%m-%dT%H:%M:%SZ; } # -> ISO, 10 min past the hour +DESIGNATED="$(date -u -d "@$((PUB_EPOCH + 300))" +%Y-%m-%dT%H:%M:%SZ)" # designation merged 5 min after T0 mkdir -p "$tmp/bin" cat >"$tmp/bin/gh" <<'EOF' @@ -43,8 +46,10 @@ if [[ "$1" == api ]]; then echo "gh: HTTP 404 (unrouted $2)" >&2; exit 1 fi if [[ "$1 $2" == "release download" ]]; then - dir=""; while [[ $# -gt 0 ]]; do [[ "$1" == --dir ]] && dir="$2"; shift; done - mkdir -p "$dir"; cp "$FIX/SHA256SUMS" "$dir/SHA256SUMS"; exit 0 + dir="" pat="" + while [[ $# -gt 0 ]]; do case "$1" in --dir) dir="$2";; --pattern) pat="$2";; esac; shift; done + if [[ -f "$FIX/download-fail" ]]; then echo "gh: HTTP 403" >&2; exit 1; fi + mkdir -p "$dir"; cp "$FIX/$pat" "$dir/$pat"; exit 0 fi echo "gh shim: refusing non-read command: $*" >&2; exit 9 EOF @@ -70,7 +75,12 @@ scenario() { contents "$ROOT_DIR/distribution/policy.json" >"$F/policy.json" jq -n --arg t "$TAG" --arg c "$C1" '{tag: $t, commit_sha: $c}' >"$F/cc-raw.json"; contents "$F/cc-raw.json" >"$F/cc.json" printf ' - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" - printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$TAG" "$PUB" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" + printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n out: reports/rc%s-action.json\n' \ + "$TAG" "$C1" "$DIG" "$PUB" "$RCN" "$RCN" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" + jq -n --arg i "ghcr.io/kernel-guard/bpfcompat@$DIG" '{image: $i}' >"$F/release-candidate-evidence.json" + jq -n '[{sha: "e000000000000000000000000000000000000000", commit: {committer: {date: "2020-01-01T00:00:00Z"}}}]' >"$F/commits.json" + jq -n --arg m "$DESIGNATED" '[{merged_at: $m, base: {ref: "main"}}]' >"$F/pulls.json" + jq -n '{jobs: [{name: "Select RC observation window", conclusion: "success"}, {name: "Clean install", conclusion: "success"}]}' >"$F/jobs.json" jq -n --arg m "$(at 0)" --arg a "$(at 25)" --arg b "$(at 70)" '{workflow_runs: [ {id: 1, run_attempt: 1, event: "workflow_dispatch", status: "completed", conclusion: "success", run_started_at: $m}, {id: 2, run_attempt: 1, event: "schedule", status: "completed", conclusion: "success", run_started_at: $a}, @@ -89,6 +99,9 @@ repos/*/contents/.github/workflows/release-candidate-canary.yml?ref=main 200 rcc repos/*/actions/workflows/release-candidate-canary.yml/runs* 200 cruns.json repos/*/actions/workflows/latest-kernel-compatibility.yml/runs* 200 camp.json repos/*/actions/workflows/rollback-drill.yml/runs* 200 drill.json +repos/*/commits?sha=main&path=* 200 commits.json +repos/*/commits/*/pulls 200 pulls.json +repos/*/actions/runs/*/jobs* 200 jobs.json EOF } route() { sed -i "1i $2" "$tmp/$1/routes"; } # prepend an overriding route @@ -129,12 +142,13 @@ if [[ $rc -eq 0 ]] && line VERIFIED "tag exists" && grep -q "as approved" "$tmp/ line VERIFIED "optional assets match tagged policy" && line VERIFIED "RC published as prerelease" && line VERIFIED "designated: campaign-candidate.json" && line VERIFIED "designated: consumer-canary candidate pin" && line VERIFIED "designated: release-candidate-canary" && line VERIFIED "canary manual" && line VERIFIED "canary t-plus-24h" && - line VERIFIED "canary t-plus-72h" && line VERIFIED "scheduled campaigns" && line VERIFIED "rollback drill run" && + line VERIFIED "canary t-plus-72h" && line VERIFIED "scheduled campaigns" && line PENDING "rollback drill run" && + grep -q 'which candidate it drilled' "$tmp/out" && line PENDING "operator promotion confirmation"; then ok "annotated tag dereferenced; every observable item VERIFIED" else fail "full scenario (rc=$rc)"; show; fi -if awk '!/^api / && !/^release download .*--pattern SHA256SUMS/' "$tmp/gh.log" | grep -q .; then fail "non-read gh call: $(cat "$tmp/gh.log")" -elif grep -qE -- '(-X|--method)' "$tmp/gh.log"; then fail "gh api called with a method" -else ok "only GET api calls and the SHA256SUMS download were made"; fi +if awk '!/^api / && !/^release download .*--pattern (SHA256SUMS|release-candidate-evidence\.json) /' "$tmp/gh.log" | grep -q .; then fail "non-read gh call: $(cat "$tmp/gh.log")" +elif grep -qE -- '(-X|--method|-f |-F |--field|--input)' "$tmp/gh.log"; then fail "gh api called with a method or body" +else ok "only GET api calls and the two text downloads were made"; fi # 4. Wrong identities are MISSING. run full 200 --commit "$C2" @@ -177,7 +191,47 @@ if [[ $rc -eq 0 ]] && line PENDING "RC published as prerelease" && line PENDING ok "draft: verified assets, publication-dependent items PENDING" else fail "draft (rc=$rc)"; show; fi -# 7. Usage. +# 7. Designation must cover every candidate-bound value; runs before it do not count. +scenario stale; sed -i "s/rc${RCN}-canary-component-clean-install-1/rc4-canary-component-clean-install-1/" "$tmp/stale/rcc-raw.yml" +contents "$tmp/stale/rcc-raw.yml" >"$tmp/stale/rcc.json" +run stale 200 +if line PENDING "designated: release-candidate-canary" && grep -q "artifact/report names" "$tmp/out" && line PENDING "canary manual" && + grep -q 'requires main to be designated' "$tmp/out"; then ok "leftover previous-candidate artifact names: not designated, canaries not counted" +else fail "stale designation (rc=$rc)"; show; fi +scenario olddigest; sed -i "s/CANDIDATE_DIGEST: .*/CANDIDATE_DIGEST: sha256:$(printf 'e%.0s' {1..64})/" "$tmp/olddigest/rcc-raw.yml" +contents "$tmp/olddigest/rcc-raw.yml" >"$tmp/olddigest/rcc.json" +run olddigest 200 +if line PENDING "designated: release-candidate-canary" && grep -q 'CANDIDATE_DIGEST=' "$tmp/out"; then ok "previous candidate digest left in the canary: not designated" +else fail "stale digest (rc=$rc)"; show; fi +scenario prerun; LATE_DES="$(date -u -d "@$((PUB_EPOCH + 3 * 3600))" +%Y-%m-%dT%H:%M:%SZ)" +jq --arg d "$LATE_DES" '.[0].merged_at = $d' "$tmp/prerun/pulls.json" >"$tmp/x" && mv "$tmp/x" "$tmp/prerun/pulls.json" +# The API filter is what excludes earlier runs: serve only what created>=designation would return. +jq '.workflow_runs |= map(select(.event == "schedule"))' "$tmp/prerun/cruns.json" >"$tmp/prerun/cruns-filtered.json" +route prerun $'repos/*/actions/workflows/release-candidate-canary.yml/runs*\t200\tcruns-filtered.json' +run prerun 200 +if grep -qF "created=%3E%3D${LATE_DES}" "$tmp/gh.log" && line PENDING "canary manual" && line VERIFIED "canary t-plus-24h"; then + ok "runs are listed from the designation merge time, not from publication" +else fail "pre-designation run (rc=$rc)"; show; fi +scenario planonly; jq '.jobs = [.jobs[0], {name: "Clean install", conclusion: "skipped"}]' "$tmp/planonly/jobs.json" >"$tmp/x" && mv "$tmp/x" "$tmp/planonly/jobs.json" +run planonly 200 +if line MISSING "canary t-plus-24h" && line MISSING "canary t-plus-72h"; then ok "a run whose later jobs the plan job skipped does not count" +else fail "plan-only run counted (rc=$rc)"; show; fi + +# 8. Malformed answers abort; nothing is inferred from them. +scenario badjson; printf 'not json' >"$tmp/badjson/releases.json" +run badjson 200 +if [[ $rc -eq 3 ]] && grep -q 'not JSON' "$tmp/out" && ! grep -q 'release staged' "$tmp/out"; then ok "non-JSON API body aborts"; else fail "non-JSON body (rc=$rc)"; show; fi +scenario badb64; printf '{"content": "%%%%not-base64%%%%"}' >"$tmp/badb64/policy.json" +run badb64 200 +if [[ $rc -eq 3 ]] && grep -q 'cannot decode' "$tmp/out"; then ok "undecodable file contents abort"; else fail "bad base64 (rc=$rc)"; show; fi +scenario dlfail; touch "$tmp/dlfail/download-fail" +run dlfail 200 +if [[ $rc -eq 3 ]] && grep -q 'cannot download SHA256SUMS' "$tmp/out" && ! grep -q '^MISSING validator identity' "$tmp/out"; then + ok "a failed asset download aborts instead of reporting MISSING" +else fail "download failure (rc=$rc)"; show; fi + +# 9. Usage. + if bash "$PRE" --tag v0.4.0 >/dev/null 2>&1; then fail "stable tag accepted"; else ok "non-RC tag refused"; fi if [[ $failures -ne 0 ]]; then echo "[rc-preflight-test] $failures failure(s)" >&2; exit 1; fi From bd6f60b594e3e529144897c97ffec7e5ee0fa2a8 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 16:54:55 +0300 Subject: [PATCH 04/38] release: check the canary's Action pin; no silent exits in rc-preflight - A designated release-candidate canary must also use exactly the candidate commit in its published-action job; a pin left on another candidate keeps the designation PENDING. - A missing consumer-canary candidate marker is reported instead of ending the script silently; unparsable candidate evidence aborts with exit 3. - Canary windows accept a run started up to an hour before the window when its plan job accepted it (proven by an executed later job). - CHANGELOG known-limitations wording names the stable-blocking decisions. Tests cover the new cases. Signed-off-by: ErenAri --- CHANGELOG.md | 4 ++-- scripts/rc-preflight.sh | 15 ++++++++++++--- scripts/rc-preflight_test.sh | 19 +++++++++++++++++-- 3 files changed, 31 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b628cf1..641a3a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -120,8 +120,8 @@ qualification evidence does not carry over to this candidate. (`BPFCOMPAT_DOC_DIR` to change it) when the release publishes one. ### Known limitations -- When this section was drafted, every decision in `distribution/policy.json` - was undecided. The license bundle and relink kit record facts and ship texts +- When this section was drafted, every stable-blocking decision in + `distribution/policy.json` was undecided. The license bundle and relink kit record facts and ship texts verbatim; they do not elect a license or state that any license condition is met. A stable release is refused until the stable-blocking decisions are recorded. diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index a4a81d2..e18573e 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -188,7 +188,8 @@ else digest="" if grep -qxF release-candidate-evidence.json <<<"$names"; then asset release-candidate-evidence.json "$tmp/ev" - digest="$(jq -r '.image // empty' "$tmp/ev/release-candidate-evidence.json" 2>/dev/null | sed -n 's/.*@\(sha256:[0-9a-f]\{64\}\)$/\1/p')" + jq -e . "$tmp/ev/release-candidate-evidence.json" >/dev/null 2>&1 || abort "release-candidate-evidence.json of $tag is not JSON" + digest="$(jq -r '.image // empty' "$tmp/ev/release-candidate-evidence.json" | sed -n 's/.*@\(sha256:[0-9a-f]\{64\}\)$/\1/p')" fi else say MISSING "RC published as prerelease" "published but not marked prerelease"; fi fi @@ -225,7 +226,8 @@ else say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag "$tmp/cc.json") (retarget after verification)"; fi main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" - pin="$(grep -A1 'bpfcompat-pin: candidate' "$tmp/consumer.yml" | tail -1)" + pin="$({ grep -A1 'bpfcompat-pin: candidate' "$tmp/consumer.yml" || true; } | tail -1)" + [[ -n "$pin" ]] || pin="@(no '# bpfcompat-pin: candidate' marker on main)" if [[ "$pin" == *"@${commit} # ${tag}"* ]]; then say VERIFIED "designated: consumer-canary candidate pin" "pinned to $commit"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: consumer-canary candidate pin" "main pins:${pin##*@}"; fi main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml" @@ -237,6 +239,11 @@ else if [[ -z "$digest" ]]; then problems+="(release-candidate-evidence.json names no image digest) " elif [[ "$(envv CANDIDATE_DIGEST)" != "$digest" ]]; then problems+="CANDIDATE_DIGEST=$(envv CANDIDATE_DIGEST) "; fi grep -q "rc${rcnum}-canary-component-" "$tmp/rcc.yml" || problems+="no rc${rcnum}-canary-component names " + # The published-action job must use exactly this candidate's commit. + if grep -Eo 'uses: *Kernel-Guard/bpfcompat@[0-9a-f]{40}' "$tmp/rcc.yml" | grep -vqF "@${commit}" || + ! grep -Eq "uses: *Kernel-Guard/bpfcompat@${commit}( |\$)" "$tmp/rcc.yml"; then + problems+="published-action pin is not @${commit} " + fi if grep -Eo 'rc[0-9]+-canary-component-|reports/rc[0-9]+-action' "$tmp/rcc.yml" | grep -vqE "^(rc${rcnum}-|reports/rc${rcnum}-)"; then problems+="other candidates' artifact/report names remain " fi @@ -267,7 +274,9 @@ else while IFS=$'\t' read -r id st; do [[ -n "$id" ]] || continue h=$(( ($(epoch "$st") - pub) / 3600 )) - (( h >= $3 && h <= $4 + 1 )) || continue + # +-1h: the run may start just before its plan job crosses the hour; + # executed() below proves the plan job accepted the window. + (( h >= $3 - 1 && h <= $4 + 1 )) || continue if executed "$id"; then hit="$id started $st"; break; fi done < <(jq -r --arg e "$2" '.workflow_runs[] | select(.event == $e and .status == "completed" and .conclusion == "success") | "\(.id)\t\(.run_started_at)"' "$tmp/cruns.json") if [[ -n "$hit" ]]; then say VERIFIED "canary $1" "$2 run $hit, past its plan job (candidate binding: readiness report)" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index c206b18..2b8254c 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -75,8 +75,8 @@ scenario() { contents "$ROOT_DIR/distribution/policy.json" >"$F/policy.json" jq -n --arg t "$TAG" --arg c "$C1" '{tag: $t, commit_sha: $c}' >"$F/cc-raw.json"; contents "$F/cc-raw.json" >"$F/cc.json" printf ' - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" - printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n out: reports/rc%s-action.json\n' \ - "$TAG" "$C1" "$DIG" "$PUB" "$RCN" "$RCN" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" + printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' \ + "$TAG" "$C1" "$DIG" "$PUB" "$RCN" "$C1" "$TAG" "$RCN" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" jq -n --arg i "ghcr.io/kernel-guard/bpfcompat@$DIG" '{image: $i}' >"$F/release-candidate-evidence.json" jq -n '[{sha: "e000000000000000000000000000000000000000", commit: {committer: {date: "2020-01-01T00:00:00Z"}}}]' >"$F/commits.json" jq -n --arg m "$DESIGNATED" '[{merged_at: $m, base: {ref: "main"}}]' >"$F/pulls.json" @@ -217,6 +217,16 @@ run planonly 200 if line MISSING "canary t-plus-24h" && line MISSING "canary t-plus-72h"; then ok "a run whose later jobs the plan job skipped does not count" else fail "plan-only run counted (rc=$rc)"; show; fi +scenario oldaction; printf ' uses: Kernel-Guard/bpfcompat@%s # v0.4.0-rc.4\n' "$C2" >>"$tmp/oldaction/rcc-raw.yml" +contents "$tmp/oldaction/rcc-raw.yml" >"$tmp/oldaction/rcc.json" +run oldaction 200 +if line PENDING "designated: release-candidate-canary" && grep -q 'published-action pin' "$tmp/out"; then ok "canary Action pin left on another commit: not designated" +else fail "stale action pin (rc=$rc)"; show; fi +scenario nomarker; printf 'jobs: {}\n' >"$tmp/nomarker/consumer-raw.yml"; contents "$tmp/nomarker/consumer-raw.yml" >"$tmp/nomarker/consumer.json" +run nomarker 200 +if line PENDING "designated: consumer-canary candidate pin" && grep -q 'no .# bpfcompat-pin: candidate. marker' "$tmp/out"; then ok "missing pin marker is reported, not a silent exit" +else fail "missing pin marker (rc=$rc)"; show; fi + # 8. Malformed answers abort; nothing is inferred from them. scenario badjson; printf 'not json' >"$tmp/badjson/releases.json" run badjson 200 @@ -230,6 +240,11 @@ if [[ $rc -eq 3 ]] && grep -q 'cannot download SHA256SUMS' "$tmp/out" && ! grep ok "a failed asset download aborts instead of reporting MISSING" else fail "download failure (rc=$rc)"; show; fi +scenario badev; printf 'garbage' >"$tmp/badev/release-candidate-evidence.json" +run badev 200 +if [[ $rc -eq 3 ]] && grep -q 'release-candidate-evidence.json of .* is not JSON' "$tmp/out"; then ok "unparsable candidate evidence aborts" +else fail "bad evidence (rc=$rc)"; show; fi + # 9. Usage. if bash "$PRE" --tag v0.4.0 >/dev/null 2>&1; then fail "stable tag accepted"; else ok "non-RC tag refused"; fi From 115a3745f7697eea061ca3cd35ba19d53934b2aa Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 21:46:17 +0300 Subject: [PATCH 05/38] release: parse active workflow pins in the RC preflight --- CHANGELOG.md | 3 + internal/tools/rcpreflight/main.go | 174 ++++++++++++++++++++++++ internal/tools/rcpreflight/main_test.go | 46 +++++++ scripts/rc-preflight.sh | 18 ++- scripts/rc-preflight_test.sh | 63 ++++++++- 5 files changed, 296 insertions(+), 8 deletions(-) create mode 100644 internal/tools/rcpreflight/main.go create mode 100644 internal/tools/rcpreflight/main_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 641a3a1..52b8cde 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,6 +61,9 @@ qualification evidence does not carry over to this candidate. prefix workspace files with `./`. ### Added +- A read-only RC preflight inventories observed prerequisites without counting + pending qualification as a pass. Candidate Action pins are read from actual + workflow YAML nodes; comments and shell-script text cannot establish a pin. - Releases carry the third-party license material of their binaries as signed assets listed in `SHA256SUMS`: `bpfcompat-third-party-licenses.tar.gz` (verbatim license, notice and patent texts of every linked Go module, the Go diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go new file mode 100644 index 0000000..1702b07 --- /dev/null +++ b/internal/tools/rcpreflight/main.go @@ -0,0 +1,174 @@ +// Command rcpreflight checks active candidate Action pins in workflow YAML. +// It reads local files only; the shell inventory builds it from vendored code. +package main + +import ( + "errors" + "fmt" + "io" + "os" + "strings" + + "gopkg.in/yaml.v3" +) + +const target = "kernel-guard/bpfcompat" +const marker = "bpfcompat-pin: candidate" + +func main() { + if len(os.Args) != 5 { + fmt.Fprintln(os.Stderr, "usage: rcpreflight ") + fmt.Println("PENDING") + return + } + // #nosec G703 -- this local CLI intentionally reads the operator-supplied workflow path; it never writes it. + f, err := os.Open(os.Args[4]) + if err == nil { + defer f.Close() + err = check(f, os.Args[1], os.Args[2], os.Args[3]) + } + if err != nil { + fmt.Fprintln(os.Stderr, "rc-preflight pins:", err) + fmt.Println("PENDING") + return + } + fmt.Println("VERIFIED") +} + +func commentsHave(s, want string) bool { + for _, line := range strings.Split(s, "\n") { + if strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "#")) == want { + return true + } + } + return false +} + +// validate rejects ambiguous keys and aliases rather than guessing which use +// a runner would execute. Comment markers inside run strings are not comments. +func validate(n *yaml.Node) (int, error) { + if n.Kind == yaml.AliasNode { + return 0, fmt.Errorf("YAML aliases require manual verification") + } + count := 0 + for _, s := range []string{n.HeadComment, n.LineComment, n.FootComment} { + for _, line := range strings.Split(s, "\n") { + if commentsHave(line, marker) { + count++ + } + } + } + if n.Kind == yaml.MappingNode { + keys := map[string]bool{} + for i := 0; i < len(n.Content); i += 2 { + k := n.Content[i] + if k.Kind != yaml.ScalarNode || keys[k.Value] { + return 0, fmt.Errorf("non-scalar or duplicate mapping key at line %d", k.Line) + } + keys[k.Value] = true + } + } + for _, child := range n.Content { + c, err := validate(child) + if err != nil { + return 0, err + } + count += c + } + return count, nil +} + +func field(n *yaml.Node, name string) (*yaml.Node, *yaml.Node) { + if n.Kind == yaml.MappingNode { + for i := 0; i < len(n.Content); i += 2 { + if n.Content[i].Value == name { + return n.Content[i], n.Content[i+1] + } + } + } + return nil, nil +} + +func check(r io.Reader, mode, commit, tag string) error { + if mode != "all" && mode != "candidate" { + return fmt.Errorf("unknown mode %q", mode) + } + d := yaml.NewDecoder(r) + var doc yaml.Node + if err := d.Decode(&doc); err != nil { + return err + } + var extra yaml.Node + if err := d.Decode(&extra); !errors.Is(err, io.EOF) { + return fmt.Errorf("expected one YAML document") + } + markers, err := validate(&doc) + if err != nil { + return err + } + if len(doc.Content) != 1 { + return fmt.Errorf("expected workflow mapping") + } + _, jobs := field(doc.Content[0], "jobs") + if jobs == nil || jobs.Kind != yaml.MappingNode { + return fmt.Errorf("expected jobs mapping") + } + if mode == "candidate" && markers != 1 { + return fmt.Errorf("expected exactly one candidate marker, got %d", markers) + } + found := 0 + inspect := func(n *yaml.Node) error { + key, value := field(n, "uses") + if key == nil { + return nil + } + selected := mode == "candidate" && commentsHave(key.HeadComment+"\n"+value.HeadComment, marker) + if value.Kind != yaml.ScalarNode || value.Tag != "!!str" { + return fmt.Errorf("uses must be a string at line %d", value.Line) + } + ref := strings.ToLower(value.Value) + if mode == "all" { + selected = ref == target || strings.HasPrefix(ref, target+"@") || strings.HasPrefix(ref, target+"/") + } + if !selected { + return nil + } + found++ + if ref != target+"@"+commit { + return fmt.Errorf("candidate uses is not @%s at line %d", commit, value.Line) + } + note := strings.Fields(strings.TrimPrefix(value.LineComment, "#")) + if len(note) == 0 || note[0] != tag { + return fmt.Errorf("candidate uses lacks # %s at line %d", tag, value.Line) + } + return nil + } + for i := 1; i < len(jobs.Content); i += 2 { + job := jobs.Content[i] + if job.Kind != yaml.MappingNode { + return fmt.Errorf("expected job mapping") + } + if err := inspect(job); err != nil { + return err + } + _, steps := field(job, "steps") + if steps == nil { + continue + } + if steps.Kind != yaml.SequenceNode { + return fmt.Errorf("expected steps sequence") + } + for _, step := range steps.Content { + if step.Kind != yaml.MappingNode { + return fmt.Errorf("expected step mapping") + } + if err := inspect(step); err != nil { + return err + } + } + } + if found == 0 || (mode == "candidate" && found != 1) { + return fmt.Errorf("expected active candidate uses, got %d", found) + } + return nil +} diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go new file mode 100644 index 0000000..bc072a9 --- /dev/null +++ b/internal/tools/rcpreflight/main_test.go @@ -0,0 +1,46 @@ +package main + +import ( + "strings" + "testing" +) + +func TestActiveWorkflowPins(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag + workflow := func(steps string) string { return "jobs:\n test:\n steps:\n" + steps } + tests := []struct { + name, mode, text string + ok bool + }{ + {"plain", "all", workflow(" - uses: " + good + "\n"), true}, + {"quoted", "all", workflow(" - uses: 'kernel-guard/bpfcompat@" + sha + "' # " + tag + "\n"), true}, + {"marker", "candidate", workflow(" - name: candidate\n # bpfcompat-pin: candidate\n uses: " + good + "\n"), true}, + {"comment-only", "all", workflow(" # - uses: " + good + "\n - run: true\n"), false}, + {"hidden-ref", "all", workflow(" - uses: " + good + "\n - uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"folded-ref", "all", workflow(" - uses: " + good + "\n - uses: >-\n Kernel-Guard/bpfcompat@main\n"), false}, + {"unicode-ref", "all", workflow(" - uses: " + good + "\n - uses: \"Kernel-\\u0047uard/bpfcompat@main\"\n"), false}, + {"run-text", "all", workflow(" - run: |\n uses: " + good + "\n"), false}, + {"run-marker", "candidate", workflow(" - run: |\n # bpfcompat-pin: candidate\n uses: " + good + "\n - uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"flow-stale", "all", workflow(" - uses: " + good + "\n - {uses: 'Kernel-Guard/bpfcompat@main'}\n"), false}, + {"flow-correct", "all", "jobs: {test: {steps: [{uses: 'Kernel-Guard/bpfcompat@" + sha + "'}]}}\n", false}, // missing tag comment + {"quoted-key", "all", workflow(" - uses: " + good + "\n - 'uses': Kernel-Guard/bpfcompat@main\n"), false}, + {"wrong-tag", "all", workflow(" - uses: Kernel-Guard/bpfcompat@" + sha + " # v0.4.0-rc.4\n"), false}, + {"duplicate-key", "all", workflow(" - uses: " + good + "\n uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"duplicate-marker", "candidate", workflow(" - name: first\n # bpfcompat-pin: candidate\n uses: " + good + "\n - name: second\n # bpfcompat-pin: candidate\n uses: " + good + "\n"), false}, + {"alias", "all", workflow(" - uses: &pin Kernel-Guard/bpfcompat@" + sha + " # " + tag + "\n - uses: *pin\n"), false}, + {"multi-document", "all", workflow(" - uses: "+good+"\n") + "---\njobs: {}\n", false}, + {"no-jobs", "all", "run: 'uses: " + good + "'\n", false}, + {"wrong-type", "all", workflow(" - uses: [invalid]\n"), false}, + {"subpath", "all", workflow(" - uses: " + good + "\n - uses: Kernel-Guard/bpfcompat/sub@" + sha + " # " + tag + "\n"), false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), tt.mode, sha, tag) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index e18573e..1b7af77 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -8,6 +8,10 @@ # `gh release download` of two text files (SHA256SUMS and # release-candidate-evidence.json), and files in this checkout. It never dispatches or reruns a workflow, never writes to GitHub, # never executes a downloaded file and never writes operator evidence. +# Active workflow pins are parsed by a helper compiled from this checkout's +# Go sources and vendored YAML library into the temporary directory. Its build +# disables toolchain/dependency downloads; Go meeting go.mod's version must +# already be on PATH (as in CI's setup-go environment). # # Status meanings: # VERIFIED observed directly (a ref, a run, a file, a hash) and as required. @@ -226,10 +230,12 @@ else say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag "$tmp/cc.json") (retarget after verification)"; fi main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" - pin="$({ grep -A1 'bpfcompat-pin: candidate' "$tmp/consumer.yml" || true; } | tail -1)" - [[ -n "$pin" ]] || pin="@(no '# bpfcompat-pin: candidate' marker on main)" - if [[ "$pin" == *"@${commit} # ${tag}"* ]]; then say VERIFIED "designated: consumer-canary candidate pin" "pinned to $commit"; ok_designations=$((ok_designations + 1)) - else say PENDING "designated: consumer-canary candidate pin" "main pins:${pin##*@}"; fi + command -v go >/dev/null || abort "Go is required to parse active workflow pins" + (cd "$ROOT" && GOTOOLCHAIN=local GOPROXY=off GOSUMDB=off go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || abort "cannot build vendored workflow-pin parser; put Go meeting go.mod's version on PATH" + pin_result="$("$tmp/pincheck" candidate "$commit" "$tag" "$tmp/consumer.yml")" || abort "workflow-pin parser failed" + if [[ "$pin_result" == VERIFIED ]]; then + say VERIFIED "designated: consumer-canary candidate pin" "one active candidate marker pinned to $commit # $tag"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: consumer-canary candidate pin" "requires one '# bpfcompat-pin: candidate' marker followed by an active @$commit # $tag uses scalar"; fi main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml" envv() { sed -nE "s/^ *$1: *\"?([^\"]*)\"?\$/\\1/p" "$tmp/rcc.yml" | head -1; } problems="" @@ -240,8 +246,8 @@ else elif [[ "$(envv CANDIDATE_DIGEST)" != "$digest" ]]; then problems+="CANDIDATE_DIGEST=$(envv CANDIDATE_DIGEST) "; fi grep -q "rc${rcnum}-canary-component-" "$tmp/rcc.yml" || problems+="no rc${rcnum}-canary-component names " # The published-action job must use exactly this candidate's commit. - if grep -Eo 'uses: *Kernel-Guard/bpfcompat@[0-9a-f]{40}' "$tmp/rcc.yml" | grep -vqF "@${commit}" || - ! grep -Eq "uses: *Kernel-Guard/bpfcompat@${commit}( |\$)" "$tmp/rcc.yml"; then + pin_result="$("$tmp/pincheck" all "$commit" "$tag" "$tmp/rcc.yml")" || abort "workflow-pin parser failed" + if [[ "$pin_result" != VERIFIED ]]; then problems+="published-action pin is not @${commit} " fi if grep -Eo 'rc[0-9]+-canary-component-|reports/rc[0-9]+-action' "$tmp/rcc.yml" | grep -vqE "^(rc${rcnum}-|reports/rc${rcnum}-)"; then diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index 2b8254c..33c5537 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -74,7 +74,7 @@ scenario() { printf '%s bpfcompat-validator-static-linux-amd64\n' "$VSHA"; } >"$F/SHA256SUMS" contents "$ROOT_DIR/distribution/policy.json" >"$F/policy.json" jq -n --arg t "$TAG" --arg c "$C1" '{tag: $t, commit_sha: $c}' >"$F/cc-raw.json"; contents "$F/cc-raw.json" >"$F/cc.json" - printf ' - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" + printf 'jobs:\n test:\n steps:\n - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' \ "$TAG" "$C1" "$DIG" "$PUB" "$RCN" "$C1" "$TAG" "$RCN" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" jq -n --arg i "ghcr.io/kernel-guard/bpfcompat@$DIG" '{image: $i}' >"$F/release-candidate-evidence.json" @@ -224,9 +224,68 @@ if line PENDING "designated: release-candidate-canary" && grep -q 'published-act else fail "stale action pin (rc=$rc)"; show; fi scenario nomarker; printf 'jobs: {}\n' >"$tmp/nomarker/consumer-raw.yml"; contents "$tmp/nomarker/consumer-raw.yml" >"$tmp/nomarker/consumer.json" run nomarker 200 -if line PENDING "designated: consumer-canary candidate pin" && grep -q 'no .# bpfcompat-pin: candidate. marker' "$tmp/out"; then ok "missing pin marker is reported, not a silent exit" +if line PENDING "designated: consumer-canary candidate pin" && grep -q "requires one '# bpfcompat-pin: candidate' marker" "$tmp/out"; then ok "missing pin marker is reported, not a silent exit" else fail "missing pin marker (rc=$rc)"; show; fi +# An actual stale action cannot be hidden by a correct commented line or an +# alternative YAML scalar spelling. Keep these end-to-end with the fake API. +for variant in commented commentonly quoted lowercase tabbed ref subpath wrongtag; do + scenario "pin-$variant" + file="$F/rcc-raw.yml" + case "$variant" in + commented) + sed -i "s/uses: Kernel-Guard\/bpfcompat@$C1/# uses: Kernel-Guard\/bpfcompat@$C1/" "$file" + printf ' uses: Kernel-Guard/bpfcompat@v0.4.0-rc.4\n' >>"$file" ;; + commentonly) sed -i "s/uses: Kernel-Guard/# uses: Kernel-Guard/" "$file" ;; + quoted) printf ' uses: "Kernel-Guard/bpfcompat@%s" # %s\n' "$C2" "$TAG" >>"$file" ;; + lowercase) printf ' uses: kernel-guard/bpfcompat@%s # %s\n' "$C2" "$TAG" >>"$file" ;; + tabbed) printf ' uses:\tKernel-Guard/bpfcompat@%s # %s\n' "$C2" "$TAG" >>"$file" ;; + ref) printf ' uses: Kernel-Guard/bpfcompat@main\n' >>"$file" ;; + subpath) printf ' uses: Kernel-Guard/bpfcompat/sub@%s # %s\n' "$C1" "$TAG" >>"$file" ;; + wrongtag) sed -i "s/# $TAG/# v9.9.9-rc.1/" "$file" ;; + esac + contents "$file" >"$F/rcc.json" + run "pin-$variant" 200 + if line PENDING "designated: release-candidate-canary" && line PENDING "canary manual"; then + ok "Action $variant cannot establish candidate designation" + else fail "Action $variant falsely designated (rc=$rc)"; show; fi +done +for variant in commented duplicate stale-after-comment; do + scenario "consumer-$variant" + file="$F/consumer-raw.yml" + case "$variant" in + commented) sed -i 's/ uses:/ # uses:/' "$file" ;; + duplicate) cat "$file" >>"$F/second"; cat "$F/second" >>"$file" ;; + stale-after-comment) + sed -i 's/ uses:/ # uses:/' "$file" + printf ' uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C2" "$TAG" >>"$file" ;; + esac + contents "$file" >"$F/consumer.json" + run "consumer-$variant" 200 + if line PENDING "designated: consumer-canary candidate pin" && line PENDING "canary manual"; then + ok "consumer $variant cannot establish candidate designation" + else fail "consumer $variant falsely designated (rc=$rc)"; show; fi +done +scenario quotedgood +sed -i "s|uses: Kernel-Guard/bpfcompat@$C1|uses: 'kernel-guard/bpfcompat@$C1'|" "$F/rcc-raw.yml" +sed -i "s|uses: Kernel-Guard/bpfcompat@$C1|uses: \"Kernel-Guard/bpfcompat@$C1\"|" "$F/consumer-raw.yml" +contents "$F/rcc-raw.yml" >"$F/rcc.json"; contents "$F/consumer-raw.yml" >"$F/consumer.json" +run quotedgood 200 +if line VERIFIED "designated: release-candidate-canary" && line VERIFIED "designated: consumer-canary candidate pin"; then + ok "correct quoted pins and case-insensitive repository names are accepted" +else fail "valid quoted pins refused (rc=$rc)"; show; fi + +# The workflow's plan job can start after the run crosses its hour boundary. +scenario hour19 +jq --arg t "$(at 19)" '.workflow_runs[1].run_started_at = $t' "$F/cruns.json" >"$tmp/x" && mv "$tmp/x" "$F/cruns.json" +run hour19 200 +if line VERIFIED "canary t-plus-24h"; then ok "hour-19 run with successful executed jobs remains observable" +else fail "hour-19 executed run refused (rc=$rc)"; show; fi +jq '.jobs = [.jobs[0], {name: "Clean install", conclusion: "skipped"}]' "$F/jobs.json" >"$tmp/x" && mv "$tmp/x" "$F/jobs.json" +run hour19 200 +if line MISSING "canary t-plus-24h"; then ok "hour-19 plan-only run never becomes VERIFIED" +else fail "hour-19 plan-only run counted (rc=$rc)"; show; fi + # 8. Malformed answers abort; nothing is inferred from them. scenario badjson; printf 'not json' >"$tmp/badjson/releases.json" run badjson 200 From 135889f075ce42e989f0e45801796e31893d3615 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 21:59:22 +0300 Subject: [PATCH 06/38] release: let the RC preflight build its pin parser with a cached toolchain rc-preflight.sh built its workflow-pin parser with GOTOOLCHAIN=local and GOSUMDB=off, so it worked only where Go at go.mod's version (1.26.9) is first on PATH. With an older Go on PATH (go 1.22 on the operator workstation) every post-publication run aborted (exit 3), and rc-preflight_test.sh failed 31 checks outside CI's setup-go image. Keep the build offline (GOPROXY=off, GOWORK=off, -mod=vendor) but leave GOTOOLCHAIN as the caller set it and keep checksum verification, so Go can switch to a go.mod toolchain already in the module cache. Verified with go 1.22 on PATH: the parser builds with go1.26.9 and the build makes no network connections. The helper now exits 2, with nothing on stdout, on a usage error or an unreadable file instead of printing PENDING, so the shell aborts rather than reporting a pin status it never judged. Tests: TestRunExitCodes; rc-preflight_test.sh checks the build environment and that an unusable Go aborts the report. Signed-off-by: ErenAri --- internal/tools/rcpreflight/main.go | 39 +++++++++++++++---------- internal/tools/rcpreflight/main_test.go | 36 +++++++++++++++++++++++ scripts/rc-preflight.sh | 10 ++++--- scripts/rc-preflight_test.sh | 32 +++++++++++++++++++- 4 files changed, 96 insertions(+), 21 deletions(-) diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go index 1702b07..04271fe 100644 --- a/internal/tools/rcpreflight/main.go +++ b/internal/tools/rcpreflight/main.go @@ -15,24 +15,31 @@ import ( const target = "kernel-guard/bpfcompat" const marker = "bpfcompat-pin: candidate" -func main() { - if len(os.Args) != 5 { - fmt.Fprintln(os.Stderr, "usage: rcpreflight ") - fmt.Println("PENDING") - return - } - // #nosec G703 -- this local CLI intentionally reads the operator-supplied workflow path; it never writes it. - f, err := os.Open(os.Args[4]) - if err == nil { - defer f.Close() - err = check(f, os.Args[1], os.Args[2], os.Args[3]) - } +func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } + +// run prints VERIFIED or PENDING and returns 0 for a workflow it could judge. +// A usage error or an unreadable file is not a judgement about the workflow: +// it prints nothing to stdout and returns 2, so the caller aborts instead of +// reporting a status. +func run(args []string, stdout, stderr io.Writer) int { + if len(args) != 4 || (args[0] != "all" && args[0] != "candidate") { + fmt.Fprintln(stderr, "usage: rcpreflight ") + return 2 + } + // #nosec G304 G703 -- this local CLI intentionally reads the operator-supplied workflow path; it never writes it. + f, err := os.Open(args[3]) if err != nil { - fmt.Fprintln(os.Stderr, "rc-preflight pins:", err) - fmt.Println("PENDING") - return + fmt.Fprintln(stderr, "rc-preflight pins:", err) + return 2 + } + defer f.Close() + if err := check(f, args[0], args[1], args[2]); err != nil { + fmt.Fprintln(stderr, "rc-preflight pins:", err) + fmt.Fprintln(stdout, "PENDING") + return 0 } - fmt.Println("VERIFIED") + fmt.Fprintln(stdout, "VERIFIED") + return 0 } func commentsHave(s, want string) bool { diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go index bc072a9..91f29e7 100644 --- a/internal/tools/rcpreflight/main_test.go +++ b/internal/tools/rcpreflight/main_test.go @@ -1,6 +1,8 @@ package main import ( + "bytes" + "os" "strings" "testing" ) @@ -44,3 +46,37 @@ func TestActiveWorkflowPins(t *testing.T) { }) } } + +func TestRunExitCodes(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + dir := t.TempDir() + good := dir + "/good.yml" + stale := dir + "/stale.yml" + if err := os.WriteFile(good, []byte("jobs:\n t:\n steps:\n - uses: Kernel-Guard/bpfcompat@"+sha+" # "+tag+"\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(stale, []byte("jobs:\n t:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\n"), 0o600); err != nil { + t.Fatal(err) + } + tests := []struct { + name string + args []string + code int + out string + }{ + {"verified", []string{"all", sha, tag, good}, 0, "VERIFIED\n"}, + {"pending", []string{"all", sha, tag, stale}, 0, "PENDING\n"}, + {"usage", []string{"all", sha, tag}, 2, ""}, + {"unknown-mode", []string{"some", sha, tag, good}, 2, ""}, + {"unreadable", []string{"all", sha, tag, dir + "/missing.yml"}, 2, ""}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var stdout, stderr bytes.Buffer + if code := run(tt.args, &stdout, &stderr); code != tt.code || stdout.String() != tt.out { + t.Fatalf("run() = %d, %q; want %d, %q (stderr %q)", code, stdout.String(), tt.code, tt.out, stderr.String()) + } + }) + } +} diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index 1b7af77..d5882fd 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -9,9 +9,10 @@ # release-candidate-evidence.json), and files in this checkout. It never dispatches or reruns a workflow, never writes to GitHub, # never executes a downloaded file and never writes operator evidence. # Active workflow pins are parsed by a helper compiled from this checkout's -# Go sources and vendored YAML library into the temporary directory. Its build -# disables toolchain/dependency downloads; Go meeting go.mod's version must -# already be on PATH (as in CI's setup-go environment). +# Go sources and vendored YAML library into the temporary directory (writing +# only Go's build cache). The build never downloads: GOPROXY=off, GOWORK=off, +# -mod=vendor. GOTOOLCHAIN is left as the caller set it, so the Go on PATH, or +# a toolchain at go.mod's version already in the module cache, can build it. # # Status meanings: # VERIFIED observed directly (a ref, a run, a file, a hash) and as required. @@ -231,7 +232,8 @@ else else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag "$tmp/cc.json") (retarget after verification)"; fi main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" command -v go >/dev/null || abort "Go is required to parse active workflow pins" - (cd "$ROOT" && GOTOOLCHAIN=local GOPROXY=off GOSUMDB=off go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || abort "cannot build vendored workflow-pin parser; put Go meeting go.mod's version on PATH" + (cd "$ROOT" && GOWORK=off GOPROXY=off go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || + abort "cannot build the vendored workflow-pin parser without downloads; it needs Go at go.mod's version on PATH or already in the module cache" pin_result="$("$tmp/pincheck" candidate "$commit" "$tag" "$tmp/consumer.yml")" || abort "workflow-pin parser failed" if [[ "$pin_result" == VERIFIED ]]; then say VERIFIED "designated: consumer-canary candidate pin" "one active candidate marker pinned to $commit # $tag"; ok_designations=$((ok_designations + 1)) diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index 33c5537..b3162dc 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -304,7 +304,37 @@ run badev 200 if [[ $rc -eq 3 ]] && grep -q 'release-candidate-evidence.json of .* is not JSON' "$tmp/out"; then ok "unparsable candidate evidence aborts" else fail "bad evidence (rc=$rc)"; show; fi -# 9. Usage. +# 9. Building the pin parser: no downloads, and the caller's toolchain choice +# is kept (an older Go on PATH may switch to a cached go.mod toolchain). +REAL_GO="$(command -v go)" +mkdir -p "$tmp/goshim" "$tmp/gofail" +cat >"$tmp/goshim/go" <}" "\${GOPROXY-}" "\${GOWORK-}" "\$*" >>"$tmp/go.log" +exec "$REAL_GO" "\$@" +SHIM +printf '#!/usr/bin/env bash\necho "go: go.mod requires go >= 9.9 (running go 1.0)" >&2\nexit 1\n' >"$tmp/gofail/go" +chmod +x "$tmp/goshim/go" "$tmp/gofail/go" +scenario gobuild; : >"$tmp/go.log" +set +e +PATH="$tmp/bin:$tmp/goshim:$PATH" FIX="$tmp/gobuild" GH_LOG="$tmp/gh.log" GOTOOLCHAIN="${GOTOOLCHAIN:-auto}" \ + BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + 200 * 3600)) bash "$PRE" --tag "$TAG" >"$tmp/out" 2>&1 +rc=$? +set -e +if [[ $rc -eq 0 ]] && grep -q "^GOTOOLCHAIN=${GOTOOLCHAIN:-auto} GOPROXY=off GOWORK=off ARGS=build -mod=vendor " "$tmp/go.log"; then + ok "parser build keeps the caller's GOTOOLCHAIN, with GOPROXY=off, GOWORK=off and -mod=vendor" +else fail "parser build environment (rc=$rc): $(cat "$tmp/go.log")"; show; fi +scenario gofails +set +e +PATH="$tmp/bin:$tmp/gofail:$PATH" FIX="$tmp/gofails" GH_LOG="$tmp/gh.log" \ + BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + 200 * 3600)) bash "$PRE" --tag "$TAG" >"$tmp/out" 2>&1 +rc=$? +set -e +if [[ $rc -eq 3 ]] && grep -q 'cannot build the vendored workflow-pin parser' "$tmp/out" && ! grep -q 'designated: consumer-canary' "$tmp/out"; then + ok "an unusable Go aborts the report instead of reporting a pin status" +else fail "unusable Go (rc=$rc)"; show; fi + +# 10. Usage. if bash "$PRE" --tag v0.4.0 >/dev/null 2>&1; then fail "stable tag accepted"; else ok "non-RC tag refused"; fi From 334ce96e555f3d905aeeab8329ef89f600b3e1c3 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 22:17:57 +0300 Subject: [PATCH 07/38] release: close review findings on the RC preflight pin parser - A candidate pin on a step or job whose if is literally false (false, ${{ false }}, 0, '', null) never runs, so it is PENDING, not VERIFIED. - The build env pins GONOPROXY and GONOSUMDB to a pattern nothing matches, so a caller GOPRIVATE or GONOPROXY cannot make Go download an unverified toolchain despite GOPROXY=off. - The canary's CANDIDATE_* bindings are read from the workflow-level env node by the helper (env mode), and a job or step env may not rebind them; run text can no longer stand in for them. - Merge keys and case variants of jobs/steps/uses/if/env are rejected. - The commit in a pin must be spelled exactly; the owner/repo match is ASCII case-insensitive only. - Correct the fetch comment and the build comment, which overstated that no network is ever contacted. Signed-off-by: ErenAri --- CHANGELOG.md | 5 +- internal/tools/rcpreflight/main.go | 115 ++++++++++++++++++++++-- internal/tools/rcpreflight/main_test.go | 37 ++++++++ scripts/rc-preflight.sh | 37 +++++--- scripts/rc-preflight_test.sh | 22 +++-- 5 files changed, 187 insertions(+), 29 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52b8cde..095cc15 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,8 +62,9 @@ qualification evidence does not carry over to this candidate. ### Added - A read-only RC preflight inventories observed prerequisites without counting - pending qualification as a pass. Candidate Action pins are read from actual - workflow YAML nodes; comments and shell-script text cannot establish a pin. + pending qualification as a pass. Candidate Action pins and the canary's + candidate env are read from actual workflow YAML nodes; comments, shell-script + text and steps or jobs whose `if` is literally false cannot establish them. - Releases carry the third-party license material of their binaries as signed assets listed in `SHA256SUMS`: `bpfcompat-third-party-licenses.tar.gz` (verbatim license, notice and patent texts of every linked Go module, the Go diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go index 04271fe..52da240 100644 --- a/internal/tools/rcpreflight/main.go +++ b/internal/tools/rcpreflight/main.go @@ -1,4 +1,5 @@ -// Command rcpreflight checks active candidate Action pins in workflow YAML. +// Command rcpreflight checks active candidate Action pins and workflow-level +// env bindings in workflow YAML. // It reads local files only; the shell inventory builds it from vendored code. package main @@ -22,8 +23,8 @@ func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } // it prints nothing to stdout and returns 2, so the caller aborts instead of // reporting a status. func run(args []string, stdout, stderr io.Writer) int { - if len(args) != 4 || (args[0] != "all" && args[0] != "candidate") { - fmt.Fprintln(stderr, "usage: rcpreflight ") + if len(args) != 4 || (args[0] != "all" && args[0] != "candidate" && args[0] != "env") { + fmt.Fprintln(stderr, "usage: rcpreflight \n rcpreflight env ") return 2 } // #nosec G304 G703 -- this local CLI intentionally reads the operator-supplied workflow path; it never writes it. @@ -51,6 +52,11 @@ func commentsHave(s, want string) bool { return false } +// keywords are the workflow keys this checker reads. GitHub matches them +// case-sensitively and has no merge keys, so a case variant or "<<" would be +// ignored here while a YAML tool might honour it: both are rejected. +var keywords = map[string]bool{"jobs": true, "steps": true, "uses": true, "if": true, "env": true} + // validate rejects ambiguous keys and aliases rather than guessing which use // a runner would execute. Comment markers inside run strings are not comments. func validate(n *yaml.Node) (int, error) { @@ -72,6 +78,9 @@ func validate(n *yaml.Node) (int, error) { if k.Kind != yaml.ScalarNode || keys[k.Value] { return 0, fmt.Errorf("non-scalar or duplicate mapping key at line %d", k.Line) } + if k.Value == "<<" || k.Tag == "!!merge" || (keywords[strings.ToLower(k.Value)] && !keywords[k.Value]) { + return 0, fmt.Errorf("merge or case-variant key %q at line %d", k.Value, k.Line) + } keys[k.Value] = true } } @@ -96,8 +105,81 @@ func field(n *yaml.Node, name string) (*yaml.Node, *yaml.Node) { return nil, nil } +func asciiEqualFold(a, b string) bool { + for i := 0; i < len(a); i++ { + if a[i] >= 0x80 { + return false + } + } + return strings.EqualFold(a, b) +} + +// disabled reports whether a job or step can never run: its if is a literal +// false. Conditions that read a context are the runner's to evaluate and are +// left alone (the canary jobs all depend on their plan job's output). +func disabled(n *yaml.Node) (bool, error) { + _, cond := field(n, "if") + if cond == nil { + return false, nil + } + if cond.Kind != yaml.ScalarNode { + return false, fmt.Errorf("if must be a scalar at line %d", cond.Line) + } + v := strings.TrimSpace(cond.Value) + if strings.HasPrefix(v, "${{") && strings.HasSuffix(v, "}}") { + v = strings.TrimSpace(v[3 : len(v)-2]) + } + switch strings.ToLower(v) { + case "false", "0", "''", "null": + return true, nil + } + return false, nil +} + +// checkEnv requires the workflow-level env to bind name to want, and no job or +// step env to rebind it to anything else. +func checkEnv(root, jobs *yaml.Node, name, want string) error { + binding := func(n *yaml.Node) (*yaml.Node, error) { + _, env := field(n, "env") + if env == nil { + return nil, nil + } + if env.Kind != yaml.MappingNode { + return nil, fmt.Errorf("env must be a mapping at line %d", env.Line) + } + _, v := field(env, name) + if v != nil && v.Kind != yaml.ScalarNode { + return nil, fmt.Errorf("%s must be a scalar at line %d", name, v.Line) + } + return v, nil + } + v, err := binding(root) + if err != nil { + return err + } + if v == nil || v.Value != want { + return fmt.Errorf("workflow env %s is not %q", name, want) + } + for i := 1; i < len(jobs.Content); i += 2 { + scopes := []*yaml.Node{jobs.Content[i]} + if _, steps := field(jobs.Content[i], "steps"); steps != nil && steps.Kind == yaml.SequenceNode { + scopes = append(scopes, steps.Content...) + } + for _, n := range scopes { + v, err := binding(n) + if err != nil { + return err + } + if v != nil && v.Value != want { + return fmt.Errorf("%s rebound to %q at line %d", name, v.Value, v.Line) + } + } + } + return nil +} + func check(r io.Reader, mode, commit, tag string) error { - if mode != "all" && mode != "candidate" { + if mode != "all" && mode != "candidate" && mode != "env" { return fmt.Errorf("unknown mode %q", mode) } d := yaml.NewDecoder(r) @@ -120,11 +202,14 @@ func check(r io.Reader, mode, commit, tag string) error { if jobs == nil || jobs.Kind != yaml.MappingNode { return fmt.Errorf("expected jobs mapping") } + if mode == "env" { + return checkEnv(doc.Content[0], jobs, commit, tag) + } if mode == "candidate" && markers != 1 { return fmt.Errorf("expected exactly one candidate marker, got %d", markers) } found := 0 - inspect := func(n *yaml.Node) error { + inspect := func(n *yaml.Node, jobOff bool) error { key, value := field(n, "uses") if key == nil { return nil @@ -141,7 +226,17 @@ func check(r io.Reader, mode, commit, tag string) error { return nil } found++ - if ref != target+"@"+commit { + off, err := disabled(n) + if err != nil { + return err + } + if off || jobOff { + return fmt.Errorf("candidate uses at line %d never runs (if is false)", value.Line) + } + // The owner and repository match ASCII case-insensitively, as GitHub + // resolves them; the commit must be spelled exactly. + repo, sha, _ := strings.Cut(value.Value, "@") + if !asciiEqualFold(repo, target) || sha != commit { return fmt.Errorf("candidate uses is not @%s at line %d", commit, value.Line) } note := strings.Fields(strings.TrimPrefix(value.LineComment, "#")) @@ -155,7 +250,11 @@ func check(r io.Reader, mode, commit, tag string) error { if job.Kind != yaml.MappingNode { return fmt.Errorf("expected job mapping") } - if err := inspect(job); err != nil { + jobOff, err := disabled(job) + if err != nil { + return err + } + if err := inspect(job, jobOff); err != nil { return err } _, steps := field(job, "steps") @@ -169,7 +268,7 @@ func check(r io.Reader, mode, commit, tag string) error { if step.Kind != yaml.MappingNode { return fmt.Errorf("expected step mapping") } - if err := inspect(step); err != nil { + if err := inspect(step, jobOff); err != nil { return err } } diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go index 91f29e7..2bbbc7d 100644 --- a/internal/tools/rcpreflight/main_test.go +++ b/internal/tools/rcpreflight/main_test.go @@ -36,6 +36,17 @@ func TestActiveWorkflowPins(t *testing.T) { {"no-jobs", "all", "run: 'uses: " + good + "'\n", false}, {"wrong-type", "all", workflow(" - uses: [invalid]\n"), false}, {"subpath", "all", workflow(" - uses: " + good + "\n - uses: Kernel-Guard/bpfcompat/sub@" + sha + " # " + tag + "\n"), false}, + {"disabled-step", "all", workflow(" - if: false\n uses: " + good + "\n"), false}, + {"disabled-expression", "all", workflow(" - if: ${{ false }}\n uses: " + good + "\n"), false}, + {"disabled-job", "candidate", "jobs:\n decoy:\n if: ${{ false }}\n steps:\n - name: decoy\n # bpfcompat-pin: candidate\n uses: " + good + "\n real:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\n", false}, + {"disabled-marked-step", "candidate", workflow(" - name: decoy\n if: 0\n # bpfcompat-pin: candidate\n uses: " + good + "\n - uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"conditional-step", "all", workflow(" - if: needs.plan.outputs.should_run == 'true'\n uses: " + good + "\n"), true}, + {"conditional-job", "all", "jobs:\n test:\n if: ${{ needs.plan.outputs.should_run == 'true' }}\n uses: " + good + "\n", true}, + {"merge-key", "all", workflow(" - uses: " + good + "\n - <<: {uses: Kernel-Guard/bpfcompat@main}\n"), false}, + {"case-variant-uses", "all", workflow(" - uses: " + good + "\n - Uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"case-variant-steps", "all", "jobs:\n test:\n Steps:\n - uses: Kernel-Guard/bpfcompat@main\n ok:\n steps:\n - uses: " + good + "\n", false}, + {"kelvin-sign-owner", "all", workflow(" - uses: \"\u212Aernel-Guard/bpfcompat@" + sha + "\" # " + tag + "\n"), false}, + {"uppercase-sha", "all", workflow(" - uses: Kernel-Guard/bpfcompat@" + strings.ToUpper(sha) + " # " + tag + "\n"), false}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { @@ -47,6 +58,31 @@ func TestActiveWorkflowPins(t *testing.T) { } } +func TestWorkflowEnvBindings(t *testing.T) { + head := "env:\n CANDIDATE_TAG: v0.4.0-rc.5\n CANDIDATE_PUBLISHED_AT: \"2026-11-02T10:00:00Z\"\n" + tests := []struct { + name, text, key, want string + ok bool + }{ + {"top-level", head + "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", true}, + {"quoted", head + "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_PUBLISHED_AT", "2026-11-02T10:00:00Z", true}, + {"stale", head + "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.6", false}, + {"run-text-before-stale-env", "jobs:\n x:\n steps:\n - run: |\n CANDIDATE_TAG: v0.4.0-rc.6\nenv:\n CANDIDATE_TAG: v0.4.0-rc.5\n", "CANDIDATE_TAG", "v0.4.0-rc.6", false}, + {"absent", "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"job-shadow", head + "jobs:\n x:\n env:\n CANDIDATE_TAG: v0.4.0-rc.4\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"step-shadow", head + "jobs:\n x:\n steps:\n - run: true\n env:\n CANDIDATE_TAG: v0.4.0-rc.4\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"duplicate", head + " CANDIDATE_TAG: v0.4.0-rc.4\njobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), "env", tt.key, tt.want) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + func TestRunExitCodes(t *testing.T) { sha := strings.Repeat("a", 40) tag := "v0.4.0-rc.5" @@ -69,6 +105,7 @@ func TestRunExitCodes(t *testing.T) { {"pending", []string{"all", sha, tag, stale}, 0, "PENDING\n"}, {"usage", []string{"all", sha, tag}, 2, ""}, {"unknown-mode", []string{"some", sha, tag, good}, 2, ""}, + {"env-mode", []string{"env", "CANDIDATE_TAG", tag, good}, 0, "PENDING\n"}, {"unreadable", []string{"all", sha, tag, dir + "/missing.yml"}, 2, ""}, } for _, tt := range tests { diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index d5882fd..f2efbfd 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -8,11 +8,15 @@ # `gh release download` of two text files (SHA256SUMS and # release-candidate-evidence.json), and files in this checkout. It never dispatches or reruns a workflow, never writes to GitHub, # never executes a downloaded file and never writes operator evidence. -# Active workflow pins are parsed by a helper compiled from this checkout's -# Go sources and vendored YAML library into the temporary directory (writing -# only Go's build cache). The build never downloads: GOPROXY=off, GOWORK=off, -# -mod=vendor. GOTOOLCHAIN is left as the caller set it, so the Go on PATH, or -# a toolchain at go.mod's version already in the module cache, can build it. +# Active workflow pins and the canary's workflow env are parsed by a helper +# compiled from this checkout's Go sources and vendored YAML library into the +# temporary directory (writing only Go's build cache). The build cannot +# download modules or a toolchain: GOPROXY=off, GONOPROXY and GONOSUMDB set to +# a pattern nothing matches (so a caller GOPRIVATE cannot reopen direct +# fetches), GOWORK=off, -mod=vendor. Verifying a cached toolchain may still +# query the checksum database. GOTOOLCHAIN is left as the caller set it, so the +# Go on PATH, or a toolchain at go.mod's version already in the module cache, +# can build it. # # Status meanings: # VERIFIED observed directly (a ref, a run, a file, a hash) and as required. @@ -59,8 +63,9 @@ say() { # } # fetch : GET JSON into . Returns 0, or 4 for HTTP 404; any # other failure, or a body that is not JSON, aborts the whole report (exit 3). -# Runs in the main shell, never in $(...), so the abort cannot be mistaken for -# "not found". +# Status decisions call it in the main shell, so the abort cannot be mistaken +# for "not found"; inside last_change's $(...) the abort still ends the report +# through set -e. abort() { echo "rc-preflight: $*; report incomplete" >&2; exit 3; } fetch() { if gh api "$1" >"$2" 2>"$tmp/err"; then @@ -232,20 +237,26 @@ else else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag "$tmp/cc.json") (retarget after verification)"; fi main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" command -v go >/dev/null || abort "Go is required to parse active workflow pins" - (cd "$ROOT" && GOWORK=off GOPROXY=off go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || + (cd "$ROOT" && GOWORK=off GOPROXY=off GONOPROXY=none.invalid GONOSUMDB=none.invalid go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || abort "cannot build the vendored workflow-pin parser without downloads; it needs Go at go.mod's version on PATH or already in the module cache" pin_result="$("$tmp/pincheck" candidate "$commit" "$tag" "$tmp/consumer.yml")" || abort "workflow-pin parser failed" if [[ "$pin_result" == VERIFIED ]]; then say VERIFIED "designated: consumer-canary candidate pin" "one active candidate marker pinned to $commit # $tag"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: consumer-canary candidate pin" "requires one '# bpfcompat-pin: candidate' marker followed by an active @$commit # $tag uses scalar"; fi main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml" - envv() { sed -nE "s/^ *$1: *\"?([^\"]*)\"?\$/\\1/p" "$tmp/rcc.yml" | head -1; } + # envis : the workflow-level env binds it (parsed, not grepped, + # so run text cannot stand in for it) and no job or step rebinds it. + envis() { + local r + r="$("$tmp/pincheck" env "$1" "$2" "$tmp/rcc.yml" 2>/dev/null)" || abort "workflow-pin parser failed" + [[ "$r" == VERIFIED ]] + } problems="" - [[ "$(envv CANDIDATE_TAG)" == "$tag" ]] || problems+="CANDIDATE_TAG=$(envv CANDIDATE_TAG) " - [[ "$(envv CANDIDATE_COMMIT)" == "$commit" ]] || problems+="CANDIDATE_COMMIT=$(envv CANDIDATE_COMMIT) " - [[ "$(envv CANDIDATE_PUBLISHED_AT)" == "$published_at" ]] || problems+="CANDIDATE_PUBLISHED_AT=$(envv CANDIDATE_PUBLISHED_AT) " + envis CANDIDATE_TAG "$tag" || problems+="CANDIDATE_TAG=(not $tag) " + envis CANDIDATE_COMMIT "$commit" || problems+="CANDIDATE_COMMIT=(not $commit) " + envis CANDIDATE_PUBLISHED_AT "$published_at" || problems+="CANDIDATE_PUBLISHED_AT=(not $published_at) " if [[ -z "$digest" ]]; then problems+="(release-candidate-evidence.json names no image digest) " - elif [[ "$(envv CANDIDATE_DIGEST)" != "$digest" ]]; then problems+="CANDIDATE_DIGEST=$(envv CANDIDATE_DIGEST) "; fi + elif ! envis CANDIDATE_DIGEST "$digest"; then problems+="CANDIDATE_DIGEST=(not $digest) "; fi grep -q "rc${rcnum}-canary-component-" "$tmp/rcc.yml" || problems+="no rc${rcnum}-canary-component names " # The published-action job must use exactly this candidate's commit. pin_result="$("$tmp/pincheck" all "$commit" "$tag" "$tmp/rcc.yml")" || abort "workflow-pin parser failed" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index b3162dc..3190785 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -203,6 +203,14 @@ contents "$tmp/olddigest/rcc-raw.yml" >"$tmp/olddigest/rcc.json" run olddigest 200 if line PENDING "designated: release-candidate-canary" && grep -q 'CANDIDATE_DIGEST=' "$tmp/out"; then ok "previous candidate digest left in the canary: not designated" else fail "stale digest (rc=$rc)"; show; fi +scenario runtext +{ printf 'jobs:\n notes:\n steps:\n - run: |\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$TAG" "$C1" "$DIG" "$PUB" + printf ' x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' "$RCN" "$C1" "$TAG" "$RCN" + printf 'env:\n CANDIDATE_TAG: v0.0.0-rc.1\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$C1" "$DIG" "$PUB"; } >"$tmp/runtext/rcc-raw.yml" +contents "$tmp/runtext/rcc-raw.yml" >"$tmp/runtext/rcc.json" +run runtext 200 +if line PENDING "designated: release-candidate-canary" && grep -q 'CANDIDATE_TAG=' "$tmp/out"; then ok "current values in run text do not hide a stale workflow env" +else fail "run-text env (rc=$rc)"; show; fi scenario prerun; LATE_DES="$(date -u -d "@$((PUB_EPOCH + 3 * 3600))" +%Y-%m-%dT%H:%M:%SZ)" jq --arg d "$LATE_DES" '.[0].merged_at = $d' "$tmp/prerun/pulls.json" >"$tmp/x" && mv "$tmp/x" "$tmp/prerun/pulls.json" # The API filter is what excludes earlier runs: serve only what created>=designation would return. @@ -304,25 +312,27 @@ run badev 200 if [[ $rc -eq 3 ]] && grep -q 'release-candidate-evidence.json of .* is not JSON' "$tmp/out"; then ok "unparsable candidate evidence aborts" else fail "bad evidence (rc=$rc)"; show; fi -# 9. Building the pin parser: no downloads, and the caller's toolchain choice -# is kept (an older Go on PATH may switch to a cached go.mod toolchain). +# 9. Building the pin parser: no module or toolchain download, and the caller's +# toolchain choice is kept (an older Go on PATH may switch to a cached +# go.mod toolchain). A caller GOPRIVATE would otherwise let Go fetch a +# toolchain directly despite GOPROXY=off. REAL_GO="$(command -v go)" mkdir -p "$tmp/goshim" "$tmp/gofail" cat >"$tmp/goshim/go" <}" "\${GOPROXY-}" "\${GOWORK-}" "\$*" >>"$tmp/go.log" +printf 'GOTOOLCHAIN=%s GOPROXY=%s GONOPROXY=%s GONOSUMDB=%s GOWORK=%s ARGS=%s\n' "\${GOTOOLCHAIN-}" "\${GOPROXY-}" "\${GONOPROXY-}" "\${GONOSUMDB-}" "\${GOWORK-}" "\$*" >>"$tmp/go.log" exec "$REAL_GO" "\$@" SHIM printf '#!/usr/bin/env bash\necho "go: go.mod requires go >= 9.9 (running go 1.0)" >&2\nexit 1\n' >"$tmp/gofail/go" chmod +x "$tmp/goshim/go" "$tmp/gofail/go" scenario gobuild; : >"$tmp/go.log" set +e -PATH="$tmp/bin:$tmp/goshim:$PATH" FIX="$tmp/gobuild" GH_LOG="$tmp/gh.log" GOTOOLCHAIN="${GOTOOLCHAIN:-auto}" \ +PATH="$tmp/bin:$tmp/goshim:$PATH" FIX="$tmp/gobuild" GH_LOG="$tmp/gh.log" GOTOOLCHAIN="${GOTOOLCHAIN:-auto}" GOPRIVATE=golang.org GONOPROXY=golang.org/toolchain \ BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + 200 * 3600)) bash "$PRE" --tag "$TAG" >"$tmp/out" 2>&1 rc=$? set -e -if [[ $rc -eq 0 ]] && grep -q "^GOTOOLCHAIN=${GOTOOLCHAIN:-auto} GOPROXY=off GOWORK=off ARGS=build -mod=vendor " "$tmp/go.log"; then - ok "parser build keeps the caller's GOTOOLCHAIN, with GOPROXY=off, GOWORK=off and -mod=vendor" +if [[ $rc -eq 0 ]] && grep -q "^GOTOOLCHAIN=${GOTOOLCHAIN:-auto} GOPROXY=off GONOPROXY=none.invalid GONOSUMDB=none.invalid GOWORK=off ARGS=build -mod=vendor " "$tmp/go.log"; then + ok "parser build keeps the caller's GOTOOLCHAIN and overrides GOPRIVATE/GONOPROXY, with GOPROXY=off, GOWORK=off and -mod=vendor" else fail "parser build environment (rc=$rc): $(cat "$tmp/go.log")"; show; fi scenario gofails set +e From a6b0918f261f512d2308ec265cfd214c8a842fd9 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 22:42:46 +0300 Subject: [PATCH 08/38] release: count only pins that can run in the RC preflight parser - Any if that reads no context and calls no function is a constant; only the literal true runs. This covers every falsy spelling GitHub accepts (-0, 0.0, 0e0, 0x0, NaN, '', null, ...) instead of a fixed list. - A job that needs a job that can never run is skipped unless its if calls always(), failure() or cancelled(); the skip is carried through chains of needs. A needs value that is not a name or a list of names is rejected. - Case variants of workflow keywords (now including needs) are rejected only in the workflow, job and step mappings; env names, inputs and matrix keys are user data. - The canary designation detail shows the value the parser found instead of the expected one, and the build abort names GOSUMDB=off as a cause. Signed-off-by: ErenAri --- CHANGELOG.md | 3 +- internal/tools/rcpreflight/main.go | 148 ++++++++++++++++++------ internal/tools/rcpreflight/main_test.go | 87 ++++++++++++++ scripts/rc-preflight.sh | 20 ++-- scripts/rc-preflight_test.sh | 3 +- 5 files changed, 215 insertions(+), 46 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 095cc15..3f9c0a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,7 +64,8 @@ qualification evidence does not carry over to this candidate. - A read-only RC preflight inventories observed prerequisites without counting pending qualification as a pass. Candidate Action pins and the canary's candidate env are read from actual workflow YAML nodes; comments, shell-script - text and steps or jobs whose `if` is literally false cannot establish them. + text and steps or jobs that can never run (an `if` that is a constant other + than `true`, or a job that needs one of those) cannot establish them. - Releases carry the third-party license material of their binaries as signed assets listed in `SHA256SUMS`: `bpfcompat-third-party-licenses.tar.gz` (verbatim license, notice and patent texts of every linked Go module, the Go diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go index 52da240..fa00090 100644 --- a/internal/tools/rcpreflight/main.go +++ b/internal/tools/rcpreflight/main.go @@ -8,6 +8,7 @@ import ( "fmt" "io" "os" + "regexp" "strings" "gopkg.in/yaml.v3" @@ -53,9 +54,21 @@ func commentsHave(s, want string) bool { } // keywords are the workflow keys this checker reads. GitHub matches them -// case-sensitively and has no merge keys, so a case variant or "<<" would be -// ignored here while a YAML tool might honour it: both are rejected. -var keywords = map[string]bool{"jobs": true, "steps": true, "uses": true, "if": true, "env": true} +// case-sensitively, so a case variant would be ignored here while a YAML tool +// might honour it. They are checked only where GitHub reads its own keywords +// (the workflow, each job and each step); env names, inputs and matrix keys +// are user data and may be spelled any way. Merge keys ("<<"), which GitHub +// does not support, are rejected everywhere. +var keywords = map[string]bool{"jobs": true, "steps": true, "uses": true, "if": true, "env": true, "needs": true} + +func keywordCase(n *yaml.Node) error { + for i := 0; i < len(n.Content); i += 2 { + if k := n.Content[i].Value; keywords[strings.ToLower(k)] && !keywords[k] { + return fmt.Errorf("case-variant key %q at line %d", k, n.Content[i].Line) + } + } + return nil +} // validate rejects ambiguous keys and aliases rather than guessing which use // a runner would execute. Comment markers inside run strings are not comments. @@ -78,8 +91,8 @@ func validate(n *yaml.Node) (int, error) { if k.Kind != yaml.ScalarNode || keys[k.Value] { return 0, fmt.Errorf("non-scalar or duplicate mapping key at line %d", k.Line) } - if k.Value == "<<" || k.Tag == "!!merge" || (keywords[strings.ToLower(k.Value)] && !keywords[k.Value]) { - return 0, fmt.Errorf("merge or case-variant key %q at line %d", k.Value, k.Line) + if k.Value == "<<" || k.Tag == "!!merge" { + return 0, fmt.Errorf("merge key at line %d", k.Line) } keys[k.Value] = true } @@ -114,26 +127,41 @@ func asciiEqualFold(a, b string) bool { return strings.EqualFold(a, b) } -// disabled reports whether a job or step can never run: its if is a literal -// false. Conditions that read a context are the runner's to evaluate and are -// left alone (the canary jobs all depend on their plan job's output). -func disabled(n *yaml.Node) (bool, error) { +var ( + quoted = regexp.MustCompile(`'(?:[^']|'')*'`) + identifier = regexp.MustCompile(`(?:^|[^A-Za-z0-9_.])([A-Za-z_][A-Za-z0-9_-]*)`) + resume = regexp.MustCompile(`(?i)\b(?:always|failure|cancelled)\s*\(`) +) + +// condition reads the if of a job or step. never is true when the condition +// is a constant other than true: it names no context and calls no function, +// so the runner can only ever skip it. GitHub treats false, 0, -0, ”, null +// and NaN as false and accepts any JSON number form, so every constant except +// the literal true is treated as never running instead of being evaluated +// here. Conditions that read a context are the runner's to evaluate (the +// canary jobs all depend on their plan job's output). resumes is true when +// the condition calls always(), failure() or cancelled(), which let a job run +// after a job it needs was skipped. +func condition(n *yaml.Node) (never, resumes bool, err error) { _, cond := field(n, "if") if cond == nil { - return false, nil + return false, false, nil } if cond.Kind != yaml.ScalarNode { - return false, fmt.Errorf("if must be a scalar at line %d", cond.Line) + return false, false, fmt.Errorf("if must be a scalar at line %d", cond.Line) } v := strings.TrimSpace(cond.Value) if strings.HasPrefix(v, "${{") && strings.HasSuffix(v, "}}") { v = strings.TrimSpace(v[3 : len(v)-2]) } - switch strings.ToLower(v) { - case "false", "0", "''", "null": - return true, nil + for _, m := range identifier.FindAllStringSubmatch(quoted.ReplaceAllString(v, "''"), -1) { + switch strings.ToLower(m[1]) { + case "true", "false", "null", "nan", "infinity": + default: + return false, resume.MatchString(v), nil + } } - return false, nil + return v != "true", false, nil } // checkEnv requires the workflow-level env to bind name to want, and no job or @@ -157,8 +185,11 @@ func checkEnv(root, jobs *yaml.Node, name, want string) error { if err != nil { return err } - if v == nil || v.Value != want { - return fmt.Errorf("workflow env %s is not %q", name, want) + if v == nil { + return fmt.Errorf("absent from the workflow env") + } + if v.Value != want { + return fmt.Errorf("%q in the workflow env", v.Value) } for i := 1; i < len(jobs.Content); i += 2 { scopes := []*yaml.Node{jobs.Content[i]} @@ -171,7 +202,7 @@ func checkEnv(root, jobs *yaml.Node, name, want string) error { return err } if v != nil && v.Value != want { - return fmt.Errorf("%s rebound to %q at line %d", name, v.Value, v.Line) + return fmt.Errorf("rebound to %q at line %d", v.Value, v.Line) } } } @@ -198,10 +229,62 @@ func check(r io.Reader, mode, commit, tag string) error { if len(doc.Content) != 1 { return fmt.Errorf("expected workflow mapping") } + if err := keywordCase(doc.Content[0]); err != nil { + return err + } _, jobs := field(doc.Content[0], "jobs") if jobs == nil || jobs.Kind != yaml.MappingNode { return fmt.Errorf("expected jobs mapping") } + // A job can never run when its own if is a constant other than true, or + // when a job it needs can never run and its if does not resume after a + // skipped need. + never, resumes, needs := map[string]bool{}, map[string]bool{}, map[string][]string{} + for i := 0; i < len(jobs.Content); i += 2 { + name, job := jobs.Content[i].Value, jobs.Content[i+1] + if job.Kind != yaml.MappingNode { + return fmt.Errorf("expected job mapping") + } + if err := keywordCase(job); err != nil { + return err + } + if _, steps := field(job, "steps"); steps != nil && steps.Kind == yaml.SequenceNode { + for _, step := range steps.Content { + if err := keywordCase(step); err != nil { + return err + } + } + } + off, res, err := condition(job) + if err != nil { + return err + } + never[name], resumes[name] = off, res + switch _, n := field(job, "needs"); { + case n == nil: + case n.Kind == yaml.ScalarNode: + needs[name] = []string{n.Value} + case n.Kind == yaml.SequenceNode: + for _, dep := range n.Content { + if dep.Kind != yaml.ScalarNode { + return fmt.Errorf("needs must list job names at line %d", dep.Line) + } + needs[name] = append(needs[name], dep.Value) + } + default: + return fmt.Errorf("needs must be a job name or a list at line %d", n.Line) + } + } + for changed := true; changed; { + changed = false + for name, deps := range needs { + for _, dep := range deps { + if !never[name] && !resumes[name] && never[dep] { + never[name], changed = true, true + } + } + } + } if mode == "env" { return checkEnv(doc.Content[0], jobs, commit, tag) } @@ -209,7 +292,7 @@ func check(r io.Reader, mode, commit, tag string) error { return fmt.Errorf("expected exactly one candidate marker, got %d", markers) } found := 0 - inspect := func(n *yaml.Node, jobOff bool) error { + inspect := func(n *yaml.Node, off bool) error { key, value := field(n, "uses") if key == nil { return nil @@ -226,12 +309,8 @@ func check(r io.Reader, mode, commit, tag string) error { return nil } found++ - off, err := disabled(n) - if err != nil { - return err - } - if off || jobOff { - return fmt.Errorf("candidate uses at line %d never runs (if is false)", value.Line) + if off { + return fmt.Errorf("candidate uses at line %d can never run (a constant if other than true, here or on a needed job)", value.Line) } // The owner and repository match ASCII case-insensitively, as GitHub // resolves them; the commit must be spelled exactly. @@ -245,16 +324,9 @@ func check(r io.Reader, mode, commit, tag string) error { } return nil } - for i := 1; i < len(jobs.Content); i += 2 { - job := jobs.Content[i] - if job.Kind != yaml.MappingNode { - return fmt.Errorf("expected job mapping") - } - jobOff, err := disabled(job) - if err != nil { - return err - } - if err := inspect(job, jobOff); err != nil { + for i := 0; i < len(jobs.Content); i += 2 { + name, job := jobs.Content[i].Value, jobs.Content[i+1] + if err := inspect(job, never[name]); err != nil { return err } _, steps := field(job, "steps") @@ -268,7 +340,11 @@ func check(r io.Reader, mode, commit, tag string) error { if step.Kind != yaml.MappingNode { return fmt.Errorf("expected step mapping") } - if err := inspect(step, jobOff); err != nil { + off, _, err := condition(step) + if err != nil { + return err + } + if err := inspect(step, off || never[name]); err != nil { return err } } diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go index 2bbbc7d..9b3d827 100644 --- a/internal/tools/rcpreflight/main_test.go +++ b/internal/tools/rcpreflight/main_test.go @@ -58,6 +58,92 @@ func TestActiveWorkflowPins(t *testing.T) { } } +// A pin on a step or job that can never run must not count. GitHub treats +// false, 0, -0, ”, null (and NaN) as falsy and accepts any JSON number form. +func TestNeverRunningPins(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag + for _, cond := range []string{"false", "${{ false }}", "-0", "${{ -0 }}", "-0.0", "0.0", "${{ 0.0 }}", "0e0", ".0", "00", + "0x0", "${{ 0x0 }}", "0x00", "NaN", "${{ NaN }}", "''", "\"''\"", "null", "~", "false && true", "!true", "1 == 2"} { + t.Run("step if "+cond, func(t *testing.T) { + text := "jobs:\n test:\n steps:\n - if: " + cond + "\n uses: " + good + "\n" + if err := check(strings.NewReader(text), "all", sha, tag); err == nil { + t.Fatalf("pin under if: %s accepted", cond) + } + }) + t.Run("job if "+cond, func(t *testing.T) { + text := "jobs:\n test:\n if: " + cond + "\n uses: " + good + "\n" + if err := check(strings.NewReader(text), "all", sha, tag); err == nil { + t.Fatalf("job pin under if: %s accepted", cond) + } + }) + } + for _, cond := range []string{"true", "${{ true }}", "needs.plan.outputs.should_run == 'true'", "${{ github.event_name == 'schedule' }}", "always()"} { + t.Run("runs "+cond, func(t *testing.T) { + text := "jobs:\n test:\n steps:\n - if: " + cond + "\n uses: " + good + "\n" + if err := check(strings.NewReader(text), "all", sha, tag); err != nil { + t.Fatalf("pin under if: %s rejected: %v", cond, err) + } + }) + } + needs := []struct { + name, mode, text string + ok bool + }{ + {"needs-disabled", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n steps:\n - uses: " + good + "\n", false}, + {"needs-list-disabled-candidate", "candidate", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: [a]\n steps:\n - name: x\n # bpfcompat-pin: candidate\n uses: " + good + "\n c:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\n", false}, + {"needs-chain", "all", "jobs:\n c:\n needs: b\n uses: " + good + "\n b:\n needs: [a]\n steps:\n - run: true\n a:\n if: ${{ -0 }}\n steps:\n - run: true\n", false}, + {"needs-always", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: ${{ always() }}\n steps:\n - uses: " + good + "\n", true}, + {"needs-running", "all", "jobs:\n plan:\n steps:\n - run: true\n b:\n needs: plan\n if: needs.plan.outputs.should_run == 'true'\n steps:\n - uses: " + good + "\n", true}, + {"needs-mapping", "all", "jobs:\n a:\n steps:\n - run: true\n b:\n needs: {a: x}\n steps:\n - uses: " + good + "\n", false}, + } + for _, tt := range needs { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), tt.mode, sha, tag) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + +// Case variants are rejected only where GitHub reads its own keywords; user +// data such as env names, inputs and matrix keys may use any spelling. +func TestKeywordPositions(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag + tests := []struct { + name, text string + ok bool + }{ + {"env-name", "env:\n ENV: prod\njobs:\n test:\n steps:\n - uses: " + good + "\n", true}, + {"input-name", "jobs:\n test:\n steps:\n - uses: " + good + "\n with:\n Steps: x\n", true}, + {"matrix-key", "jobs:\n test:\n strategy:\n matrix:\n Jobs: [1]\n steps:\n - uses: " + good + "\n", true}, + {"job-needs-variant", "jobs:\n a:\n if: false\n steps:\n - run: true\n test:\n Needs: a\n steps:\n - uses: " + good + "\n", false}, + {"step-if-variant", "jobs:\n test:\n steps:\n - If: false\n uses: " + good + "\n", false}, + {"root-jobs-variant", "Jobs:\n x:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\njobs:\n test:\n steps:\n - uses: " + good + "\n", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), "all", sha, tag) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + +// The env check names what it found, so the report can show the stale value. +func TestEnvBindingDetail(t *testing.T) { + text := "env:\n CANDIDATE_TAG: v0.4.0-rc.4\njobs:\n x:\n steps:\n - run: true\n" + err := check(strings.NewReader(text), "env", "CANDIDATE_TAG", "v0.4.0-rc.5") + if err == nil || !strings.Contains(err.Error(), "v0.4.0-rc.4") { + t.Fatalf("check() = %v, want an error naming the found value", err) + } +} + func TestWorkflowEnvBindings(t *testing.T) { head := "env:\n CANDIDATE_TAG: v0.4.0-rc.5\n CANDIDATE_PUBLISHED_AT: \"2026-11-02T10:00:00Z\"\n" tests := []struct { @@ -71,6 +157,7 @@ func TestWorkflowEnvBindings(t *testing.T) { {"absent", "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, {"job-shadow", head + "jobs:\n x:\n env:\n CANDIDATE_TAG: v0.4.0-rc.4\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, {"step-shadow", head + "jobs:\n x:\n steps:\n - run: true\n env:\n CANDIDATE_TAG: v0.4.0-rc.4\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"step-env-variant", head + "jobs:\n x:\n steps:\n - run: true\n Env:\n CANDIDATE_TAG: v0.4.0-rc.4\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, {"duplicate", head + " CANDIDATE_TAG: v0.4.0-rc.4\njobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, } for _, tt := range tests { diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index f2efbfd..188200c 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -238,25 +238,29 @@ else main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" command -v go >/dev/null || abort "Go is required to parse active workflow pins" (cd "$ROOT" && GOWORK=off GOPROXY=off GONOPROXY=none.invalid GONOSUMDB=none.invalid go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || - abort "cannot build the vendored workflow-pin parser without downloads; it needs Go at go.mod's version on PATH or already in the module cache" + abort "cannot build the vendored workflow-pin parser without downloads; it needs Go at go.mod's version on PATH, or that toolchain already in the module cache with checksum verification on (GOSUMDB not off)" pin_result="$("$tmp/pincheck" candidate "$commit" "$tag" "$tmp/consumer.yml")" || abort "workflow-pin parser failed" if [[ "$pin_result" == VERIFIED ]]; then say VERIFIED "designated: consumer-canary candidate pin" "one active candidate marker pinned to $commit # $tag"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: consumer-canary candidate pin" "requires one '# bpfcompat-pin: candidate' marker followed by an active @$commit # $tag uses scalar"; fi main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml" # envis : the workflow-level env binds it (parsed, not grepped, - # so run text cannot stand in for it) and no job or step rebinds it. + # so run text cannot stand in for it) and no job or step rebinds it. When it + # does not, why= holds what the parser found instead. envis() { local r - r="$("$tmp/pincheck" env "$1" "$2" "$tmp/rcc.yml" 2>/dev/null)" || abort "workflow-pin parser failed" - [[ "$r" == VERIFIED ]] + r="$("$tmp/pincheck" env "$1" "$2" "$tmp/rcc.yml" 2>"$tmp/envis.err")" || + abort "workflow-pin parser failed: $(cat "$tmp/envis.err")" + [[ "$r" == VERIFIED ]] && return 0 + why="$(sed -n 's/^rc-preflight pins: //p' "$tmp/envis.err" | tail -1)" + return 1 } problems="" - envis CANDIDATE_TAG "$tag" || problems+="CANDIDATE_TAG=(not $tag) " - envis CANDIDATE_COMMIT "$commit" || problems+="CANDIDATE_COMMIT=(not $commit) " - envis CANDIDATE_PUBLISHED_AT "$published_at" || problems+="CANDIDATE_PUBLISHED_AT=(not $published_at) " + envis CANDIDATE_TAG "$tag" || problems+="CANDIDATE_TAG=($why) " + envis CANDIDATE_COMMIT "$commit" || problems+="CANDIDATE_COMMIT=($why) " + envis CANDIDATE_PUBLISHED_AT "$published_at" || problems+="CANDIDATE_PUBLISHED_AT=($why) " if [[ -z "$digest" ]]; then problems+="(release-candidate-evidence.json names no image digest) " - elif ! envis CANDIDATE_DIGEST "$digest"; then problems+="CANDIDATE_DIGEST=(not $digest) "; fi + elif ! envis CANDIDATE_DIGEST "$digest"; then problems+="CANDIDATE_DIGEST=($why) "; fi grep -q "rc${rcnum}-canary-component-" "$tmp/rcc.yml" || problems+="no rc${rcnum}-canary-component names " # The published-action job must use exactly this candidate's commit. pin_result="$("$tmp/pincheck" all "$commit" "$tag" "$tmp/rcc.yml")" || abort "workflow-pin parser failed" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index 3190785..0320465 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -201,7 +201,8 @@ else fail "stale designation (rc=$rc)"; show; fi scenario olddigest; sed -i "s/CANDIDATE_DIGEST: .*/CANDIDATE_DIGEST: sha256:$(printf 'e%.0s' {1..64})/" "$tmp/olddigest/rcc-raw.yml" contents "$tmp/olddigest/rcc-raw.yml" >"$tmp/olddigest/rcc.json" run olddigest 200 -if line PENDING "designated: release-candidate-canary" && grep -q 'CANDIDATE_DIGEST=' "$tmp/out"; then ok "previous candidate digest left in the canary: not designated" +if line PENDING "designated: release-candidate-canary" && grep -q "CANDIDATE_DIGEST=(\"sha256:$(printf 'e%.0s' {1..64})\" in the workflow env)" "$tmp/out"; then + ok "previous candidate digest left in the canary: not designated, and the stale value is shown" else fail "stale digest (rc=$rc)"; show; fi scenario runtext { printf 'jobs:\n notes:\n steps:\n - run: |\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$TAG" "$C1" "$DIG" "$PUB" From 87889aab204d1f345297bdcc0fbd5a551e1c9537 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:06:24 +0300 Subject: [PATCH 09/38] release: let the stable PR and tag build pass the stable canary-pin check check-canary-pins.sh always ran check_pin stable "v": the stable pin had to carry that exact version comment and equal the commit the tag resolves to. With no bootstrap for the stable pin (DEF-1), the stable release PR (stable_version = release_version = X.Y.Z, release_channel stable) fails required CI because vX.Y.Z is not a tag yet, and the vX.Y.Z tag build fails too, because the pin would have to name the tagged commit itself, which that commit cannot contain. docs/production-release-process.md already says to repin the stable step once the new stable tag exists; the check made that sequence impossible to follow. Mirror the existing candidate bootstrap for the stable pin. With release_channel stable, while vX.Y.Z positively does not exist, or resolves to the commit under check (the tag build), the stable pin may instead name the most recent published stable release before X.Y.Z: the newest vA.B.C, A.B.C < X.Y.Z, listed by the GitHub releases API as non-draft and non-prerelease, whose tag resolves to the pinned full SHA. As for the candidate, a missing, mutable or short pin fails, and every lookup failure fails closed: an unreachable or malformed tag answer, a pinned tag that does not resolve, a failed or partial API listing, or a malformed line. Once vX.Y.Z exists at any other commit the rule is strict exactly as before, so the first commit after the tag must carry the repin. The prerelease channel's stable pin stays strict, and the candidate rules are unchanged. check-canary-pins_test.sh: section 27 no longer asserts that the stable pin never bootstraps; it now tags the new stable at another commit and asserts the previous stable pin is still refused there. Sections 29-38 cover the stable PR, the tag build (also with the v0.4.0 draft staged and after its publication), the strict commit after the tag and its repin, pins on the new or a later version, an older-than-latest stable, an rc, a draft, a release marked prerelease, a tag without a release, a mismatched SHA, a missing tag, mutable and short refs, a missing pin, each lookup failure, and the unchanged prerelease channel. The gh stub also answers the paginated release listing, for the exact projection the script requests. The new sections are red against the unmodified script and green here. The release process doc spells out the stable PR and tag build phase. Signed-off-by: ErenAri --- docs/production-release-process.md | 17 ++- scripts/check-canary-pins.sh | 122 ++++++++++++++++++- scripts/check-canary-pins_test.sh | 183 +++++++++++++++++++++++++++-- 3 files changed, 306 insertions(+), 16 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index f15d6bd..10d2b2b 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -234,11 +234,18 @@ graduation report and draft assets, records Only this path may update the `X.Y` and `latest` image aliases or GitHub's latest release. +The stable release pull request and the `vX.Y.Z` tag build cannot pin a tag +that does not exist yet or that names their own commit, so they keep the +`# bpfcompat-pin: stable` step on the most recent published (non-draft, +non-prerelease) stable release before `X.Y.Z`. `scripts/check-canary-pins.sh` +accepts that pin only while `vX.Y.Z` is absent or resolves to the commit under +check; a tag or GitHub release lookup it cannot complete fails the check. + Once the new stable tag exists, repin the `# bpfcompat-pin: stable` step in `.github/workflows/consumer-canary.yml` to its commit SHA and update the `# vX.Y.Z` comment beside it. `scripts/check-canary-pins.sh` runs inside the -release-consistency gate and fails the build until that is done, because a -canary left on the previous release stops testing the path the documentation -now sends every reader down. The `# bpfcompat-pin: candidate` step is not -enforced while `release_channel` is `stable`; repin it with the next -`X.Y.Z-rc.1`. +release-consistency gate and fails every later commit until that is done, +because a canary left on the previous release stops testing the path the +documentation now sends every reader down. The `# bpfcompat-pin: candidate` +step is not enforced while `release_channel` is `stable`; repin it with the +next `X.Y.Z-rc.1`. diff --git a/scripts/check-canary-pins.sh b/scripts/check-canary-pins.sh index 09c2602..4f6bd7c 100755 --- a/scripts/check-canary-pins.sh +++ b/scripts/check-canary-pins.sh @@ -23,7 +23,7 @@ # documented consumer path has no canary at all, which is the gap this guard # exists to close, so it is a failure and not a skip. # -# Preparing the next release candidate is the one state the strict rule +# Preparing the next release candidate is one state the strict rule # cannot be met in: the commit that sets release_version to a new RC (and the # tag build of that very commit) cannot pin the canary to a tag that does not # exist yet, or to itself. While v positively does not exist @@ -32,7 +32,18 @@ # (vX.Y.Z-rc.M, M < N) that is a published, non-draft GitHub prerelease and # whose tag resolves to the pinned full SHA. Any lookup or API failure fails # closed. Every later commit is strict again, so the canary must be repinned -# to the new RC before anything else lands. The stable pin rule is unchanged. +# to the new RC before anything else lands. +# +# The stable release is the same state for the stable pin: the PR that sets +# stable_version to X.Y.Z (release_channel `stable`) and the vX.Y.Z tag build +# cannot pin the canary to vX.Y.Z either. With release_channel `stable`, while +# vX.Y.Z positively does not exist or resolves to the commit under check, the +# stable pin may still name the most recent published stable release before +# X.Y.Z -- the newest vA.B.C, A.B.C < X.Y.Z, that is a non-draft, +# non-prerelease GitHub release -- whose tag resolves to the pinned full SHA. +# Any lookup or API failure fails closed, and every later commit is strict +# again: repin the stable step once the new stable tag exists. The prerelease +# channel's stable pin is always strict. # # `candidate` is enforced only while release_channel is `prerelease`. In the # stable channel there is no active prerelease to graduate, release_version @@ -201,8 +212,6 @@ check_pin() { echo "[canary-pins] ${channel} ${expected} ${sha}" } -check_pin stable "v${stable_version}" - # A release published as a non-draft GitHub prerelease. Any failure to ask # (no gh, no token, API error) is a failure, never a pass. canary_repo="Kernel-Guard/bpfcompat" @@ -212,6 +221,111 @@ published_prerelease() { [[ "$state" == "false true" ]] } +# Every GitHub release, one " " line each. Any +# failure to ask (no gh, no token, API error, a page that fails part-way) or +# a line that is not in that shape is a failure, never a pass. +release_states() { + local out line + out="$(gh api --paginate "repos/${canary_repo}/releases?per_page=100" \ + --jq '.[] | "\(.tag_name) \(.draft) \(.prerelease)"' 2>/dev/null)" || return 1 + while IFS= read -r line; do + [[ -z "$line" || "$line" =~ ^[^[:space:]]+\ (true|false)\ (true|false)$ ]] || return 1 + done <<<"$out" + printf '%s\n' "$out" +} + +# version_lt A B: X.Y.Z version A sorts strictly before X.Y.Z version B. +version_lt() { + local -a a b + local i + IFS=. read -r -a a <<<"$1" + IFS=. read -r -a b <<<"$2" + for i in 0 1 2; do + if (( 10#${a[i]} != 10#${b[i]} )); then + (( 10#${a[i]} < 10#${b[i]} )) + return + fi + done + return 1 +} + +# check_bootstrap_stable: while v is being prepared or built, +# the canary may still pin the most recent published stable release before +# it, provided that release's tag resolves to the pinned commit. +check_bootstrap_stable() { + local sha="${pin_sha[stable]:-}" line="${pin_line[stable]:-}" version="${pin_version[stable]:-}" + if [[ -z "$sha" ]]; then + note_fail "${workflow} declares no '# bpfcompat-pin: stable' pin, so the stable consumer path has no canary" + return + fi + if [[ ! "$sha" =~ ^[0-9a-f]{40}$ ]]; then + note_fail "${workflow}:${line} stable pin '${sha}' is not a full 40-character commit SHA" + return + fi + if [[ ! "$version" =~ ^v([0-9]+\.[0-9]+\.[0-9]+)$ ]]; then + note_fail "${workflow}:${line} stable pin '${version:-}' must be an earlier vX.Y.Z stable release while v${stable_version} is being prepared" + return + fi + if ! version_lt "${BASH_REMATCH[1]}" "$stable_version"; then + note_fail "${workflow}:${line} stable pin ${version} is not earlier than v${stable_version}" + return + fi + tag_lookup "$version" + if [[ "$LOOKUP_STATE" != found ]]; then + note_fail "${workflow}:${line} could not resolve ${version} (${LOOKUP_STATE}); failing closed" + return + fi + if [[ "$LOOKUP_SHA" != "$sha" ]]; then + note_fail "${workflow}:${line} stable pin is ${sha} but ${version} is ${LOOKUP_SHA}" + return + fi + local states tag draft prerelease newest="" + if ! states="$(release_states)"; then + note_fail "${workflow}:${line} could not list the GitHub releases to verify ${version} (no gh, no token, API error, or a malformed answer); failing closed" + return + fi + if ! grep -Fxq -- "${version} false false" <<<"$states"; then + note_fail "${workflow}:${line} ${version} is not a verified published stable release (draft, prerelease, or no GitHub release)" + return + fi + while read -r tag draft prerelease; do + [[ "$draft $prerelease" == "false false" && "$tag" =~ ^v([0-9]+\.[0-9]+\.[0-9]+)$ ]] || continue + version_lt "${BASH_REMATCH[1]}" "$stable_version" || continue + if [[ -z "$newest" ]] || version_lt "${newest#v}" "${tag#v}"; then + newest="$tag" + fi + done <<<"$states" + if [[ "$newest" != "$version" ]]; then + note_fail "${workflow}:${line} stable pin ${version} is not the most recent published stable release before v${stable_version} (${newest} is)" + return + fi + echo "[canary-pins] stable bootstrap: v${stable_version} is being prepared; canary still pins published stable ${version} ${sha} (repin after the tag)" +} + +if [[ "$release_channel" == stable && "$stable_version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + tag_lookup "v${stable_version}" + case "$LOOKUP_STATE" in + absent) + # The new stable release is being prepared: its tag positively does not exist. + check_bootstrap_stable + ;; + found) + head_commit="$(git rev-parse -q --verify HEAD 2>/dev/null || true)" + if [[ "$head_commit" =~ ^[0-9a-f]{40}$ && "$LOOKUP_SHA" == "$head_commit" ]]; then + # The tag build of the new stable release itself. + check_bootstrap_stable + else + check_pin stable "v${stable_version}" + fi + ;; + *) + note_fail "cannot determine whether v${stable_version} exists (remote unreachable or malformed answer); failing closed" + ;; + esac +else + check_pin stable "v${stable_version}" +fi + # check_bootstrap_candidate : while v-rc. is # being prepared or built, the canary may still pin an EARLIER rc of the SAME # release that is a published prerelease and resolves to the pinned commit. diff --git a/scripts/check-canary-pins_test.sh b/scripts/check-canary-pins_test.sh index 3e6a4b6..b24da25 100755 --- a/scripts/check-canary-pins_test.sh +++ b/scripts/check-canary-pins_test.sh @@ -234,11 +234,23 @@ fx="$tmp/bootstrap" mkdir -p "$fx/bin" cat >"$fx/bin/gh" <<'STUB' #!/usr/bin/env bash +# $GH_RELEASES holds one " " line per GitHub release. # gh api repos/Kernel-Guard/bpfcompat/releases/tags/ --jq ... -[[ "$1" == api && "$2" == repos/Kernel-Guard/bpfcompat/releases/tags/* ]] || exit 99 -tag="${2##*/}" -line="$(grep -m1 "^${tag} " "$GH_RELEASES" 2>/dev/null)" || exit 1 -echo "${line#* }" +if [[ "$1" == api && "$2" == repos/Kernel-Guard/bpfcompat/releases/tags/* ]]; then + tag="${2##*/}" + line="$(grep -m1 "^${tag} " "$GH_RELEASES" 2>/dev/null)" || exit 1 + echo "${line#* }" + exit 0 +fi +# gh api --paginate repos/Kernel-Guard/bpfcompat/releases?per_page=100 --jq ... +# (the stable bootstrap; only the exact projection it asks for is answered). +if [[ "$1" == api && "$2" == --paginate && + "$3" == "repos/Kernel-Guard/bpfcompat/releases?per_page=100" && + "$4" == --jq && "$5" == '.[] | "\(.tag_name) \(.draft) \(.prerelease)"' ]]; then + cat -- "$GH_RELEASES" 2>/dev/null || exit 1 + exit 0 +fi +exit 99 STUB chmod +x "$fx/bin/gh" @@ -346,16 +358,21 @@ ffail "commit after the rc tag with a stale candidate pin" "candidate pin is com # ...and repinned to the new RC it passes strictly. { pin stable "$S" v0.3.7; pin candidate "$(fsha v0.4.0-rc.4)" v0.4.0-rc.4; } | fcanary fpass "repinned to the new rc after the tag" -# 27. Stable channel is unchanged: wrong stable pin fails, correct passes, -# and no bootstrap applies to the stable pin. +# 27. Stable channel, strict path: whenever v exists at a +# commit other than the one under check, a wrong stable pin fails, the +# correct one passes, and a pin left on the previous stable release fails +# even though that release is published -- the stable bootstrap (29-38) +# never outlives the new stable tag. fmeta 0.3.7 0.3.7 stable { pin stable "$R3" v0.3.7; } | fcanary ffail "stable channel: stale stable pin" "stable pin is ${R3} but v0.3.7 is ${S}" { pin stable "$S" v0.3.7; } | fcanary fpass "stable channel: correct stable pin" +printf 'v0.3.7 false false\n' >>"$fx/releases" +( cd "$fx/work" && gitq tag -a v0.3.8 -m v0.3.8 HEAD^ && gitq push -q origin --tags ) fmeta 0.3.8 0.3.8 stable { pin stable "$S" v0.3.7; } | fcanary -ffail "stable channel: stable pin never bootstraps to an older release" "stable pin is commented 'v0.3.7' but release.yaml says v0.3.8" +ffail "stable channel: previous stable pin once v0.3.8 is tagged at another commit" "stable pin is commented 'v0.3.7' but release.yaml says v0.3.8" # 28. Every workflow step that reaches the canary-pin check -- directly or via # `make production-tech-check` -- must provide GH_TOKEN, or the RC @@ -388,4 +405,156 @@ if [[ -n "$missing" ]]; then fi echo "[canary-pins-test] every workflow path into the canary-pin check provides GH_TOKEN" +# 29-38. The stable bootstrap (DEF-1). The stable release PR sets +# stable_version = release_version = X.Y.Z with release_channel stable +# before vX.Y.Z exists, and the vX.Y.Z tag build checks the very commit +# that tag names; neither can pin the canary to vX.Y.Z. While vX.Y.Z +# positively does not exist, or resolves to the commit under check, the +# stable pin may still name the most recent published stable release +# before X.Y.Z (non-draft, non-prerelease GitHub release); every later +# commit is strict again. Same harness as 16-27. The fixture mirrors the +# real cycle: v0.3.6 and v0.3.7 published, v0.3.8 only a draft, +# v0.4.0-rc.3 a prerelease, and research-v1 a published release whose +# tag is not a version; the API lists them in no particular order. +new_stable_fixture() { + rm -rf "$fx/work" "$fx/origin.git" + git init -q --bare "$fx/origin.git" + git init -q -b main "$fx/work" + mkdir -p "$fx/work/scripts" "$fx/work/.github/workflows" + cp "$ROOT_DIR/$script" "$fx/work/scripts/check-canary-pins.sh" + ( + cd "$fx/work" + gitq remote add origin "$fx/origin.git" + for t in v0.3.6 v0.3.7 v0.3.8 v0.4.0-rc.3; do + gitq commit --allow-empty -m "$t" + gitq tag -a "$t" -m "$t" + done + gitq commit --allow-empty -m "release: v0.4.0 stable metadata" + gitq push -q origin main --tags + ) + printf 'v0.3.6 false false\nv0.4.0-rc.3 false true\nresearch-v1 false false\nv0.3.8 true false\nv0.3.7 false false\n' >"$fx/releases" +} +new_stable_fixture +T6="$(fsha v0.3.6)"; T7="$(fsha v0.3.7)"; T8="$(fsha v0.3.8)"; RC3="$(fsha v0.4.0-rc.3)" + +# 29. The stable PR: v0.4.0 positively absent, the pin stays on v0.3.7. +fmeta 0.4.0 0.4.0 stable +pin stable "$T7" v0.3.7 | fcanary +fpass "stable PR before its tag: most recent published stable accepted" +grep -Fq "[canary-pins] stable bootstrap: v0.4.0 is being prepared; canary still pins published stable v0.3.7 ${T7}" "$fx/out.log" || + { echo "[canary-pins-test] stable bootstrap must be announced" >&2; cat "$fx/out.log" >&2; exit 1; } +{ pin stable "$T7" v0.3.7; pin candidate "$RC3" v0.4.0-rc.3; } | fcanary +fpass "stable PR before its tag, last cycle's candidate pin left in place" +grep -Fq "candidate pin not enforced" "$fx/out.log" || + { echo "[canary-pins-test] candidate pin must stay unenforced in the stable channel" >&2; exit 1; } + +# 30. Naming the new (or a later) release before its tag exists still fails. +pin stable "$T7" v0.4.0 | fcanary +ffail "stable PR pins v0.4.0 before the tag exists" "stable pin v0.4.0 is not earlier than v0.4.0" +pin stable "$T7" v0.4.1 | fcanary +ffail "stable PR pins a later release" "stable pin v0.4.1 is not earlier than v0.4.0" + +# 31. A published stable release that is not the most recent one before v0.4.0. +pin stable "$T6" v0.3.6 | fcanary +ffail "bootstrap pin on an older-than-latest stable" "stable pin v0.3.6 is not the most recent published stable release before v0.4.0 (v0.3.7 is)" + +# 32. Not a published stable release: an rc, a draft, a release marked +# prerelease, a tag with no release at all. +pin stable "$RC3" v0.4.0-rc.3 | fcanary +ffail "bootstrap pin on a release candidate" "stable pin 'v0.4.0-rc.3' must be an earlier vX.Y.Z stable release while v0.4.0 is being prepared" +pin stable "$T7" "" | fcanary +ffail "bootstrap pin with no version comment" "stable pin '' must be an earlier vX.Y.Z stable release" +pin stable "$T8" v0.3.8 | fcanary +ffail "bootstrap pin on a draft release" "v0.3.8 is not a verified published stable release" +pin stable "$T7" v0.3.7 | fcanary +sed -i 's/^v0.3.7 false false$/v0.3.7 false true/' "$fx/releases" +ffail "bootstrap pin on a release marked prerelease" "v0.3.7 is not a verified published stable release" +sed -i '/^v0.3.7 /d' "$fx/releases" +ffail "bootstrap pin on a tag with no GitHub release" "v0.3.7 is not a verified published stable release" +printf 'v0.3.7 false false\n' >>"$fx/releases" +fpass "bootstrap restored once v0.3.7 is published again" + +# 33. The pinned commit must be the one the named tag resolves to, and the +# named tag must resolve at all. +pin stable "$T6" v0.3.7 | fcanary +ffail "bootstrap pin whose tag resolves elsewhere" "stable pin is ${T6} but v0.3.7 is ${T7}" +pin stable "$T7" v0.3.5 | fcanary +ffail "bootstrap pin names a tag that does not exist" "could not resolve v0.3.5 (absent); failing closed" + +# 34. A mutable or short ref is not a pin, and a missing pin is not a skip. +pin stable v0.3.7 v0.3.7 | fcanary +ffail "bootstrap pin is a tag ref" "stable pin 'v0.3.7' is not a full 40-character commit SHA" +pin stable "${T7:0:12}" v0.3.7 | fcanary +ffail "bootstrap pin is a short SHA" "stable pin '${T7:0:12}' is not a full 40-character commit SHA" +pin candidate "$RC3" v0.4.0-rc.3 | fcanary +ffail "bootstrap without a stable pin" "declares no '# bpfcompat-pin: stable' pin" + +# 35. Every lookup failure fails closed: an unanswerable API, an API that +# fails part-way through pagination, a malformed API answer, and a remote +# that cannot say whether v0.4.0 exists. +pin stable "$T7" v0.3.7 | fcanary +mv "$fx/releases" "$fx/releases.off" +ffail "GitHub API unanswerable" "could not list the GitHub releases to verify v0.3.7" +mv "$fx/releases.off" "$fx/releases" +mkdir -p "$fx/partialgh" +cat >"$fx/partialgh/gh" <<'STUB' +#!/usr/bin/env bash +# A first page that looks fine, then a failed request for the next one. +printf 'v0.3.7 false false\n' +exit 1 +STUB +chmod +x "$fx/partialgh/gh" +ffail "GitHub API fails part-way through" "could not list the GitHub releases to verify v0.3.7" "$fx/partialgh" +cp "$fx/releases" "$fx/releases.good" +printf 'v0.3.9 false\n' >>"$fx/releases" +ffail "malformed GitHub API answer" "could not list the GitHub releases to verify v0.3.7" +cp "$fx/releases.good" "$fx/releases" +git -C "$fx/work" remote set-url origin "$fx/no-such-origin.git" +ffail "unreachable remote" "cannot determine whether v0.4.0 exists" +git -C "$fx/work" remote remove origin +ffail "no remote configured" "cannot determine whether v0.4.0 exists" +git -C "$fx/work" remote add origin "$fx/origin.git" +cat >"$fx/badgit/git" <>"$fx/releases" +fpass "tag build with the v0.4.0 draft staged" +sed -i 's/^v0.4.0 true false$/v0.4.0 false false/' "$fx/releases" +fpass "tag build re-checked after v0.4.0 is published" +pin stable "$(git -C "$fx/work" rev-parse HEAD^)" v0.4.0 | fcanary +ffail "tag build pins its parent commit as v0.4.0" "stable pin v0.4.0 is not earlier than v0.4.0" +pin stable "$T6" v0.3.6 | fcanary +ffail "tag build pins an older-than-latest stable" "stable pin v0.3.6 is not the most recent published stable release before v0.4.0 (v0.3.7 is)" + +# 37. Any later commit is strict again: the previous stable pin fails, and the +# repin to the new stable passes on the strict path, not the bootstrap. +( cd "$fx/work" && gitq commit --allow-empty -m "next change on main" ) +pin stable "$T7" v0.3.7 | fcanary +ffail "commit after the stable tag with the previous stable pin" "stable pin is commented 'v0.3.7' but release.yaml says v0.4.0" +pin stable "$(fsha v0.4.0)" v0.4.0 | fcanary +fpass "repinned to the new stable after the tag" +if ! grep -Fq "[canary-pins] stable v0.4.0 $(fsha v0.4.0)" "$fx/out.log" || grep -q "stable bootstrap" "$fx/out.log"; then + echo "[canary-pins-test] the repinned stable must pass strictly" >&2; cat "$fx/out.log" >&2; exit 1 +fi + +# 38. release_channel: prerelease is untouched: its stable pin is always +# strict, even while v does not exist and the pin names +# the most recent published stable before it. +fmeta 0.4.1 0.5.0-rc.1 prerelease +pin stable "$(fsha v0.4.0)" v0.4.0 | fcanary +ffail "prerelease channel: no stable bootstrap" "stable pin is commented 'v0.4.0' but release.yaml says v0.4.1" +fmeta 0.4.1 0.4.1 stable +fpass "the same pin is the stable bootstrap once the channel is stable" + echo "[canary-pins-test] PASS" From 0f8f3b2b61c9c28f268db00c8fa85456fa2d5c04 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:07:21 +0300 Subject: [PATCH 10/38] release: match GitHub's if and needs semantics in the RC preflight parser - An if that is empty or YAML null is no condition (GitHub runs it as success()), and true is matched case-insensitively, so such pins count. - Only always() and !cancelled() let a job run after a skipped need, and a status function inside a quoted string no longer counts. - Job IDs in needs compare case-insensitively; job IDs that differ only by case, needs on an unknown job, on the job itself or in a cycle are rejected, as GitHub rejects them. - The CHANGELOG states that conditions calling functions are left to the runner; doc comments no longer contain '' (gofmt turned it into a curly quote). Signed-off-by: ErenAri --- CHANGELOG.md | 6 +- internal/tools/rcpreflight/main.go | 80 +++++++++++++++++++------ internal/tools/rcpreflight/main_test.go | 18 ++++-- 3 files changed, 80 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f9c0a8..9486f1e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,8 +64,10 @@ qualification evidence does not carry over to this candidate. - A read-only RC preflight inventories observed prerequisites without counting pending qualification as a pass. Candidate Action pins and the canary's candidate env are read from actual workflow YAML nodes; comments, shell-script - text and steps or jobs that can never run (an `if` that is a constant other - than `true`, or a job that needs one of those) cannot establish them. + text and steps or jobs that can never run (an `if` that reads no context, + calls no function and is not `true`, or a job that needs one of those) + cannot establish them. Conditions that call functions are left to the + runner. - Releases carry the third-party license material of their binaries as signed assets listed in `SHA256SUMS`: `bpfcompat-third-party-licenses.tar.gz` (verbatim license, notice and patent texts of every linked Go module, the Go diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go index fa00090..b58ffb8 100644 --- a/internal/tools/rcpreflight/main.go +++ b/internal/tools/rcpreflight/main.go @@ -130,18 +130,20 @@ func asciiEqualFold(a, b string) bool { var ( quoted = regexp.MustCompile(`'(?:[^']|'')*'`) identifier = regexp.MustCompile(`(?:^|[^A-Za-z0-9_.])([A-Za-z_][A-Za-z0-9_-]*)`) - resume = regexp.MustCompile(`(?i)\b(?:always|failure|cancelled)\s*\(`) + resume = regexp.MustCompile(`(?i)\balways\s*\(|!\s*cancelled\s*\(`) ) -// condition reads the if of a job or step. never is true when the condition -// is a constant other than true: it names no context and calls no function, -// so the runner can only ever skip it. GitHub treats false, 0, -0, ”, null -// and NaN as false and accepts any JSON number form, so every constant except -// the literal true is treated as never running instead of being evaluated -// here. Conditions that read a context are the runner's to evaluate (the -// canary jobs all depend on their plan job's output). resumes is true when -// the condition calls always(), failure() or cancelled(), which let a job run -// after a job it needs was skipped. +// condition reads the if of a job or step. An if that is empty or YAML null +// is no condition: GitHub runs it as success(). never is true when the +// condition is a constant other than true: it names no context and calls no +// function, so the runner can only ever skip it. GitHub treats false, 0, -0, +// the empty string, null and NaN as false and accepts any JSON number form, +// so every constant except true (keywords are case-insensitive) is treated as +// never running instead of being evaluated here. A condition that reads a +// context or calls a function is the runner's to evaluate (the canary jobs +// all depend on their plan job's output). resumes is true when the condition +// calls always() or !cancelled(), which can run a job after a job it needs +// was skipped; success() and failure() are both false then. func condition(n *yaml.Node) (never, resumes bool, err error) { _, cond := field(n, "if") if cond == nil { @@ -151,17 +153,21 @@ func condition(n *yaml.Node) (never, resumes bool, err error) { return false, false, fmt.Errorf("if must be a scalar at line %d", cond.Line) } v := strings.TrimSpace(cond.Value) + if cond.Tag == "!!null" || v == "" { + return false, false, nil + } if strings.HasPrefix(v, "${{") && strings.HasSuffix(v, "}}") { v = strings.TrimSpace(v[3 : len(v)-2]) } - for _, m := range identifier.FindAllStringSubmatch(quoted.ReplaceAllString(v, "''"), -1) { + code := quoted.ReplaceAllString(v, "''") + for _, m := range identifier.FindAllStringSubmatch(code, -1) { switch strings.ToLower(m[1]) { case "true", "false", "null", "nan", "infinity": default: - return false, resume.MatchString(v), nil + return false, resume.MatchString(code), nil } } - return v != "true", false, nil + return !strings.EqualFold(v, "true"), false, nil } // checkEnv requires the workflow-level env to bind name to want, and no job or @@ -209,6 +215,37 @@ func checkEnv(root, jobs *yaml.Node, name, want string) error { return nil } +// needsGraph rejects needs that name no job, the job itself, or a cycle. +func needsGraph(needs map[string][]string, jobs map[string]bool) error { + state := map[string]int{} // 0 unvisited, 1 on the current path, 2 done + var visit func(string) error + visit = func(name string) error { + switch state[name] { + case 1: + return fmt.Errorf("needs cycle through job %q", name) + case 2: + return nil + } + state[name] = 1 + for _, dep := range needs[name] { + if _, ok := jobs[dep]; !ok { + return fmt.Errorf("job %q needs unknown job %q", name, dep) + } + if err := visit(dep); err != nil { + return err + } + } + state[name] = 2 + return nil + } + for name := range needs { + if err := visit(name); err != nil { + return err + } + } + return nil +} + func check(r io.Reader, mode, commit, tag string) error { if mode != "all" && mode != "candidate" && mode != "env" { return fmt.Errorf("unknown mode %q", mode) @@ -238,13 +275,17 @@ func check(r io.Reader, mode, commit, tag string) error { } // A job can never run when its own if is a constant other than true, or // when a job it needs can never run and its if does not resume after a - // skipped need. + // skipped need. Job IDs compare case-insensitively, as GitHub resolves + // needs; a needs graph GitHub would reject is rejected here too. never, resumes, needs := map[string]bool{}, map[string]bool{}, map[string][]string{} for i := 0; i < len(jobs.Content); i += 2 { - name, job := jobs.Content[i].Value, jobs.Content[i+1] + name, job := strings.ToLower(jobs.Content[i].Value), jobs.Content[i+1] if job.Kind != yaml.MappingNode { return fmt.Errorf("expected job mapping") } + if _, dup := never[name]; dup { + return fmt.Errorf("job IDs differ only by case at line %d", jobs.Content[i].Line) + } if err := keywordCase(job); err != nil { return err } @@ -263,18 +304,21 @@ func check(r io.Reader, mode, commit, tag string) error { switch _, n := field(job, "needs"); { case n == nil: case n.Kind == yaml.ScalarNode: - needs[name] = []string{n.Value} + needs[name] = []string{strings.ToLower(n.Value)} case n.Kind == yaml.SequenceNode: for _, dep := range n.Content { if dep.Kind != yaml.ScalarNode { return fmt.Errorf("needs must list job names at line %d", dep.Line) } - needs[name] = append(needs[name], dep.Value) + needs[name] = append(needs[name], strings.ToLower(dep.Value)) } default: return fmt.Errorf("needs must be a job name or a list at line %d", n.Line) } } + if err := needsGraph(needs, never); err != nil { + return err + } for changed := true; changed; { changed = false for name, deps := range needs { @@ -325,7 +369,7 @@ func check(r io.Reader, mode, commit, tag string) error { return nil } for i := 0; i < len(jobs.Content); i += 2 { - name, job := jobs.Content[i].Value, jobs.Content[i+1] + name, job := strings.ToLower(jobs.Content[i].Value), jobs.Content[i+1] if err := inspect(job, never[name]); err != nil { return err } diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go index 9b3d827..404185e 100644 --- a/internal/tools/rcpreflight/main_test.go +++ b/internal/tools/rcpreflight/main_test.go @@ -58,14 +58,16 @@ func TestActiveWorkflowPins(t *testing.T) { } } -// A pin on a step or job that can never run must not count. GitHub treats -// false, 0, -0, ”, null (and NaN) as falsy and accepts any JSON number form. +// A pin on a step or job that can never run must not count. In an expression +// GitHub treats false, 0, -0, the empty string, null and NaN as falsy and +// accepts any JSON number form. An if that is empty or YAML null is no +// condition at all, and keywords are case-insensitive. func TestNeverRunningPins(t *testing.T) { sha := strings.Repeat("a", 40) tag := "v0.4.0-rc.5" good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag for _, cond := range []string{"false", "${{ false }}", "-0", "${{ -0 }}", "-0.0", "0.0", "${{ 0.0 }}", "0e0", ".0", "00", - "0x0", "${{ 0x0 }}", "0x00", "NaN", "${{ NaN }}", "''", "\"''\"", "null", "~", "false && true", "!true", "1 == 2"} { + "0x0", "${{ 0x0 }}", "0x00", "NaN", "${{ NaN }}", "\"''\"", "${{ '' }}", "${{ null }}", "False", "false && true", "${{ !true }}", "1 == 2"} { t.Run("step if "+cond, func(t *testing.T) { text := "jobs:\n test:\n steps:\n - if: " + cond + "\n uses: " + good + "\n" if err := check(strings.NewReader(text), "all", sha, tag); err == nil { @@ -79,7 +81,7 @@ func TestNeverRunningPins(t *testing.T) { } }) } - for _, cond := range []string{"true", "${{ true }}", "needs.plan.outputs.should_run == 'true'", "${{ github.event_name == 'schedule' }}", "always()"} { + for _, cond := range []string{"true", "${{ true }}", "True", "TRUE", "''", "null", "~", "", "needs.plan.outputs.should_run == 'true'", "${{ github.event_name == 'schedule' }}", "always()"} { t.Run("runs "+cond, func(t *testing.T) { text := "jobs:\n test:\n steps:\n - if: " + cond + "\n uses: " + good + "\n" if err := check(strings.NewReader(text), "all", sha, tag); err != nil { @@ -97,6 +99,14 @@ func TestNeverRunningPins(t *testing.T) { {"needs-always", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: ${{ always() }}\n steps:\n - uses: " + good + "\n", true}, {"needs-running", "all", "jobs:\n plan:\n steps:\n - run: true\n b:\n needs: plan\n if: needs.plan.outputs.should_run == 'true'\n steps:\n - uses: " + good + "\n", true}, {"needs-mapping", "all", "jobs:\n a:\n steps:\n - run: true\n b:\n needs: {a: x}\n steps:\n - uses: " + good + "\n", false}, + {"needs-quoted-always", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: github.ref != 'always()'\n steps:\n - uses: " + good + "\n", false}, + {"needs-failure", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: success() || failure()\n steps:\n - uses: " + good + "\n", false}, + {"needs-not-cancelled", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: ${{ !cancelled() }}\n steps:\n - uses: " + good + "\n", true}, + {"needs-case", "all", "jobs:\n plan:\n if: false\n steps:\n - run: true\n b:\n needs: PLAN\n steps:\n - uses: " + good + "\n", false}, + {"job-ids-differ-by-case", "all", "jobs:\n plan:\n steps:\n - run: true\n Plan:\n steps:\n - uses: " + good + "\n", false}, + {"needs-unknown", "all", "jobs:\n b:\n needs: nope\n steps:\n - uses: " + good + "\n", false}, + {"needs-self", "all", "jobs:\n b:\n needs: b\n steps:\n - uses: " + good + "\n", false}, + {"needs-cycle", "all", "jobs:\n a:\n needs: b\n steps:\n - run: true\n b:\n needs: a\n steps:\n - uses: " + good + "\n", false}, } for _, tt := range needs { t.Run(tt.name, func(t *testing.T) { From 462287dfbbbea82f9296a2c4628664d6be410182 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:12:10 +0300 Subject: [PATCH 11/38] docs: list the stable-only tag gates and the operator-note order The Stable Release runbook omitted two gates the stable tag build enforces (every blocks_stable distribution decision recorded; the stable validator byte-identical to the graduated RC's), and a v* tag cannot be reused after a refused build. It also had the operator record the promotion marker after checking the readiness report, while the report reads that record as an input; the record now comes first and the promotion dispatch stays a separate decision. Signed-off-by: ErenAri --- docs/production-release-process.md | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index 10d2b2b..f202859 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -228,11 +228,22 @@ test; production evidence must verify the candidate attestation online. After every operational gate passes, set both `stable_version` and `release_version` to `X.Y.Z`, set `release_channel` to `stable`, update stable -documentation, and tag the checked merge commit. The operator checks the -graduation report and draft assets, records -`[bpfcompat-solo-promotion:v1]`, and dispatches the exact promotion inputs. -Only this path may update the `X.Y` and `latest` image aliases or GitHub's -latest release. +documentation, and tag the checked merge commit. The operator's +`[bpfcompat-solo-promotion:v1]` record is an input of the final readiness +report, so it is written before that report is generated; the operator then +checks the graduation report and draft assets and dispatches the exact +promotion inputs as a separate decision. Only this path may update the `X.Y` +and `latest` image aliases or GitHub's latest release. + +Before tagging, confirm two stable-only gates, because the stable tag build +refuses otherwise and a `v*` tag cannot be reused: + +- `scripts/check-distribution-policy.sh --require-decided` passes, that is, + every decision listed in `blocks_stable` in `distribution/policy.json` is + recorded. +- The stable commit's validator rebuilds to the graduated release candidate's + `bpfcompat-validator-static-linux-amd64` sha256 + (`scripts/check-validator-matches-candidate.sh`). The stable release pull request and the `vX.Y.Z` tag build cannot pin a tag that does not exist yet or that names their own commit, so they keep the From 3eb880ca5a9556d090d7be67b7111ce3c02e5722 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:19:41 +0300 Subject: [PATCH 12/38] release: check the canary schedule and campaign digests in the RC preflight - The designated release-candidate canary must have a schedule cron that fires inside each timed window the plan job accepts, [T0+20h, T0+31h) and [T0+68h, T0+79h); the previous candidate's dated crons left in place would otherwise pass every other designation check while no timed canary can ever count. The helper gains a schedule mode with a small cron matcher (values, ranges, steps, lists, JAN-DEC/SUN-SAT names, the day-of-month/day-of-week OR rule); anything else is refused. - campaign-candidate.json on main must also record this release's SHA256SUMS and candidate-evidence digests, which the scheduled campaigns bind. - CHANGELOG: correct the rc.5 notes against the code: the Action's dot-directory rule is unchanged (now documented), only path and option inputs became literal (command is still run by the guest shell), the relink kit carries only the validator's own sources, the registry-ref rule needs a slash after the host, name:tag refs are affected too and the Action never pulled implicit short refs, the incident note's rejection is operator-asserted, the installer hardening is already live on main, and BPFCOMPAT_DOC_DIR/sudo behaviour. Signed-off-by: ErenAri --- CHANGELOG.md | 93 ++++++++----- internal/tools/rcpreflight/main.go | 168 +++++++++++++++++++++++- internal/tools/rcpreflight/main_test.go | 48 +++++++ scripts/rc-preflight.sh | 28 +++- scripts/rc-preflight_test.sh | 30 ++++- 5 files changed, 320 insertions(+), 47 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9486f1e..2d1785e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,7 +19,9 @@ qualification evidence does not carry over to this candidate. a prerelease) no longer vouches for it. `BPFCOMPAT_VERSION` and the resolved latest tag must look like `vX.Y.Z` or `vX.Y.Z-rc.N`; the installer body runs only when the whole script was received; whether sudo is needed is decided per - target directory before anything is installed. + target directory before anything is installed. (The installer is served from + `main`, so this already applies to `curl … /main/scripts/install.sh | sh` + users since #236 merged, including v0.3.7 installs.) - The GitHub Action no longer pastes inputs into its `run:` scripts. Every input reaches the scripts through `env:` and is read as quoted data, so a value containing quotes, `$(...)`, backticks or newlines can no longer run as @@ -27,7 +29,8 @@ qualification evidence does not carry over to this candidate. (`scripts/check-action-input-interpolation.sh`) and an end-to-end injection test keep it that way. - Values containing `$VAR`, backticks or `\` used to be expanded by bash as - a side effect; they are now passed literally. + a side effect; they are now passed literally. `command` is unchanged: it + was already passed through `env:` and is still run by the guest shell. ### Fixed - A release-pinned GitHub Action now runs the validator it downloaded and @@ -44,21 +47,24 @@ qualification evidence does not carry over to this candidate. with no record of which kernel ran, and a required target could pass. - A missing local `--artifact` path is no longer treated as a registry reference, unless it reads like one: a relative path without a `./`, `../`, - `/` or `~` prefix that does not end in `.bpf.o` and that either has a - host-like first component (it contains `.` or `:`, or is `localhost`) or - contains `@sha256:`. A path with one of those prefixes is never pulled. + `/` or `~` prefix that does not end in `.bpf.o` and that either contains a + `/` after a host-like first component (it contains `.` or `:`, or is + `localhost`) or contains `@sha256:`. A path with one of those prefixes is + never pulled. - Before, any non-existent path that happened to parse as an image reference - (`/home/u/prog.bpf.o`, `build/prog.bpf.o`, `./prog.bpf.o`) was pulled from - Docker Hub. That leaked the local path and failed with a registry error. + (`/home/u/prog.bpf.o`, `build/prog.bpf.o`, `./prog.bpf.o`) was treated as a + registry reference (Docker Hub for most, e.g. `build/prog.bpf.o`). That + leaked the local path and failed with a registry error. - It now fails locally with `artifact "": no such file`. This covers `test`, `test-command`, `suite`, the API and `pkg/bpfcompat.Validate`. - - The CLI now classifies remote refs like the Action's `artifact` input: - the first path component must be a registry host (contains `.` or `:`, or - is `localhost`), or the ref must pin an `@sha256:` digest. - - The Action treats an `artifact` input that looks remote (see - `docs/compatibility-contract.md`, including dot-directories such as - `.output/`) as remote even when a workspace file of that name exists; - prefix workspace files with `./`. + - The CLI now classifies remote refs like the Action's `artifact` input, but + stricter in the safe direction (see `docs/compatibility-contract.md`): the + first path component must be a registry host (contains `.` or `:`, or is + `localhost`), or the ref must pin an `@sha256:` digest. + - Unchanged behaviour, now documented in `docs/compatibility-contract.md`: + the Action treats an `artifact` input that looks remote (including + dot-directories such as `.output/`) as remote even when a workspace file of + that name exists; prefix workspace files with `./`. ### Added - A read-only RC preflight inventories observed prerequisites without counting @@ -74,8 +80,11 @@ qualification evidence does not carry over to this candidate. toolchain and the validator's native components, with an `INVENTORY.json` generated from the shipped binaries) and `bpfcompat-validator-relink-kit-linux-amd64.tar.gz` (the validator's `main.o`, - link commands, linker trace and map, inputs with owning packages, sources and - `relink.sh`). `validator-build-inputs.json` is now a release asset too. + link commands, linker trace and map, inputs with owning packages, the + validator's own C sources and `relink.sh`, which installs the locked library + packages from snapshot.ubuntu.com). The static archives and the native source + packages are not included while their distribution decisions are undecided. + `validator-build-inputs.json` is now a release asset too. - `install.sh` and the GitHub Action install the license bundle, verified with the binaries, when a release lists it; releases without it install as before. The container image carries the CLI's texts in @@ -98,40 +107,54 @@ qualification evidence does not carry over to this candidate. image digest and complete after the candidate was built; the operator note must contain the exact candidate-evidence hash; the Falco run must start after the candidate was built (whether it ran the candidate's validator is recorded, - not required); the promotion incident note must link exactly one rejected - run, and an earlier candidate's note is accepted with that fact recorded. + not required); the promotion incident note must link exactly one workflow run + (the rejected `promote-release` run; that it was rejected is asserted by the + operator, not checked), and an earlier candidate's note is accepted with that + fact recorded. - CI: top-level workflow tokens are read-only; the hosted compatibility gate is required for `pkg/` and `scripts/` changes; untrusted-input parsers have fuzz coverage. -- **Breaking:** implicit Docker Hub short refs in tag form, such as - `org/name:tag`, are no longer pulled. Spell them with the host, e.g. - `docker.io/org/name:tag`; the "no such file" error suggests that spelling. - Digest-pinned short refs (`org/name@sha256:…`) are still pulled. +- **Breaking** (CLI, API and `pkg/bpfcompat`; the Action already required an + explicit host): implicit Docker Hub short refs in tag form, such as + `org/name:tag` or `name:tag` (e.g. `alpine:3`), are no longer pulled. Spell + them with the host, e.g. `docker.io/org/name:tag`; the "no such file" error + suggests that spelling. Digest-pinned short refs (`org/name@sha256:…`) are + still pulled. ### Migration notes -- Docker Hub images: write `docker.io/org/name:tag` instead of `org/name:tag`. +- Docker Hub images (CLI, API, `pkg/bpfcompat`): write `docker.io/org/name:tag` + instead of `org/name:tag`, and `docker.io/name:tag` instead of `name:tag`. The "no such file" error suggests that spelling. -- GitHub Action users with workspace artifacts whose first path component - contains `.` or `:` (for example `.output/` from libbpf-bootstrap, or - `out.v2/`): prefix the path with `./`. Without it the Action treats the input - as a remote reference. -- Action inputs containing `$VAR`, backticks or `\` are now passed literally; - workflows that relied on bash expanding them must expand them in the workflow. +- GitHub Action (unchanged, now documented): an `artifact` input whose first + path component contains `.` or `:` (for example `.output/` from + libbpf-bootstrap, or `out.v2/`) is passed to the CLI as a remote reference. + Prefix workspace files with `./`. In this release such a missing `.bpf.o` + path fails with "no such file" instead of a registry error. +- Action path and option inputs (`artifact`, `matrix`, `out`, `manifest`, + `markdown`, `workdir`, `suite*`, `binary`, `timeout`, `concurrency`, + `disk-resize`, `validation-mode`, `keep-vm-on-failure`) containing `$VAR`, + backticks or `\` are now passed literally. For example, + `out: $RUNNER_TEMP/r.json` must become `out: ${{ runner.temp }}/r.json`. + `command` is unchanged: it is still run by the guest shell, so + `$BPFCOMPAT_BIN`, `$BPFCOMPAT_ARTIFACT` and `$BPFCOMPAT_REMOTE_ROOT` work as + before. - Command mode: a target whose guest kernel release cannot be read is now `INFRA_ERROR` (and fails a required run) instead of passing or failing. - Self-hosted runners with a validator in `/usr/local/libexec` or `/usr/libexec`: a release-pinned Action now runs the validator it downloaded; set `BPFCOMPAT_VALIDATOR_BIN`/`BPFCOMPAT_VALIDATOR_SHA256` in the workflow to keep using another one (the run then warns). -- `install.sh` installs the license bundle under `/../share/doc/bpfcompat//` - (`BPFCOMPAT_DOC_DIR` to change it) when the release publishes one. +- `install.sh` installs the license bundle under `//` when the + release publishes one (default `/../share/doc/bpfcompat`; set + `BPFCOMPAT_DOC_DIR` to change it). If that directory is not writable, the + installer uses sudo or stops before installing anything. ### Known limitations - When this section was drafted, every stable-blocking decision in - `distribution/policy.json` was undecided. The license bundle and relink kit record facts and ship texts - verbatim; they do not elect a license or state that any license condition is - met. A stable release is refused until the stable-blocking decisions are - recorded. + `distribution/policy.json` was undecided. The license bundle and relink kit + record facts and ship texts verbatim; they do not elect a license or state + that any license condition is met. A stable release is refused until the + stable-blocking decisions are recorded. - The prebuilt static validator is published for x86_64 only. - See "Known limitations of the supported path" in `docs/compatibility-contract.md` (artifact-path classification, untrusted diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go index b58ffb8..4829ff2 100644 --- a/internal/tools/rcpreflight/main.go +++ b/internal/tools/rcpreflight/main.go @@ -1,5 +1,5 @@ -// Command rcpreflight checks active candidate Action pins and workflow-level -// env bindings in workflow YAML. +// Command rcpreflight checks active candidate Action pins, workflow-level env +// bindings and the timed canary schedule in workflow YAML. // It reads local files only; the shell inventory builds it from vendored code. package main @@ -9,7 +9,9 @@ import ( "io" "os" "regexp" + "strconv" "strings" + "time" "gopkg.in/yaml.v3" ) @@ -24,8 +26,8 @@ func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } // it prints nothing to stdout and returns 2, so the caller aborts instead of // reporting a status. func run(args []string, stdout, stderr io.Writer) int { - if len(args) != 4 || (args[0] != "all" && args[0] != "candidate" && args[0] != "env") { - fmt.Fprintln(stderr, "usage: rcpreflight \n rcpreflight env ") + if len(args) != 4 || (args[0] != "all" && args[0] != "candidate" && args[0] != "env" && args[0] != "schedule") { + fmt.Fprintln(stderr, "usage: rcpreflight \n rcpreflight env \n rcpreflight schedule - ") return 2 } // #nosec G304 G703 -- this local CLI intentionally reads the operator-supplied workflow path; it never writes it. @@ -246,8 +248,161 @@ func needsGraph(needs map[string][]string, jobs map[string]bool) error { return nil } +// cronField is one field of a five-field cron line: the values it matches, +// and whether it was "*" (which matters for the day-of-month/day-of-week +// rule). +type cronField struct { + match [60]bool + star bool +} + +var cronNames = map[string]int{ + "jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6, "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12, + "sun": 0, "mon": 1, "tue": 2, "wed": 3, "thu": 4, "fri": 5, "sat": 6, +} + +// parseCronField accepts what GitHub documents for schedule crons: *, a +// value, a range a-b, a step */n or a-b/n, comma lists, and JAN-DEC/SUN-SAT +// names. Anything else is refused rather than guessed. +func parseCronField(s string, lo, hi int) (cronField, error) { + var f cronField + f.star = s == "*" + value := func(v string) (int, error) { + if n, ok := cronNames[strings.ToLower(v)]; ok && hi <= 12 { + return n, nil + } + n, err := strconv.Atoi(v) + if err != nil || n < lo || n > hi { + return 0, fmt.Errorf("cron value %q outside %d-%d", v, lo, hi) + } + return n, nil + } + for _, part := range strings.Split(s, ",") { + base, stepText, hasStep := strings.Cut(part, "/") + step := 1 + if hasStep { + n, err := strconv.Atoi(stepText) + if err != nil || n < 1 { + return f, fmt.Errorf("cron step %q", part) + } + step = n + } + first, last := lo, hi + switch a, b, isRange := strings.Cut(base, "-"); { + case base == "*": + case isRange: + var err error + if first, err = value(a); err != nil { + return f, err + } + if last, err = value(b); err != nil { + return f, err + } + if first > last { + return f, fmt.Errorf("cron range %q", part) + } + case hasStep: + return f, fmt.Errorf("cron step needs * or a range: %q", part) + default: + n, err := value(base) + if err != nil { + return f, err + } + first, last = n, n + } + for n := first; n <= last; n += step { + f.match[n] = true + } + } + return f, nil +} + +type cronSpec [5]cronField + +func parseCron(line string) (cronSpec, error) { + var c cronSpec + fields := strings.Fields(line) + if len(fields) != 5 { + return c, fmt.Errorf("cron %q does not have 5 fields", line) + } + bounds := [5][2]int{{0, 59}, {0, 23}, {1, 31}, {1, 12}, {0, 7}} + for i, field := range fields { + f, err := parseCronField(field, bounds[i][0], bounds[i][1]) + if err != nil { + return c, err + } + c[i] = f + } + if c[4].match[7] { + c[4].match[0] = true // 7 is Sunday too + } + return c, nil +} + +// fires reports whether the cron fires at minute t (UTC). As in cron, when +// both day of month and day of week are restricted either one may match. +func (c *cronSpec) fires(t time.Time) bool { + if !c[0].match[t.Minute()] || !c[1].match[t.Hour()] || !c[3].match[int(t.Month())] { + return false + } + dom, dow := c[2].match[t.Day()], c[4].match[int(t.Weekday())] + if c[2].star || c[4].star { + return dom && dow + } + return dom || dow +} + +// checkSchedule requires the workflow's on.schedule to fire inside each timed +// canary window the plan job accepts: 20-30 and 68-78 whole hours after +// publication, that is [T0+20h, T0+31h) and [T0+68h, T0+79h). A cron firing +// at the window's start leaves no room for schedule delay; that margin is the +// operator's choice and is not judged here. +func checkSchedule(root *yaml.Node, published string) error { + t0, err := time.Parse(time.RFC3339, published) + if err != nil { + return fmt.Errorf("publication time %q: %w", published, err) + } + _, on := field(root, "on") + if on == nil { + return fmt.Errorf("no on.schedule list") + } + _, schedule := field(on, "schedule") + if schedule == nil || schedule.Kind != yaml.SequenceNode { + return fmt.Errorf("no on.schedule list") + } + var crons []cronSpec + for _, item := range schedule.Content { + _, line := field(item, "cron") + if line == nil || line.Kind != yaml.ScalarNode { + return fmt.Errorf("schedule entry without a cron string at line %d", item.Line) + } + c, err := parseCron(line.Value) + if err != nil { + return err + } + crons = append(crons, c) + } + for _, w := range [][2]int{{20, 31}, {68, 79}} { + start, end := t0.UTC().Add(time.Duration(w[0])*time.Hour), t0.UTC().Add(time.Duration(w[1])*time.Hour) + t := start.Truncate(time.Minute) + if t.Before(start) { + t = t.Add(time.Minute) + } + found := false + for ; t.Before(end) && !found; t = t.Add(time.Minute) { + for i := range crons { + found = found || crons[i].fires(t) + } + } + if !found { + return fmt.Errorf("no schedule cron fires in [T0+%dh, T0+%dh)", w[0], w[1]) + } + } + return nil +} + func check(r io.Reader, mode, commit, tag string) error { - if mode != "all" && mode != "candidate" && mode != "env" { + if mode != "all" && mode != "candidate" && mode != "env" && mode != "schedule" { return fmt.Errorf("unknown mode %q", mode) } d := yaml.NewDecoder(r) @@ -269,6 +424,9 @@ func check(r io.Reader, mode, commit, tag string) error { if err := keywordCase(doc.Content[0]); err != nil { return err } + if mode == "schedule" { + return checkSchedule(doc.Content[0], commit) + } _, jobs := field(doc.Content[0], "jobs") if jobs == nil || jobs.Kind != yaml.MappingNode { return fmt.Errorf("expected jobs mapping") diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go index 404185e..e7bbb6b 100644 --- a/internal/tools/rcpreflight/main_test.go +++ b/internal/tools/rcpreflight/main_test.go @@ -180,6 +180,53 @@ func TestWorkflowEnvBindings(t *testing.T) { } } +// The timed canaries count only from a schedule run whose plan job starts +// 20-30 or 68-78 whole hours after publication, so the designated crons must +// fire inside [T0+20h, T0+31h) and [T0+68h, T0+79h). T0 below is synthetic. +func TestScheduleWindows(t *testing.T) { + t0 := "2026-11-02T10:00:00Z" // a Monday + wf := func(crons ...string) string { + s := "on:\n workflow_dispatch:\n schedule:\n" + for _, c := range crons { + s += " - cron: \"" + c + "\"\n" + } + return s + "jobs:\n plan:\n steps:\n - run: true\n" + } + tests := []struct { + name, text string + ok bool + }{ + {"precedent-offsets", wf("14 10 3 11 *", "14 10 5 11 *"), true}, + {"window-starts", wf("0 6 3 11 *", "0 6 5 11 *"), true}, + {"last-minutes", wf("59 16 3 11 *", "59 16 5 11 *"), true}, + {"first-window-end-excluded", wf("0 17 3 11 *", "14 10 5 11 *"), false}, + {"second-window-end-excluded", wf("14 10 3 11 *", "0 17 5 11 *"), false}, + {"before-first-window", wf("59 5 3 11 *", "14 10 5 11 *"), false}, + {"previous-candidate-crons", wf("44 17 15 10 *", "44 17 17 10 *"), false}, + {"only-first", wf("14 10 3 11 *"), false}, + {"every-five-minutes", wf("*/5 * * * *"), true}, + {"names-and-ranges", wf("14 10 * NOV TUE", "14 10 * nov thu"), true}, + {"dom-or-dow", wf("14 10 1 11 TUE", "14 10 1 11 THU"), true}, + {"dow-seven-is-sunday", wf("14 10 * 11 7"), false}, + {"list-and-step", wf("0,30 8-12/2 3,5 11 *"), true}, + {"invalid-field", wf("14 10 3 11 *", "61 10 5 11 *"), false}, + {"too-few-fields", wf("14 10 3 11"), false}, + {"no-schedule", "on:\n workflow_dispatch:\njobs:\n plan:\n steps:\n - run: true\n", false}, + {"cron-not-string", "on:\n schedule:\n - cron: [1]\njobs:\n plan:\n steps:\n - run: true\n", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), "schedule", t0, "-") + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } + if err := check(strings.NewReader(wf("14 10 3 11 *", "14 10 5 11 *")), "schedule", "not-a-time", "-"); err == nil { + t.Fatal("an unparsable publication time was accepted") + } +} + func TestRunExitCodes(t *testing.T) { sha := strings.Repeat("a", 40) tag := "v0.4.0-rc.5" @@ -203,6 +250,7 @@ func TestRunExitCodes(t *testing.T) { {"usage", []string{"all", sha, tag}, 2, ""}, {"unknown-mode", []string{"some", sha, tag, good}, 2, ""}, {"env-mode", []string{"env", "CANDIDATE_TAG", tag, good}, 0, "PENDING\n"}, + {"schedule-mode", []string{"schedule", "2026-11-02T10:00:00Z", "-", good}, 0, "PENDING\n"}, {"unreadable", []string{"all", sha, tag, dir + "/missing.yml"}, 2, ""}, } for _, tt := range tests { diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index 188200c..bf979a9 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -8,7 +8,8 @@ # `gh release download` of two text files (SHA256SUMS and # release-candidate-evidence.json), and files in this checkout. It never dispatches or reruns a workflow, never writes to GitHub, # never executes a downloaded file and never writes operator evidence. -# Active workflow pins and the canary's workflow env are parsed by a helper +# Active workflow pins, the canary's workflow env and its schedule are parsed +# by a helper # compiled from this checkout's Go sources and vendored YAML library into the # temporary directory (writing only Go's build cache). The build cannot # download modules or a toolchain: GOPROXY=off, GONOPROXY and GONOSUMDB set to @@ -232,9 +233,21 @@ else ok_designations=0 main_file .github/campaign-candidate.json "$tmp/cc.json" jq -e . "$tmp/cc.json" >/dev/null 2>&1 || abort "campaign-candidate.json on main is not JSON" - if [[ "$(jq -r .tag "$tmp/cc.json")" == "$tag" && "$(jq -r .commit_sha "$tmp/cc.json")" == "$commit" ]]; then - say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit"; ok_designations=$((ok_designations + 1)) - else say PENDING "designated: campaign-candidate.json" "main binds $(jq -r .tag "$tmp/cc.json") (retarget after verification)"; fi + # The scheduled campaigns also bind the two digests recorded there, so they + # must be this release's SHA256SUMS and candidate evidence. + cc_problems="" + [[ "$(jq -r .tag "$tmp/cc.json")" == "$tag" && "$(jq -r .commit_sha "$tmp/cc.json")" == "$commit" ]] || + cc_problems+="binds $(jq -r .tag "$tmp/cc.json") $(jq -r .commit_sha "$tmp/cc.json"); " + for pair in "checksums_sha256=$tmp/sums/SHA256SUMS" "candidate_evidence_sha256=$tmp/ev/release-candidate-evidence.json"; do + key="${pair%%=*}" file="${pair#*=}" + if [[ ! -f "$file" ]]; then cc_problems+="$key: the release has no ${file##*/} to compare; " + elif [[ "$(jq -r --arg k "$key" '.[$k] // empty' "$tmp/cc.json")" != "$(sha256sum "$file" | cut -d' ' -f1)" ]]; then + cc_problems+="$key is not this release's ${file##*/}; " + fi + done + if [[ -z "$cc_problems" ]]; then + say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit and this release's SHA256SUMS and candidate evidence digests"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: campaign-candidate.json" "main differs: ${cc_problems}retarget after verification"; fi main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" command -v go >/dev/null || abort "Go is required to parse active workflow pins" (cd "$ROOT" && GOWORK=off GOPROXY=off GONOPROXY=none.invalid GONOSUMDB=none.invalid go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || @@ -267,10 +280,15 @@ else if [[ "$pin_result" != VERIFIED ]]; then problems+="published-action pin is not @${commit} " fi + # A timed canary counts only from a schedule run inside the plan job's + # windows, so the dated crons must fire inside both of them. + sched="$("$tmp/pincheck" schedule "$published_at" - "$tmp/rcc.yml" 2>"$tmp/sched.err")" || + abort "workflow-pin parser failed: $(cat "$tmp/sched.err")" + [[ "$sched" == VERIFIED ]] || problems+="crons ($(sed -n 's/^rc-preflight pins: //p' "$tmp/sched.err" | tail -1)) " if grep -Eo 'rc[0-9]+-canary-component-|reports/rc[0-9]+-action' "$tmp/rcc.yml" | grep -vqE "^(rc${rcnum}-|reports/rc${rcnum}-)"; then problems+="other candidates' artifact/report names remain " fi - if [[ -z "$problems" ]]; then say VERIFIED "designated: release-candidate-canary" "tag, commit, digest $digest, published $published_at, rc${rcnum} names"; ok_designations=$((ok_designations + 1)) + if [[ -z "$problems" ]]; then say VERIFIED "designated: release-candidate-canary" "tag, commit, digest $digest, published $published_at, rc${rcnum} names, crons inside both timed windows"; ok_designations=$((ok_designations + 1)) else say PENDING "designated: release-candidate-canary" "main still differs: $problems"; fi pub="$(epoch "$published_at")" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index 0320465..c576952 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -26,6 +26,7 @@ RCN="${TAG##*-rc.}" PUB="2026-11-02T10:00:00Z" PUB_EPOCH="$(date -u -d "$PUB" +%s)" at() { date -u -d "@$((PUB_EPOCH + $1 * 3600 + 600))" +%Y-%m-%dT%H:%M:%SZ; } # -> ISO, 10 min past the hour +cron_at() { date -u -d "@$((PUB_EPOCH + $1))" '+%-M %-H %-d %-m *'; } # -> dated cron line DESIGNATED="$(date -u -d "@$((PUB_EPOCH + 300))" +%Y-%m-%dT%H:%M:%SZ)" # designation merged 5 min after T0 mkdir -p "$tmp/bin" @@ -73,9 +74,9 @@ scenario() { printf '%064d %s\n' 1 "$n"; done printf '%s bpfcompat-validator-static-linux-amd64\n' "$VSHA"; } >"$F/SHA256SUMS" contents "$ROOT_DIR/distribution/policy.json" >"$F/policy.json" - jq -n --arg t "$TAG" --arg c "$C1" '{tag: $t, commit_sha: $c}' >"$F/cc-raw.json"; contents "$F/cc-raw.json" >"$F/cc.json" printf 'jobs:\n test:\n steps:\n - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" - printf 'env:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' \ + printf 'on:\n schedule:\n - cron: "%s"\n - cron: "%s"\nenv:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' \ + "$(cron_at $((24 * 3600 + 14 * 60)))" "$(cron_at $((72 * 3600 + 14 * 60)))" \ "$TAG" "$C1" "$DIG" "$PUB" "$RCN" "$C1" "$TAG" "$RCN" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" jq -n --arg i "ghcr.io/kernel-guard/bpfcompat@$DIG" '{image: $i}' >"$F/release-candidate-evidence.json" jq -n '[{sha: "e000000000000000000000000000000000000000", commit: {committer: {date: "2020-01-01T00:00:00Z"}}}]' >"$F/commits.json" @@ -103,10 +104,22 @@ repos/*/commits?sha=main&path=* 200 commits.json repos/*/commits/*/pulls 200 pulls.json repos/*/actions/runs/*/jobs* 200 jobs.json EOF + bind_cc "$F" } route() { sed -i "1i $2" "$tmp/$1/routes"; } # prepend an overriding route +# bind_cc : campaign-candidate.json binds the candidate and the digests +# of the fixture's current SHA256SUMS and candidate evidence, unless the +# scenario pinned its own (cc-fixed). +bind_cc() { + [[ -e "$1/cc-fixed" ]] && return 0 + jq -n --arg t "$TAG" --arg c "$C1" --arg s "$(sha256sum "$1/SHA256SUMS" | cut -d' ' -f1)" \ + --arg e "$(sha256sum "$1/release-candidate-evidence.json" | cut -d' ' -f1)" \ + '{tag: $t, commit_sha: $c, checksums_sha256: $s, candidate_evidence_sha256: $e}' >"$1/cc-raw.json" + contents "$1/cc-raw.json" >"$1/cc.json" +} run() { # [args...] local s="$1" h="$2"; shift 2 + bind_cc "$tmp/$s" : >"$tmp/gh.log" set +e PATH="$tmp/bin:$PATH" FIX="$tmp/$s" GH_LOG="$tmp/gh.log" BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + h * 3600)) \ @@ -212,6 +225,19 @@ contents "$tmp/runtext/rcc-raw.yml" >"$tmp/runtext/rcc.json" run runtext 200 if line PENDING "designated: release-candidate-canary" && grep -q 'CANDIDATE_TAG=' "$tmp/out"; then ok "current values in run text do not hide a stale workflow env" else fail "run-text env (rc=$rc)"; show; fi +scenario ccdigest; touch "$tmp/ccdigest/cc-fixed" +jq --arg s "$(printf 'f%.0s' {1..64})" '.checksums_sha256 = $s' "$tmp/ccdigest/cc-raw.json" >"$tmp/x" && mv "$tmp/x" "$tmp/ccdigest/cc-raw.json" +contents "$tmp/ccdigest/cc-raw.json" >"$tmp/ccdigest/cc.json" +run ccdigest 200 +if line PENDING "designated: campaign-candidate.json" && grep -q "checksums_sha256 is not this release's SHA256SUMS" "$tmp/out" && line PENDING "canary manual"; then + ok "campaign pin naming another release's SHA256SUMS digest: not designated" +else fail "stale campaign digest (rc=$rc)"; show; fi +scenario stalecron +sed -i 's/^ - cron: .*/ - cron: "44 17 15 10 *"/' "$tmp/stalecron/rcc-raw.yml"; contents "$tmp/stalecron/rcc-raw.yml" >"$tmp/stalecron/rcc.json" +run stalecron 200 +if line PENDING "designated: release-candidate-canary" && grep -qF 'crons (no schedule cron fires in [T0+20h, T0+31h))' "$tmp/out"; then + ok "previous candidate's dated crons left in the canary: not designated" +else fail "stale crons (rc=$rc)"; show; fi scenario prerun; LATE_DES="$(date -u -d "@$((PUB_EPOCH + 3 * 3600))" +%Y-%m-%dT%H:%M:%SZ)" jq --arg d "$LATE_DES" '.[0].merged_at = $d' "$tmp/prerun/pulls.json" >"$tmp/x" && mv "$tmp/x" "$tmp/prerun/pulls.json" # The API filter is what excludes earlier runs: serve only what created>=designation would return. From 5d444e70f6c9aff205506b7d49724b31dce9f5f5 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:26:03 +0300 Subject: [PATCH 13/38] docs: bind manual verification commands to the exact release tag The manual verification commands were weaker than the installer they sit next to. cosign verify-blob accepted any tag's signature through an '@refs/tags/v' regexp, gh attestation verify had no --source-ref, and the README's container check used an unanchored identity regexp that any workflow SAN containing 'github.com/Kernel-Guard/bpfcompat' satisfies. - Blob signatures use the exact identity release-artifacts.yml@refs/tags/$VER, as scripts/install.sh does. - gh attestation verify pins --signer-workflow and --source-ref "refs/tags/$VER" (README, verifying-releases, SBOM check). - The README container check uses the exact signer of the pinned 0.3.7 image (ghcr-image.yml@refs/tags/v0.3.7) and says that 0.3.5-0.3.7 images were signed by ghcr-image.yml, while 0.3.8-rc.2, 0.4.0-rc.4 and later release-workflow images are signed by release-artifacts.yml. README and quickstart pin @v0.3.7, so behaviour that ships only from v0.4.0-rc.4 (promotion evidence, SBOM signature, Action attestation verification) or v0.4.0-rc.5 (license bundle, relink kit, build record, Action validator pinning, command-mode kernel-release INFRA_ERROR, missing artifact paths never pulled) now carries that version. Also: the arm64 CLI in the asset table; BPFCOMPAT_DOC_DIR and /usr/libexec/bpfcompat in the installer notes; image tag example 0.3.7; "the validator's own C sources" for the relink kit; the pinned validator build, stable==RC validator and distribution-decision gates in the production support boundary. Signed-off-by: ErenAri --- README.md | 46 +++++++++++----- docs/compatibility-contract.md | 37 ++++++++----- docs/production-support-boundary.md | 35 +++++++----- docs/quickstart.md | 8 +-- docs/supply-chain.md | 27 ++++++--- docs/verifying-releases.md | 85 ++++++++++++++++++----------- 6 files changed, 150 insertions(+), 88 deletions(-) diff --git a/README.md b/README.md index 07c14bd..1c7f17b 100644 --- a/README.md +++ b/README.md @@ -376,12 +376,14 @@ published yet, so artifact-mode VM validation there needs curl -fsSL https://raw.githubusercontent.com/Kernel-Guard/bpfcompat/main/scripts/install.sh | sh ``` -Pin or redirect with `BPFCOMPAT_VERSION`, `BPFCOMPAT_BIN_DIR`, and -`BPFCOMPAT_LIBEXEC_DIR` (see [`scripts/install.sh`](scripts/install.sh)). A -pinned version must be an exact release tag (`vX.Y.Z` or `vX.Y.Z-rc.N`), and its -signature or attestation must come from that tag's release run. A validator -installed outside `/usr/local/libexec/bpfcompat` is not auto-discovered: point -`BPFCOMPAT_VALIDATOR_BIN` at it. +Pin or redirect with `BPFCOMPAT_VERSION`, `BPFCOMPAT_BIN_DIR`, +`BPFCOMPAT_LIBEXEC_DIR`, and `BPFCOMPAT_DOC_DIR` (where the third-party license +bundle goes, for releases that publish one; see +[`scripts/install.sh`](scripts/install.sh)). A pinned version must be an exact +release tag (`vX.Y.Z` or `vX.Y.Z-rc.N`), and its signature or attestation must +come from that tag's release run. A validator installed outside +`/usr/local/libexec/bpfcompat` or `/usr/libexec/bpfcompat` is not +auto-discovered: point `BPFCOMPAT_VALIDATOR_BIN` at it. **1. Prebuilt release binary (Linux x86_64 / arm64).** Installs the CLI *and* the static validator (to a location the CLI auto-discovers), checksum- and @@ -396,10 +398,12 @@ curl -fsSLO "$base/bpfcompat-validator-static-linux-amd64" curl -fsSLO "$base/SHA256SUMS" gh attestation verify ./bpfcompat-linux-amd64 \ --repo Kernel-Guard/bpfcompat \ - --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml + --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml \ + --source-ref "refs/tags/$VER" gh attestation verify ./bpfcompat-validator-static-linux-amd64 \ --repo Kernel-Guard/bpfcompat \ - --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml + --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml \ + --source-ref "refs/tags/$VER" sha256sum -c SHA256SUMS --ignore-missing sudo install -m 0755 bpfcompat-linux-amd64 /usr/local/bin/bpfcompat sudo install -D -m 0755 bpfcompat-validator-static-linux-amd64 \ @@ -444,7 +448,7 @@ multi-arch image (amd64 + arm64), published per release: docker run --rm ghcr.io/kernel-guard/bpfcompat:latest version ``` -Tags follow the release without the `v` prefix (`0.3.6`, `0.3`, `latest`). The +Tags follow the release without the `v` prefix (`0.3.7`, `0.3`, `latest`). The image does not bundle QEMU or the validator and therefore is not a standalone VM-validation environment. Use option 0 or the GitHub Action for a kernel matrix. The experimental `serve` command remains present but does not start by @@ -452,8 +456,19 @@ default and is outside the production support boundary. Release candidates use an exact tag such as `0.4.0-rc.1`; they never update the stable minor or `latest` aliases and are not returned by the installer's default latest-release lookup. -Verify provenance with -`cosign verify ghcr.io/kernel-guard/bpfcompat:0.3.7 --certificate-identity-regexp github.com/Kernel-Guard/bpfcompat --certificate-oidc-issuer https://token.actions.githubusercontent.com`. +Verify provenance with the exact signer identity of that release: + +```bash +cosign verify ghcr.io/kernel-guard/bpfcompat:0.3.7 \ + --certificate-identity https://github.com/Kernel-Guard/bpfcompat/.github/workflows/ghcr-image.yml@refs/tags/v0.3.7 \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com +``` + +The signer workflow differs by release: the 0.3.5 to 0.3.7 images were signed +by `ghcr-image.yml`, while the 0.3.8-rc.2 and 0.4.0-rc.4 images, and later +images from the release workflow, are signed by `release-artifacts.yml` +(`--certificate-identity https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/v`). +Change the identity together with the image tag. ### What a run looks like @@ -813,9 +828,12 @@ Operator guidance: - **Risk scoring:** OpenSSF Scorecard (`scorecard.yml`), published to the public Scorecard API (badge above). - **Signed releases + SLSA provenance:** tag builds produce a CycloneDX SBOM, - cosign keyless (Sigstore OIDC) signatures over `SHA256SUMS` and the SBOM, - plus SLSA provenance attestations for the binaries and container bound to the - producing commit and workflow (`release-artifacts.yml`). Quick check: + cosign keyless (Sigstore OIDC) signatures over `SHA256SUMS` (and, from + v0.4.0-rc.4, the SBOM), plus SLSA provenance attestations for the binaries + and container bound to the producing commit and workflow + (`release-artifacts.yml`; the 0.3.5 to 0.3.7 images were signed and attested + by `ghcr-image.yml`). Quick check (it checks the repository only; the full + steps also bind the release workflow and tag): ```bash gh attestation verify ./bpfcompat-linux-amd64 --repo Kernel-Guard/bpfcompat diff --git a/docs/compatibility-contract.md b/docs/compatibility-contract.md index c657395..95a0ed7 100644 --- a/docs/compatibility-contract.md +++ b/docs/compatibility-contract.md @@ -46,16 +46,18 @@ A static libbpf validator loads the object and optionally attaches it. Provenance: `validator.basename`, `validator.sha256`, `validator.size_bytes` — the same staged bytes are reused for every target in the run. -When the GitHub Action is pinned to a release (a tag, or a commit that maps to -one) and the prebuilt assets verify, the run uses that attested validator, -selected by path and pinned digest: a validator installed elsewhere on the -runner is not used, and other bytes fail closed. A workflow that sets -`BPFCOMPAT_VALIDATOR_BIN` or `BPFCOMPAT_VALIDATOR_SHA256` itself keeps its own -settings, with a warning, and every fallback to a source build while pinned to -a release is a `::warning::` annotation. Neither of those runs is pinned to the -qualified release validator (on a source-build fallback a validator installed -under `/usr/local/libexec` or `/usr/libexec` is still searched first); compare -`validator.sha256` with the release `SHA256SUMS` when it matters. +From v0.4.0-rc.5, when the GitHub Action is pinned to a release (a tag, or a +commit that maps to one) and the prebuilt assets verify, the run uses that +attested validator, selected by path and pinned digest: a validator installed +elsewhere on the runner is not used, and other bytes fail closed. A workflow +that sets `BPFCOMPAT_VALIDATOR_BIN` or `BPFCOMPAT_VALIDATOR_SHA256` itself +keeps its own settings, with a warning, and every fallback to a source build +while pinned to a release is a `::warning::` annotation. Neither of those runs +is pinned to the qualified release validator (on a source-build fallback a +validator installed under `/usr/local/libexec` or `/usr/libexec` is still +searched first); compare `validator.sha256` with the release `SHA256SUMS` when +it matters. With v0.3.7 and v0.4.0-rc.4 that search order applies to every run, +so a validator installed in those directories is used before the verified one. ### The project's own loader (command mode) @@ -77,9 +79,11 @@ Provenance: | `targets[].functional.tests[0].{stdout_tail,stderr_tail}` | bounded diagnostic output | A command-mode target is tied to a kernel by the guest kernel release recorded -before the command starts. If that release cannot be read (or is empty), the -command is not started and the target is `INFRA_ERROR`, never `COMPATIBLE` or -`INCOMPATIBLE`; a required target then fails the run closed. +before the command starts. From v0.4.0-rc.5, if that release cannot be read (or +is empty), the command is not started and the target is `INFRA_ERROR`, never +`COMPATIBLE` or `INCOMPATIBLE`; a required target then fails the run closed. +Earlier releases record the release on a best-effort basis and run the command +even when it cannot be read. Only the invocation bpfcompat was given is recorded. Guest environment contents are not captured, and output is truncated to tails. Do not put secrets in the @@ -271,7 +275,9 @@ Known limitations of the supported path: `localhost/prog.o` are pulled. Spell local files with `./` or an absolute path. Docker Hub images need an explicit `docker.io/` host for tag references; a short digest reference (`name@sha256:…`) is still pulled from - Docker Hub. + Docker Hub. These rules apply from v0.4.0-rc.5; earlier releases treat any + missing path that parses as an image reference as one and try to pull it + (for example `build/prog.bpf.o` from Docker Hub). - **Action `artifact` input.** The Action decides "remote" before looking at the workspace. An input that contains `://` or `@sha256:`, that is `localhost` or starts with `localhost:` or `localhost/`, or whose first @@ -279,7 +285,8 @@ Known limitations of the supported path: unchanged even when a file of that name exists in the workspace. The CLI runs from the Action's own checkout, so such a path is not looked up in the workspace. Dot-directories are caught by this rule too: `.output/minimal.bpf.o` - (the libbpf-bootstrap layout) fails with "no such file", and + (the libbpf-bootstrap layout) fails with "no such file" (from v0.4.0-rc.5; + earlier releases read it as an image on registry host `.output`), and `.output/minimal.o` is read as an image on registry host `.output`. Prefix workspace files with `./` (for example `./.output/minimal.bpf.o` or `./out.v2/prog.o`). diff --git a/docs/production-support-boundary.md b/docs/production-support-boundary.md index e3eb859..e5fe75f 100644 --- a/docs/production-support-boundary.md +++ b/docs/production-support-boundary.md @@ -54,9 +54,11 @@ and do not provide evidence that the surface is production-ready. A passing result means the recorded artifact or command loader completed the requested load, attach, or behavior contract on the exact guest image bytes -recorded in that report. Command-mode reports record the invocation digest and -the loader binary's name, size, and SHA-256. Artifact-mode reports record the -same identity fields for the staged libbpf validator. +recorded in that report. Command-mode reports record the invocation digest, +the loader binary's name, size, and SHA-256, and the guest kernel release; from +v0.4.0-rc.5 a command-mode target whose guest kernel release cannot be recorded +is an infrastructure error. Artifact-mode reports record the same identity +fields for the staged libbpf validator. A pass is not a certification of: @@ -76,18 +78,25 @@ Every supported release must: 1. pass unit, race, vet, vulnerability, generated-doc, and release-consistency checks; 2. build candidate binaries once and record their checksums; -3. run the exact candidate CLI and static validator in a pinned-image positive +3. build the static validator in the digest-pinned builder from + `validator/build-inputs.lock` and, when the validator tree is unchanged, + reproduce the lock's recorded validator byte for byte (from v0.4.0-rc.5); +4. for a stable release, ship exactly the qualified release candidate's + validator bytes (from v0.4.0-rc.5); +5. for a stable release, have every stable-blocking distribution decision in + `distribution/policy.json` recorded (from v0.4.0-rc.5); +6. run the exact candidate CLI and static validator in a pinned-image positive VM test; -4. produce the expected classification from an incompatible artifact; -5. execute the exact ARM64 CLI candidate on a native ARM64 runner; -6. build and version-check a candidate multi-architecture container; -7. bind the release inputs and VM results into an attested candidate-evidence - record; -8. publish signed checksums, SBOM, and build provenance; -9. finish as a private draft without an automatic publication path; -10. receive a separate exact-input manual promotion dispatch through the +7. produce the expected classification from an incompatible artifact; +8. execute the exact ARM64 CLI candidate on a native ARM64 runner; +9. build and version-check a candidate multi-architecture container; +10. bind the release inputs and VM results into an attested candidate-evidence + record; +11. publish signed checksums, SBOM, and build provenance; +12. finish as a private draft without an automatic publication path; +13. receive a separate exact-input manual promotion dispatch through the 15-minute `production-release` environment; and -11. re-verify the run, tag, commit, assets, attestations, signatures, channel, +14. re-verify the run, tag, commit, assets, attestations, signatures, channel, image digest, and version before making the draft release public. The release metadata source is [`release.yaml`](../release.yaml). It records diff --git a/docs/quickstart.md b/docs/quickstart.md index 3b77c9f..5ece9ae 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -67,10 +67,10 @@ Shipping a whole product? Use **suite mode** to gate a collection in one run: The Action is on the [GitHub Marketplace](https://github.com/marketplace/actions/bpfcompat-ebpf-compatibility-gate). Pinning to a release tag — or to the commit SHA a release tag points at — uses -checksum-verified, attested prebuilt binaries -([verifying releases](verifying-releases.md)); otherwise it builds from source, -which needs `libbpf-dev`, `libelf-dev`, `zlib1g-dev`, and `pkg-config` on the -runner. +checksum-verified prebuilt binaries; from v0.4.0-rc.4 the Action also verifies +their build attestations ([verifying releases](verifying-releases.md)). +Otherwise it builds from source, which needs `libbpf-dev`, `libelf-dev`, +`zlib1g-dev`, and `pkg-config` on the runner. ## 2. Run it locally (CLI) diff --git a/docs/supply-chain.md b/docs/supply-chain.md index 30caf46..3c7bc55 100644 --- a/docs/supply-chain.md +++ b/docs/supply-chain.md @@ -90,14 +90,15 @@ and the `reproduces=` line from a pinned build, and cut a new release candidate: that changes the shipped validator. `dist/validator-build-inputs.json` is the validator's build record. It is -attested with the candidate provenance and, from the release that introduces the -distribution bundle on, shipped as a release asset listed in `SHA256SUMS`, so it -outlives the workflow run artifact's retention. +attested with the candidate provenance and, from v0.4.0-rc.5 on, shipped as a +release asset listed in `SHA256SUMS`, so it outlives the workflow run +artifact's retention. ### Third-party license bundle and relink kit -The same build produces two more release assets, both listed in `SHA256SUMS`, -attested, and reproducible byte for byte from the same inputs: +From v0.4.0-rc.5 (not in v0.3.7 or v0.4.0-rc.4), the same build produces two +more release assets, both listed in `SHA256SUMS`, attested, and reproducible +byte for byte from the same inputs: - `bpfcompat-third-party-licenses.tar.gz`: the license, notice and patent texts (verbatim) of every Go module linked into the shipped CLIs, taken from each @@ -109,8 +110,9 @@ attested, and reproducible byte for byte from the same inputs: - `bpfcompat-validator-relink-kit-linux-amd64.tar.gz`: the validator's `main.o`, the Makefile recipe split into compile and link commands, the linker's member trace and map, every link input and header with its owning package, the build - record and locks, the validator sources and `relink.sh`. The build refuses - unless relinking `main.o` reproduces the shipped validator byte for byte. + record and locks, the validator's own C sources and `relink.sh`. The build + refuses unless relinking `main.o` reproduces the shipped validator byte for + byte. `distribution/policy.json` records the open distribution decisions. Two of them add assets when decided: `bpfcompat-validator-link-inputs-linux-amd64.tar.gz` @@ -122,19 +124,26 @@ license or states that any license condition is met. The installer and the GitHub Action install the license bundle when the release's `SHA256SUMS` lists it, and the container image carries the Go texts -in `/usr/share/doc/bpfcompat/`. +in `/usr/share/doc/bpfcompat/`. The installer served from `main` already does +this; the GitHub Action and the container image do from v0.4.0-rc.5. ### Verifying a signed release ```bash +VER=v0.3.7 # the exact release tag you downloaded cosign verify-blob \ --certificate SHA256SUMS.crt --signature SHA256SUMS.sig \ - --certificate-identity-regexp '^https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/v' \ + --certificate-identity "https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/$VER" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ SHA256SUMS sha256sum -c SHA256SUMS ``` +The identity names the exact tag, as `scripts/install.sh` does, so a signature +made by another tag's release run (for example a prerelease) is refused. The +tag-bound attestation checks are in +[verifying-releases.md](verifying-releases.md). + ## Maintainer-side settings (configure once, in the GitHub UI/API) These are not files in the repo. Track their status here. diff --git a/docs/verifying-releases.md b/docs/verifying-releases.md index 0e6f81a..6d48324 100644 --- a/docs/verifying-releases.md +++ b/docs/verifying-releases.md @@ -1,17 +1,18 @@ # Verifying bpfcompat releases Every tagged release of bpfcompat ships with a full supply-chain bundle so you can -prove a binary came from this repository's build workflow before you run it: +prove a binary came from this repository's build workflow before you run it. +Rows marked with a version are published from that release on: | Artifact | What it is | |---|---| -| `bpfcompat-linux-amd64`, `bpfcompat-validator-static-linux-amd64` | the binaries | +| `bpfcompat-linux-amd64`, `bpfcompat-linux-arm64`, `bpfcompat-validator-static-linux-amd64` | the binaries (the static validator is published for amd64 only) | | `SHA256SUMS` (+ `.sig`, `.crt`) | checksums, cosign-signed (keyless) | | `bpfcompat.sbom.cdx.json` | CycloneDX SBOM | -| `release-candidate-evidence.json` | Tag, commit, workflow run, exact image digest, and positive/negative VM report hashes | -| `bpfcompat-third-party-licenses.tar.gz` | verbatim license/notice/patent texts of the third-party code in the binaries, with `INVENTORY.json` (from the release that introduces it) | -| `bpfcompat-validator-relink-kit-linux-amd64.tar.gz` | the validator's `main.o`, link commands, linker trace and map, inputs with owning packages, sources and `relink.sh` | -| `validator-build-inputs.json` | the validator's build record: builder image digest, package snapshot, package versions, sha256 | +| `release-candidate-evidence.json` | Tag, commit, workflow run, exact image digest, and positive/negative VM report hashes (from v0.4.0-rc.4; not in v0.3.7) | +| `bpfcompat-third-party-licenses.tar.gz` | verbatim license/notice/patent texts of the third-party code in the binaries, with `INVENTORY.json` (from v0.4.0-rc.5) | +| `bpfcompat-validator-relink-kit-linux-amd64.tar.gz` | the validator's `main.o`, link commands, linker trace and map, inputs with owning packages, the validator's own C sources and `relink.sh` (from v0.4.0-rc.5) | +| `validator-build-inputs.json` | the validator's build record: builder image digest, package snapshot, package versions, sha256 (from v0.4.0-rc.5) | | **SLSA provenance attestation** | provenance bound to the artifact digest, repository, commit, and release workflow | | **SBOM attestation** | the SBOM bound to the binary's digest | @@ -21,27 +22,31 @@ long-lived private key. The signer identity is the release workflow itself. ## 1. Verify build provenance (recommended) Requires the GitHub CLI (`gh`). This is the strongest check — it proves the binary -was built by *this repo's* release workflow: - -```bash -gh attestation verify ./bpfcompat-linux-amd64 --repo Kernel-Guard/bpfcompat -``` - -A pass confirms the artifact's digest matches an attestation produced by the -`release-artifacts` workflow on a `v*` tag. To pin the exact workflow identity: +was built by *this repo's* release workflow for the tag you downloaded. Pin the +workflow and the exact tag, as `scripts/install.sh` does: ```bash +VER=v0.3.7 # the exact release tag you downloaded gh attestation verify ./bpfcompat-linux-amd64 \ --repo Kernel-Guard/bpfcompat \ - --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml + --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml \ + --source-ref "refs/tags/$VER" ``` -The promotion evidence is independently attested by the same workflow: +A pass confirms the artifact's digest matches an attestation produced by the +`release-artifacts` workflow on the tag `$VER`. Without `--source-ref`, an +attestation from another tag's release run (for example a prerelease) also +passes, and `gh attestation verify ./bpfcompat-linux-amd64 --repo Kernel-Guard/bpfcompat` +alone accepts an attestation from any workflow in this repository. + +From v0.4.0-rc.4 (v0.3.7 has no such asset), the promotion evidence is +independently attested by the same workflow: ```bash gh attestation verify ./release-candidate-evidence.json \ --repo Kernel-Guard/bpfcompat \ - --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml + --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml \ + --source-ref "refs/tags/$VER" ``` Offline / air-gapped verification is supported by downloading the attestation @@ -50,24 +55,33 @@ bundle first (`gh attestation download`) and verifying with `--bundle`. ## 2. Verify checksums + cosign signature ```bash +VER=v0.3.7 # the exact release tag you downloaded + # 1) integrity sha256sum -c SHA256SUMS -# 2) authenticity of SHA256SUMS (keyless cosign) +# 2) authenticity of SHA256SUMS (keyless cosign), bound to the exact tag cosign verify-blob SHA256SUMS \ --signature SHA256SUMS.sig \ --certificate SHA256SUMS.crt \ - --certificate-identity-regexp '^https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/v' \ + --certificate-identity "https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/$VER" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com ``` +The identity names the exact tag, as `scripts/install.sh` does, so a signature +made by another tag's release run (for example a prerelease) is refused. + ## 3. Inspect the SBOM ```bash # human-readable component list cat bpfcompat.sbom.cdx.json | jq '.components[].name' | sort -u # or verify the SBOM attestation is bound to the binary -gh attestation verify ./bpfcompat-linux-amd64 --repo Kernel-Guard/bpfcompat --predicate-type https://cyclonedx.org/bom +gh attestation verify ./bpfcompat-linux-amd64 \ + --repo Kernel-Guard/bpfcompat \ + --signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml \ + --source-ref "refs/tags/$VER" \ + --predicate-type https://cyclonedx.org/bom ``` ## 4. Third-party license material and the relink kit @@ -90,7 +104,7 @@ cd bpfcompat-validator-relink-kit && sha256sum -c SHA256SUMS `INVENTORY.json` separates observed facts (what was linked, from where, with which texts), recorded distribution decisions (`undecided` until made) and what it does not establish. The bundle is not a statement that any license condition -is met. Releases that predate these assets do not have them; the repository's +is met. Releases before v0.4.0-rc.5 do not have these assets; the repository's `LICENSE`, `NOTICE` and `THIRD_PARTY_NOTICES.md` at that tag remain the reference for those. @@ -102,21 +116,26 @@ reference for those. - **Provenance** — a tamper-evident record of *which commit and workflow* built them. - **Promotion binding** — one attested record connects the release tag and - commit to the tested VM reports and promoted container digest. + commit to the tested VM reports and promoted container digest (from + v0.4.0-rc.4). -The GitHub Action consumes prebuilt release binaries only after both -`sha256sum -c` and `gh attestation verify` succeed for the CLI and validator. +From v0.4.0-rc.4 (the v0.3.7 Action verifies checksums only), the GitHub +Action consumes prebuilt release binaries only after both `sha256sum -c` and +`gh attestation verify` succeed for the CLI and validator, and, from +v0.4.0-rc.5, for the third-party license bundle when `SHA256SUMS` lists it. Attestation verification is restricted to `.github/workflows/release-artifacts.yml`; a checksum or attestation mismatch is a hard failure. When attestation verification is unavailable, the Action builds from its pinned source instead of trusting unverifiable release bytes. -After verification the Action runs that validator by path and pinned digest -(`BPFCOMPAT_VALIDATOR_BIN` / `BPFCOMPAT_VALIDATOR_SHA256`), so a validator left -in `/usr/local/libexec` or `/usr/libexec` on a self-hosted runner is not -selected instead. While the Action is pinned to a release, every fallback to a -source build is a `::warning::` annotation: that run does not use the attested -release validator, and the CLI's normal search order (`/usr/local/libexec`, -`/usr/libexec`, then the action checkout) applies to it. A workflow that sets -`BPFCOMPAT_VALIDATOR_BIN` or `BPFCOMPAT_VALIDATOR_SHA256` keeps its own -settings, with a warning. +From v0.4.0-rc.5, after verification the Action runs that validator by path +and pinned digest (`BPFCOMPAT_VALIDATOR_BIN` / `BPFCOMPAT_VALIDATOR_SHA256`), +so a validator left in `/usr/local/libexec` or `/usr/libexec` on a self-hosted +runner is not selected instead. While the Action is pinned to a release, every +fallback to a source build is a `::warning::` annotation: that run does not use +the attested release validator, and the CLI's normal search order +(`/usr/local/libexec`, `/usr/libexec`, then the action checkout) applies to it. +A workflow that sets `BPFCOMPAT_VALIDATOR_BIN` or `BPFCOMPAT_VALIDATOR_SHA256` +keeps its own settings, with a warning. With v0.3.7 and v0.4.0-rc.4 that normal +search order applies to every run, so a validator installed in those +directories is used before the verified one. From b9f3aa1f34d4f850c32bef01b5e6059af5612968 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:26:07 +0300 Subject: [PATCH 14/38] action: document the ./ prefix and workspace binary precedence in inputs The artifact input description now says that a workspace path whose first directory contains '.' or ':' (for example .output/) must be prefixed with ./, or it is treated as a remote reference. The binary input description says that an executable at that path in the workspace is used instead of the verified prebuilt CLI. Both behaviours are unchanged and already documented in docs/compatibility-contract.md; only description strings change. Signed-off-by: ErenAri --- action.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/action.yml b/action.yml index a12b288..258cfc9 100644 --- a/action.yml +++ b/action.yml @@ -8,7 +8,7 @@ inputs: required: false default: "" artifact: - description: "Local path to a compiled .bpf.o artifact, or a remote OCI reference the CLI pulls (for example ghcr.io/org/gadget:latest, host:port/repo, or an @sha256: digest)." + description: "Local path to a compiled .bpf.o artifact, or a remote OCI reference the CLI pulls (for example ghcr.io/org/gadget:latest, host:port/repo, or an @sha256: digest). Prefix workspace paths whose first directory contains `.` or `:` with `./` (for example ./.output/minimal.bpf.o); otherwise they are treated as remote references." required: false default: "" command: @@ -80,7 +80,7 @@ inputs: required: false default: "auto" binary: - description: Path to bpfcompat binary. Relative paths are resolved from the caller repository root. + description: Path to bpfcompat binary. Relative paths are resolved from the caller repository root. An executable at this path in the workspace is used instead of the verified prebuilt CLI. required: false default: "./bin/bpfcompat" baseline-report: From f61c2916eb3e1b9480ca1f84797c5bd3e061c564 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:30:11 +0300 Subject: [PATCH 15/38] release: let the stable PR document its own unreleased tag DEF-2, found by the independent review of DEF-1's fix: the release- consistency gate requires the stable PR's docs to name vX.Y.Z, while scripts/check-doc-pins.sh refused any tag-form pin whose tag does not exist, so the stable PR could not pass required CI even with the canary-pin fix. While release.yaml says release_channel stable, a tag-form doc pin on exactly v is accepted when the remote positively has no such tag (git ls-remote --exit-code returns 2). An unreachable remote, another version, a SHA pin or the prerelease channel still fail. The check gains its first regression test, run in CI. The runbook also states the stable order and that no other stable release may be published between the stable PR's CI and the tag build, because the stable pin may only name the newest published stable below X.Y.Z. Signed-off-by: ErenAri --- .github/workflows/ci.yml | 5 +++ docs/production-release-process.md | 14 ++++++- scripts/check-doc-pins.sh | 35 ++++++++++++++-- scripts/check-doc-pins_test.sh | 66 ++++++++++++++++++++++++++++++ 4 files changed, 115 insertions(+), 5 deletions(-) create mode 100755 scripts/check-doc-pins_test.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3d64282..7bf7091 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -61,6 +61,11 @@ jobs: # v0.3.6 shipped a broken action; the docs kept pointing at it for two # weeks because nothing checked the pins a reader copy-pastes. run: bash scripts/check-doc-pins.sh + - name: Documented action pins regression + # The stable release PR documents vX.Y.Z before its tag exists; only + # that tag, only in the stable channel, and only when the remote + # positively lacks it. + run: bash scripts/check-doc-pins_test.sh - name: Action OCI-artifact detection regression # Guards the fix that lets the action accept remote OCI references # (ghcr.io/...) as `artifact`. Extracts is_remote_artifact() from diff --git a/docs/production-release-process.md b/docs/production-release-process.md index f202859..b5a8514 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -235,8 +235,10 @@ checks the graduation report and draft assets and dispatches the exact promotion inputs as a separate decision. Only this path may update the `X.Y` and `latest` image aliases or GitHub's latest release. -Before tagging, confirm two stable-only gates, because the stable tag build -refuses otherwise and a `v*` tag cannot be reused: +The order is: the operator's record, the final readiness report, the stable +pull request, the tag, then promotion. Before tagging, also confirm these +stable-only gates, which are easy to miss; the stable tag build refuses +otherwise, and a `v*` tag cannot be reused: - `scripts/check-distribution-policy.sh --require-decided` passes, that is, every decision listed in `blocks_stable` in `distribution/policy.json` is @@ -245,6 +247,14 @@ refuses otherwise and a `v*` tag cannot be reused: `bpfcompat-validator-static-linux-amd64` sha256 (`scripts/check-validator-matches-candidate.sh`). +Publish no other stable release between the stable pull request's CI and the +`vX.Y.Z` tag build: the stable pin may only name the newest published stable +release below `X.Y.Z`, so one published in that window (a maintenance release, +for example) fails the tag build. Re-run `make check-release-consistency` on +the exact commit immediately before tagging. The stable pull request's docs may +already name `vX.Y.Z`: `scripts/check-doc-pins.sh` accepts that one tag while +it positively does not exist yet. + The stable release pull request and the `vX.Y.Z` tag build cannot pin a tag that does not exist yet or that names their own commit, so they keep the `# bpfcompat-pin: stable` step on the most recent published (non-draft, diff --git a/scripts/check-doc-pins.sh b/scripts/check-doc-pins.sh index 977c6ce..589201a 100755 --- a/scripts/check-doc-pins.sh +++ b/scripts/check-doc-pins.sh @@ -18,6 +18,13 @@ # (docs/external-ci-proof.md) is history, not an instruction, and is left alone. # `@main` and `@latest` are deliberate and exempt, as is any line carrying an # explicit `doc-pins: allow` marker. +# +# The stable release pull request is the one state rule 2 cannot be met in: +# the release-consistency gate requires its docs to name vX.Y.Z, and that tag +# is created only after the PR merges. So while release.yaml says +# release_channel `stable`, a tag-form pin on exactly v is +# accepted when the remote positively has no such tag; an unreachable remote +# still fails. Commit-SHA pins never get this exemption. set -euo pipefail cd "$(dirname "${BASH_SOURCE[0]}")/.." @@ -45,6 +52,20 @@ tag_commit() { printf '%s' "$sha" } +# The release the stable PR is preparing, or nothing outside the stable channel. +release_value() { awk -F': *' -v k="$1" '$1 == k { gsub(/["\r]/, "", $2); print $2 }' release.yaml; } +preparing="" +[[ "$(release_value release_channel)" == stable ]] && preparing="v$(release_value stable_version)" +prepared="" + +# tag_absent : the remote answered and has no such tag (git ls-remote +# --exit-code returns 2); a remote that cannot be queried is not "absent". +tag_absent() { + local rc=0 + git ls-remote --exit-code --tags origin "refs/tags/$1" >/dev/null 2>&1 || rc=$? + [[ $rc -eq 2 ]] +} + while IFS= read -r hit; do file="${hit%%:*}" rest="${hit#*:}" @@ -78,8 +99,12 @@ while IFS= read -r hit; do elif [[ "$ref" =~ ^v[0-9]+\.[0-9]+\.[0-9]+ ]]; then version="$ref" if [[ -z "$(tag_commit "$version")" ]]; then - note_fail "$file:$line pins $version, which is not a released tag" - continue + if [[ -n "$preparing" && "$version" == "$preparing" ]] && tag_absent "$version"; then + prepared="$version" + else + note_fail "$file:$line pins $version, which is not a released tag" + continue + fi fi else note_fail "$file:$line pins '$ref', which is neither a release tag nor a commit SHA" @@ -107,4 +132,8 @@ if (( fail )); then exit 1 fi -echo "[doc-pins] ok: every documented action pin resolves to ${!versions[*]}" +if [[ -n "$prepared" ]]; then + echo "[doc-pins] ok: every documented action pin names ${prepared}, the stable release being prepared (its tag does not exist yet)" +else + echo "[doc-pins] ok: every documented action pin resolves to ${!versions[*]}" +fi diff --git a/scripts/check-doc-pins_test.sh b/scripts/check-doc-pins_test.sh new file mode 100755 index 0000000..5113d3c --- /dev/null +++ b/scripts/check-doc-pins_test.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# Regression cover for scripts/check-doc-pins.sh. Each case builds a throwaway +# repository whose `origin` is a local bare repository holding the tags, with +# a release.yaml and a README that pins the action. +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +failures=0 +fail() { echo "[doc-pins-test] FAIL: $*" >&2; failures=$((failures + 1)); } +ok() { echo "[doc-pins-test] ok: $*"; } +gitq() { git -c user.name=t -c user.email=t@example.invalid -c commit.gpgsign=false -c tag.gpgsign=false "$@" >/dev/null 2>&1; } + +# fixture [tags...] +fixture() { + local d="$tmp/$1" channel="$2" stable="$3" pin="$4"; shift 4 + mkdir -p "$d/origin.git" "$d/work/scripts" + gitq -C "$d/origin.git" init --bare + gitq -C "$d/work" init -b main + cp "$ROOT_DIR/scripts/check-doc-pins.sh" "$d/work/scripts/" + printf 'stable_version: %s\nrelease_version: %s\nrelease_channel: %s\n' "$stable" "$stable" "$channel" >"$d/work/release.yaml" + printf '# x\n```yaml\n - %s\n```\n' "$pin" >"$d/work/README.md" + gitq -C "$d/work" add -A + gitq -C "$d/work" commit -m base + for t in "$@"; do gitq -C "$d/work" tag -a "$t" -m "$t"; done + gitq -C "$d/work" remote add origin "$d/origin.git" + gitq -C "$d/work" push origin main --tags + # The CI checkout carries no tags locally; the check must ask the remote. + for t in "$@"; do gitq -C "$d/work" tag -d "$t"; done + W="$d/work" +} +expect() { # [stderr-substring] + local want="$1" desc="$2" needle="${3:-}" rc=0 + bash "$W/scripts/check-doc-pins.sh" >"$tmp/out" 2>&1 || rc=$? + if [[ "$want" == pass && $rc -eq 0 ]] || [[ "$want" == fail && $rc -ne 0 && ( -z "$needle" || $(grep -cF -- "$needle" "$tmp/out") -gt 0 ) ]]; then + ok "$desc" + else + fail "$desc (rc=$rc)"; sed 's/^/ | /' "$tmp/out" >&2 + fi +} + +fixture released prerelease 0.3.7 'uses: Kernel-Guard/bpfcompat@v0.3.7' v0.3.7 +expect pass "prerelease channel: pin on an existing release tag" + +fixture stablepr stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 +expect pass "stable PR: docs name v0.4.0 while its tag positively does not exist yet" + +fixture stabletagged stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 v0.4.0 +expect pass "stable channel after the tag: the pin resolves normally" + +fixture otherversion stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.1' v0.3.7 +expect fail "stable PR: a pin on another missing version still fails" "pins v0.4.1, which is not a released tag" + +fixture rcchannel prerelease 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 +expect fail "prerelease channel: a missing tag is never accepted" "pins v0.4.0, which is not a released tag" + +fixture shapin stable 0.4.0 "uses: Kernel-Guard/bpfcompat@$(printf 'a%.0s' {1..40}) # v0.4.0" v0.3.7 +expect fail "stable PR: a SHA pin claiming the unreleased version fails" "claims v0.4.0, but that tag does not exist" + +fixture offline stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 +git -C "$W" remote set-url origin "$tmp/offline/does-not-exist.git" +expect fail "stable PR: an unreachable remote fails closed" "pins v0.4.0, which is not a released tag" + +if [[ $failures -ne 0 ]]; then echo "[doc-pins-test] $failures failure(s)" >&2; exit 1; fi +echo "[doc-pins-test] PASS" From 425335bb2a277dade9a2a2633c8032784c54ab26 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:33:27 +0300 Subject: [PATCH 16/38] readiness: date the Falco run from its first target, not the report production-readiness-report.sh required the Falco report's run.started_at to postdate the candidate's generated_at. The runner writes run.started_at when it assembles the report, after every target ran (runner.go:308-311); in each preserved real report it is minutes later than the first target's start. A run that began before the candidate was built could therefore be accepted as starting after it, contradicting the written rule that the Falco run must start after the candidate was built. The run now starts at the earliest of run.started_at and every target's started_at, and a report with a target that records no start time is refused. The manifest still names the report's run.started_at, unchanged. Signed-off-by: ErenAri --- scripts/production-readiness-report.sh | 10 ++++++++++ scripts/production-readiness-report_test.sh | 11 +++++++++++ 2 files changed, 21 insertions(+) diff --git a/scripts/production-readiness-report.sh b/scripts/production-readiness-report.sh index e8078e6..3ee3770 100755 --- a/scripts/production-readiness-report.sh +++ b/scripts/production-readiness-report.sh @@ -350,6 +350,16 @@ falco_report_started_at="$(jq -r '.run.started_at // ""' "$falco_report")" [[ "$(jq -r '.falco.started_at' "$input")" == "$falco_report_started_at" ]] || fail "Falco started_at does not match the report's run.started_at (${falco_report_started_at:-missing})" falco_started_epoch="$(iso_epoch "$falco_report_started_at" "Falco report run.started_at")" +# run.started_at is written when the report is assembled, after the targets +# ran, so it can postdate the candidate while the run itself began before it. +# The run started at the earliest of run.started_at and every target's +# started_at, and every target must record one. +jq -e '[.targets[] | select((.started_at // "") == "")] | length == 0' "$falco_report" >/dev/null || + fail "Falco report has a target without started_at, so when the run started cannot be established" +while IFS= read -r target_started_at; do + target_epoch="$(iso_epoch "$target_started_at" "Falco report target started_at")" + ((target_epoch < falco_started_epoch)) && falco_started_epoch="$target_epoch" +done < <(jq -r '.targets[].started_at' "$falco_report") [[ "$falco_started_epoch" -gt "$candidate_generated_epoch" ]] || fail "Falco run started before release candidate ${candidate_tag} was built" # Written policy does not say the Falco run must execute the candidate's own diff --git a/scripts/production-readiness-report_test.sh b/scripts/production-readiness-report_test.sh index 390cd25..3357881 100755 --- a/scripts/production-readiness-report_test.sh +++ b/scripts/production-readiness-report_test.sh @@ -23,6 +23,7 @@ make_report() { profile_id: ., required: true, status: "pass", + started_at: $started_at, duration_ms: 1000 }], summary: {status: "pass"}, @@ -478,6 +479,16 @@ jq 'del(.run.started_at)' "$falco_report.keep" >"$falco_report" rehash "$falco_report" .falco.report_sha256 "$tmp/bad-falco-nostart.json" expect_refused "a Falco report without a run start" "$tmp/bad-falco-nostart.json" \ "Falco started_at does not match the report's run.started_at" +# run.started_at is written when the report is assembled, after the targets +# ran: a run whose first target predates the candidate must not pass on it. +jq '.targets[0].started_at = "2026-07-30T13:59:59Z"' "$falco_report.keep" >"$falco_report" +rehash "$falco_report" .falco.report_sha256 "$tmp/bad-falco-early-target.json" +expect_refused "a Falco run whose first target predates the candidate" "$tmp/bad-falco-early-target.json" \ + "Falco run started before release candidate" +jq 'del(.targets[2].started_at)' "$falco_report.keep" >"$falco_report" +rehash "$falco_report" .falco.report_sha256 "$tmp/bad-falco-target-nostart.json" +expect_refused "a Falco report with a target that has no start time" "$tmp/bad-falco-target-nostart.json" \ + "Falco report has a target without started_at" restore "$falco_report" # Rollback: alias, drilled digest and recovery are this candidate's. From cc36c706484ae44c80ae99a4ad905922613781fb Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:35:50 +0300 Subject: [PATCH 17/38] install.sh: name the missing gh capability instead of reporting a failed verification The gh path probed only `gh attestation --help`, but its tag-bound check also needs `gh attestation verify --signer-workflow` (gh 2.51.0) and `--source-ref` (gh 2.68.0), with gh's errors discarded. So a host with an older gh and no cosign got either "requires cosign or GitHub CLI (gh)" (gh < 2.49, although gh is installed) or "GitHub attestation verification FAILED" (gh 2.49-2.67), which reads like a tampered release. When cosign is absent and gh is present, ask gh's own help whether it has the attestation command and both flags (capability, not version parsing) and, if not, refuse naming what it lacks and the remedy (gh >= 2.68.0 or cosign). Still fail-closed; cosign is still preferred whenever present; a real verification failure still says FAILED; the cosign identity, the --source-ref value, checksum, license-bundle and sudo logic are unchanged. The tag guard matched line by line, so a BPFCOMPAT_VERSION carrying a valid tag on one line of a multi-line value passed it and was stopped only by curl. Reject any character a tag cannot hold before the regex, so the whole value is checked. install_test.sh: the gh shim can play gh < 2.49 / < 2.51 / < 2.68; new cases for each (and an unparsable version), for cosign still winning over an old gh, and for newline-bearing tags. Signed-off-by: ErenAri --- scripts/install.sh | 28 ++++++++++++- scripts/install_test.sh | 91 +++++++++++++++++++++++++++++++++++++---- 2 files changed, 108 insertions(+), 11 deletions(-) diff --git a/scripts/install.sh b/scripts/install.sh index 55b5885..43b4731 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -77,7 +77,12 @@ if [ "$VERSION" = "latest" ]; then [ -n "$VERSION" ] || die "could not resolve the latest release tag" fi # The tag is spliced into download URLs and into the exact signing identity -# below, so it must be a plain release tag (no path segments, no regex). +# below, so it must be a plain release tag (no path segments, no regex). grep +# matches line by line, so first reject any character a tag cannot hold (a +# newline included): the whole value is checked, not just one of its lines. +case "$VERSION" in + *[!0123456789.crv-]*) die "invalid release tag '$VERSION' (expected vX.Y.Z or vX.Y.Z-rc.N)" ;; +esac printf '%s\n' "$VERSION" | grep -Eqx 'v[0-9]+\.[0-9]+\.[0-9]+(-rc\.[0-9]+)?' || die "invalid release tag '$VERSION' (expected vX.Y.Z or vX.Y.Z-rc.N)" base="https://github.com/$REPO/releases/download/$VERSION" @@ -143,7 +148,26 @@ if have cosign; then else die "cosign signature verification FAILED for SHA256SUMS - refusing to install" fi -elif have gh && gh attestation --help >/dev/null 2>&1; then +elif have gh; then + # The tag-bound check below needs `gh attestation` (gh 2.49.0), + # --signer-workflow (2.51.0) and --source-ref (2.68.0). An older gh would + # fail it like a forged release would, so ask gh's own help what it offers + # (not its version: distro builds vary) and name what is missing instead. + gh_lacks="" + if ! gh attestation --help >/dev/null 2>&1; then + gh_lacks="the 'gh attestation' command" + else + gh_help="$(gh attestation verify --help 2>&1 || true)" + for flag in --signer-workflow --source-ref; do + printf '%s\n' "$gh_help" | grep -Eq -- "(^|[^[:alnum:]-])$flag([^[:alnum:]-]|\$)" || + gh_lacks="${gh_lacks:+$gh_lacks and }$flag" + done + [ -z "$gh_lacks" ] || gh_lacks="$gh_lacks in 'gh attestation verify'" + fi + if [ -n "$gh_lacks" ]; then + gh_ver="$(gh --version 2>/dev/null | sed -n '1s/^gh version \([0-9][0-9.]*\).*/\1/p')" + die "installed gh${gh_ver:+ $gh_ver} lacks $gh_lacks, needed for tag-bound attestation verification; upgrade to gh >= 2.68.0 or install cosign - refusing checksum-only installation" + fi err "cosign not found; verifying GitHub build attestations" GH_REPO="$REPO" gh attestation verify "$CLI_ASSET" --repo "$REPO" \ --signer-workflow "$REPO/.github/workflows/release-artifacts.yml" \ diff --git a/scripts/install_test.sh b/scripts/install_test.sh index 1356743..6fb1fb3 100644 --- a/scripts/install_test.sh +++ b/scripts/install_test.sh @@ -11,15 +11,20 @@ # the identity baked into SHA256SUMS.crt (the release tag that signed it) # and the signature covers this exact SHA256SUMS; # * gh accepts an attestation only for a binary the fake release pipeline -# attested, and honours --source-ref like the real CLI does; +# attested, and honours --source-ref like the real CLI does; with +# FAKE_GH_LACKS it plays an older gh that has no `attestation` command or +# no --signer-workflow / --source-ref flag, failing like cobra does; # * sudo pretends to be root by unlocking the test's read-only tree. # # Cases: latest -> stable, arch mapping (incl. extra SHA256SUMS lines, the # v0.3.6 bug class), fail-closed without sha tools / without cosign+gh / on a # checksum mismatch, signatures bound to the requested tag, truncated -# `curl | sh` streams, custom install dirs, tag validation, and the -# third-party license bundle (installed when SHA256SUMS lists it, verified like -# the binaries, refused when corrupt/missing/unattested, absent in old releases). +# `curl | sh` streams, custom install dirs, tag validation (the whole value, +# embedded newlines included), the third-party license bundle (installed when +# SHA256SUMS lists it, verified like the binaries, refused when +# corrupt/missing/unattested, absent in old releases), and a gh too old for +# the tag-bound check (refused naming what it lacks, never reported as a +# failed verification or a missing gh; cosign still preferred when present). set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -42,6 +47,7 @@ SHIM="$tmp/shim" TOOLS="$tmp/tools" ATTEST_DB="$tmp/attestations" CURL_LOG="$tmp/curl.log" +GH_LOG="$tmp/gh.log" mkdir -p "$WWW" "$SHIM" "$TOOLS" "$tmp/home" : >"$ATTEST_DB" @@ -134,11 +140,40 @@ EOF cat >"$SHIM/gh" <>"$GH_LOG" +lacks=" \${FAKE_GH_LACKS:-} " +if [[ "\${1:-}" == "--version" ]]; then + printf 'gh version %s (2026-01-01)\nhttps://github.com/cli/cli/releases/latest\n' "\${FAKE_GH_VERSION:-2.102.0}" + exit 0 +fi +if [[ "\${1:-}" == attestation && "\$lacks" == *" attestation "* ]]; then + printf 'unknown command "attestation" for "gh"\n\nUsage: gh [flags]\n' >&2 + exit 1 +fi [[ "\${1:-} \${2:-}" == "attestation --help" ]] && exit 0 +if [[ "\${1:-} \${2:-} \${3:-}" == "attestation verify --help" ]]; then + echo 'Verify the integrity and provenance of an artifact using its associated' + echo 'cryptographically signed attestations.' + [[ "\$lacks" == *" --signer-workflow "* ]] || + echo 'When using reusable workflows, use the \`--signer-repo\` or \`--signer-workflow\` flags.' + echo + echo 'FLAGS' + echo ' -o, --owner string GitHub organization to scope attestation lookup by' + echo ' -R, --repo string Repository name in the format /' + [[ "\$lacks" == *" --signer-workflow "* ]] || + echo ' --signer-workflow string Workflow that signed attestation in the format [host/]////' + [[ "\$lacks" == *" --source-ref "* ]] || + echo ' --source-ref string Ref associated with the source workflow' + exit 0 +fi [[ "\${1:-} \${2:-}" == "attestation verify" ]] || exit 1 shift 2 file="" repo="" signer="" ref="" while [[ \$# -gt 0 ]]; do + if [[ "\$1" == --* && "\$lacks" == *" \$1 "* ]]; then echo "unknown flag: \$1" >&2; exit 1; fi case "\$1" in --repo) repo="\$2"; shift 2;; --signer-workflow) signer="\$2"; shift 2;; @@ -207,7 +242,7 @@ done OUT="$tmp/out" run() { local scen="$1" path="$2"; shift 2 - : >"$CURL_LOG"; rm -f "$tmp/sudo.log" + : >"$CURL_LOG"; : >"$GH_LOG"; rm -f "$tmp/sudo.log" set +e env -i HOME="$tmp/home" PATH="$path" FAKE_PORT="$FAKE_PORT" SCENARIO="$scen" "$@" \ sh "$INSTALL" >"$OUT" 2>&1 @@ -219,7 +254,7 @@ GH_ONLY="$tmp/ghonly:$TOOLS" show() { sed 's/^/ | /' "$OUT" >&2; } # ---------------------------------------------------------------- fixtures -for s in stable arm64 nosha noverify mismatch bindcosign bindgh trunc dirs badver badlatest; do +for s in stable arm64 nosha noverify mismatch bindcosign bindgh trunc dirs badver badlatest ghcap; do release "$s" v0.3.7 amd64 release "$s" v0.4.0-rc.4 amd64 latest "$s" v0.3.7 @@ -346,12 +381,16 @@ else fi # 7. INST-5: BPFCOMPAT_VERSION and the resolved latest tag must look like a tag. -for v in "../../../../evil/x/releases/download/v1" "v0.3.7/../v0.4.0-rc.4" "latest-ish" "v1.2" "v1.2.3-beta.1" "v1.2.3 "; do +# The whole value is checked: a valid tag on one line of a multi-line +# value must not get through either. +for v in "../../../../evil/x/releases/download/v1" "v0.3.7/../v0.4.0-rc.4" "latest-ish" "v1.2" "v1.2.3-beta.1" "v1.2.3 " \ + $'v0.3.7\nx' $'x\nv0.3.7' $'v0.3.7\n'; do + printf -v shown '%q' "$v" d="$tmp/i-badver" run badver "$WITH_COSIGN" BPFCOMPAT_VERSION="$v" BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_NO_VALIDATOR=1 if [[ $rc -ne 0 && ! -e "$d" && ! -s "$CURL_LOG" ]] && grep -q 'invalid release tag' "$OUT"; then - ok "BPFCOMPAT_VERSION '$v' refused before any download" - else fail "BPFCOMPAT_VERSION '$v' not refused up front (rc=$rc)"; show; fi + ok "BPFCOMPAT_VERSION $shown refused before any download" + else fail "BPFCOMPAT_VERSION $shown not refused up front (rc=$rc)"; show; fi done latest badlatest '../../evil' d="$tmp/i-badlatest" @@ -406,6 +445,40 @@ if [[ "$(id -u)" -ne 0 ]]; then chmod -R u+w "$tmp/ro" fi +# 9. A gh too old for the tag-bound attestation check is refused, naming what +# it lacks and the gh that has it, not reported as a failed verification +# (which reads like tampering) or as a missing gh. It is judged by what its +# help offers, so an unparsable version still gets the precise reason. +# cosign, when present, is preferred and used as before. +# gh_lacking +gh_lacking() { + local lacks="$1" ver="$2" want="$3" d="$tmp/i-ghcap" bad="" + run ghcap "$GH_ONLY" FAKE_GH_LACKS="$lacks" FAKE_GH_VERSION="$ver" \ + BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" + [[ $rc -ne 0 && ! -e "$d" ]] || bad="$bad installed-or-rc0" + grep -qF -- "$want" "$OUT" || bad="$bad reason-not-named" + grep -qF 'upgrade to gh >= 2.68.0 or install cosign - refusing checksum-only installation' "$OUT" || + bad="$bad no-remedy" + ! grep -q 'requires cosign or GitHub CLI' "$OUT" || bad="$bad says-gh-missing" + ! grep -q 'FAILED' "$OUT" || bad="$bad says-verification-failed" + ! grep -qE '^attestation verify [^-]' "$GH_LOG" || bad="$bad verify-attempted" + if [[ -z "$bad" ]]; then ok "gh $ver lacking '$lacks': refused, naming what it lacks" + else fail "gh $ver lacking '$lacks' (rc=$rc):$bad"; show; fi +} +gh_lacking attestation 2.45.0 "installed gh 2.45.0 lacks the 'gh attestation' command, needed for tag-bound attestation verification" +gh_lacking --source-ref 2.67.0 "installed gh 2.67.0 lacks --source-ref in 'gh attestation verify', needed for" +gh_lacking "--signer-workflow --source-ref" 2.50.0 "installed gh 2.50.0 lacks --signer-workflow and --source-ref in 'gh attestation verify'" +gh_lacking --source-ref "custom-build" "installed gh lacks --source-ref in 'gh attestation verify'" +for lacks in attestation --source-ref; do + d="$tmp/i-ghcap-cosign-${lacks#--}" + run ghcap "$WITH_COSIGN" FAKE_GH_LACKS="$lacks" FAKE_GH_VERSION=2.45.0 \ + BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" + if [[ $rc -eq 0 && -x "$d/bin/bpfcompat" && -x "$d/libexec/bpfcompat-validator" ]] && + grep -q 'cosign signature over SHA256SUMS verified' "$OUT"; then + ok "gh lacking '$lacks' + cosign: verified with cosign and installed" + else fail "gh lacking '$lacks' with cosign present (rc=$rc)"; show; fi +done + if [[ $failures -ne 0 ]]; then echo "[install-test] $failures failure(s)" >&2 exit 1 From 35d6ebe7db03f3d4964756468470aa2621f202f2 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:44:42 +0300 Subject: [PATCH 18/38] docs: state the gh minimum and correct when RC verification features began Review follow-up for the tag-bound verification docs. - The gh attestation commands now need --source-ref, so the docs state the minimum: gh 2.68.0 (--signer-workflow arrived in 2.51.0, --source-ref in 2.68.0). An older gh stops with "unknown flag" without verifying anything; use cosign or upgrade. gh 2.97.0 and 2.102.0 fixed published advisories in how these values are matched, so 2.102.0 or later is recommended. - Promotion evidence, the SBOM signature and the Action's attestation check are published from v0.4.0-rc.2 and v0.3.8-rc.1 (not v0.3.7), not from v0.4.0-rc.4 as the qualifiers said. - The README lists every release-artifacts.yml-signed image (0.3.8-rc.1, 0.3.8-rc.2, 0.4.0-rc.2, 0.4.0-rc.3, 0.4.0-rc.4) and notes that the 0.3 and latest aliases carry the 0.3.7 signer while they point at 0.3.7. - BPFCOMPAT_DOC_DIR: the bundle goes to //. - The validator==RC and distribution-decision gates are stable-only. - Checksum checks use --ignore-missing where only some assets are downloaded; only bpfcompat-linux-amd64 carries an SBOM attestation. - action.yml: the artifact description also names a localhost first directory, as is_remote_artifact does. Signed-off-by: ErenAri --- README.md | 35 +++++++++++++++----------- action.yml | 2 +- docs/production-support-boundary.md | 10 +++++--- docs/quickstart.md | 5 ++-- docs/supply-chain.md | 2 +- docs/verifying-releases.md | 39 ++++++++++++++++++----------- 6 files changed, 55 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index 1c7f17b..e554d36 100644 --- a/README.md +++ b/README.md @@ -367,18 +367,20 @@ host with `qemu-system-x86_64` for VM-backed runs. and the static guest validator, verifies checksums plus either the release workflow's Sigstore signature or GitHub build attestations, and installs the validator where `bpfcompat test` finds it automatically. Verification is -fail-closed, so the host needs `cosign` or an authenticated GitHub CLI with -attestation support. On arm64 the CLI installs but the static validator is not -published yet, so artifact-mode VM validation there needs -`make validator-static` or command mode: +fail-closed, so the host needs `cosign` or an authenticated GitHub CLI 2.68.0 +or later (older gh lacks `--source-ref`; see +[verifying releases](docs/verifying-releases.md)). On arm64 the CLI installs +but the static validator is not published yet, so artifact-mode VM validation +there needs `make validator-static` or command mode: ```bash curl -fsSL https://raw.githubusercontent.com/Kernel-Guard/bpfcompat/main/scripts/install.sh | sh ``` Pin or redirect with `BPFCOMPAT_VERSION`, `BPFCOMPAT_BIN_DIR`, -`BPFCOMPAT_LIBEXEC_DIR`, and `BPFCOMPAT_DOC_DIR` (where the third-party license -bundle goes, for releases that publish one; see +`BPFCOMPAT_LIBEXEC_DIR`, and `BPFCOMPAT_DOC_DIR` (for releases that publish a +third-party license bundle, it is installed under `//`; +the default directory is `/../share/doc/bpfcompat`; see [`scripts/install.sh`](scripts/install.sh)). A pinned version must be an exact release tag (`vX.Y.Z` or `vX.Y.Z-rc.N`), and its signature or attestation must come from that tag's release run. A validator installed outside @@ -387,8 +389,9 @@ auto-discovered: point `BPFCOMPAT_VALIDATOR_BIN` at it. **1. Prebuilt release binary (Linux x86_64 / arm64).** Installs the CLI *and* the static validator (to a location the CLI auto-discovers), checksum- and -attestation-verified. The validator is currently published for amd64 only; on -arm64 use the CLI plus `make validator-static` or command mode: +attestation-verified (the `gh attestation verify` flags below need gh 2.68.0 or +later). The validator is currently published for amd64 only; on arm64 use the +CLI plus `make validator-static` or command mode: ```bash VER=v0.3.7 @@ -464,9 +467,11 @@ cosign verify ghcr.io/kernel-guard/bpfcompat:0.3.7 \ --certificate-oidc-issuer https://token.actions.githubusercontent.com ``` -The signer workflow differs by release: the 0.3.5 to 0.3.7 images were signed -by `ghcr-image.yml`, while the 0.3.8-rc.2 and 0.4.0-rc.4 images, and later -images from the release workflow, are signed by `release-artifacts.yml` +The signer workflow differs by release: the 0.3.5 to 0.3.7 images (and the +`0.3` and `latest` aliases while they point at 0.3.7) were signed by +`ghcr-image.yml`, while the 0.3.8-rc.1, 0.3.8-rc.2, 0.4.0-rc.2, 0.4.0-rc.3 and +0.4.0-rc.4 images, and later images from the release workflow, are signed by +`release-artifacts.yml` (`--certificate-identity https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/v`). Change the identity together with the image tag. @@ -829,10 +834,10 @@ Operator guidance: public Scorecard API (badge above). - **Signed releases + SLSA provenance:** tag builds produce a CycloneDX SBOM, cosign keyless (Sigstore OIDC) signatures over `SHA256SUMS` (and, from - v0.4.0-rc.4, the SBOM), plus SLSA provenance attestations for the binaries - and container bound to the producing commit and workflow - (`release-artifacts.yml`; the 0.3.5 to 0.3.7 images were signed and attested - by `ghcr-image.yml`). Quick check (it checks the repository only; the full + v0.4.0-rc.2 and v0.3.8-rc.1, the SBOM; not in v0.3.7), plus SLSA provenance + attestations for the binaries and container bound to the producing commit + and workflow (`release-artifacts.yml`; the 0.3.5 to 0.3.7 images were signed + and attested by `ghcr-image.yml`). Quick check (it checks the repository only; the full steps also bind the release workflow and tag): ```bash diff --git a/action.yml b/action.yml index 258cfc9..f1e1206 100644 --- a/action.yml +++ b/action.yml @@ -8,7 +8,7 @@ inputs: required: false default: "" artifact: - description: "Local path to a compiled .bpf.o artifact, or a remote OCI reference the CLI pulls (for example ghcr.io/org/gadget:latest, host:port/repo, or an @sha256: digest). Prefix workspace paths whose first directory contains `.` or `:` with `./` (for example ./.output/minimal.bpf.o); otherwise they are treated as remote references." + description: "Local path to a compiled .bpf.o artifact, or a remote OCI reference the CLI pulls (for example ghcr.io/org/gadget:latest, host:port/repo, or an @sha256: digest). Prefix workspace paths whose first directory contains `.` or `:` or is `localhost` with `./` (for example ./.output/minimal.bpf.o); otherwise they are treated as remote references." required: false default: "" command: diff --git a/docs/production-support-boundary.md b/docs/production-support-boundary.md index e5fe75f..80aa44f 100644 --- a/docs/production-support-boundary.md +++ b/docs/production-support-boundary.md @@ -81,10 +81,12 @@ Every supported release must: 3. build the static validator in the digest-pinned builder from `validator/build-inputs.lock` and, when the validator tree is unchanged, reproduce the lock's recorded validator byte for byte (from v0.4.0-rc.5); -4. for a stable release, ship exactly the qualified release candidate's - validator bytes (from v0.4.0-rc.5); -5. for a stable release, have every stable-blocking distribution decision in - `distribution/policy.json` recorded (from v0.4.0-rc.5); +4. for a stable release only, ship exactly the qualified release candidate's + validator bytes (enforced by the release workflow from v0.4.0-rc.5 on; + release candidates do not run this gate); +5. for a stable release only, have every stable-blocking distribution decision + in `distribution/policy.json` recorded (enforced by the release workflow + from v0.4.0-rc.5 on; release candidates do not run this gate); 6. run the exact candidate CLI and static validator in a pinned-image positive VM test; 7. produce the expected classification from an incompatible artifact; diff --git a/docs/quickstart.md b/docs/quickstart.md index 5ece9ae..11dd35a 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -67,8 +67,9 @@ Shipping a whole product? Use **suite mode** to gate a collection in one run: The Action is on the [GitHub Marketplace](https://github.com/marketplace/actions/bpfcompat-ebpf-compatibility-gate). Pinning to a release tag — or to the commit SHA a release tag points at — uses -checksum-verified prebuilt binaries; from v0.4.0-rc.4 the Action also verifies -their build attestations ([verifying releases](verifying-releases.md)). +checksum-verified prebuilt binaries; from v0.4.0-rc.2 and v0.3.8-rc.1 (not in +v0.3.7) the Action also verifies their build attestations +([verifying releases](verifying-releases.md)). Otherwise it builds from source, which needs `libbpf-dev`, `libelf-dev`, `zlib1g-dev`, and `pkg-config` on the runner. diff --git a/docs/supply-chain.md b/docs/supply-chain.md index 3c7bc55..75b2a9d 100644 --- a/docs/supply-chain.md +++ b/docs/supply-chain.md @@ -136,7 +136,7 @@ cosign verify-blob \ --certificate-identity "https://github.com/Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/$VER" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ SHA256SUMS -sha256sum -c SHA256SUMS +sha256sum -c --ignore-missing SHA256SUMS ``` The identity names the exact tag, as `scripts/install.sh` does, so a signature diff --git a/docs/verifying-releases.md b/docs/verifying-releases.md index 6d48324..a051228 100644 --- a/docs/verifying-releases.md +++ b/docs/verifying-releases.md @@ -2,28 +2,36 @@ Every tagged release of bpfcompat ships with a full supply-chain bundle so you can prove a binary came from this repository's build workflow before you run it. -Rows marked with a version are published from that release on: +Rows that name a version are not in earlier releases: | Artifact | What it is | |---|---| | `bpfcompat-linux-amd64`, `bpfcompat-linux-arm64`, `bpfcompat-validator-static-linux-amd64` | the binaries (the static validator is published for amd64 only) | | `SHA256SUMS` (+ `.sig`, `.crt`) | checksums, cosign-signed (keyless) | | `bpfcompat.sbom.cdx.json` | CycloneDX SBOM | -| `release-candidate-evidence.json` | Tag, commit, workflow run, exact image digest, and positive/negative VM report hashes (from v0.4.0-rc.4; not in v0.3.7) | +| `release-candidate-evidence.json` | Tag, commit, workflow run, exact image digest, and positive/negative VM report hashes (from v0.4.0-rc.2 and v0.3.8-rc.1; not in v0.3.7) | | `bpfcompat-third-party-licenses.tar.gz` | verbatim license/notice/patent texts of the third-party code in the binaries, with `INVENTORY.json` (from v0.4.0-rc.5) | | `bpfcompat-validator-relink-kit-linux-amd64.tar.gz` | the validator's `main.o`, link commands, linker trace and map, inputs with owning packages, the validator's own C sources and `relink.sh` (from v0.4.0-rc.5) | | `validator-build-inputs.json` | the validator's build record: builder image digest, package snapshot, package versions, sha256 (from v0.4.0-rc.5) | | **SLSA provenance attestation** | provenance bound to the artifact digest, repository, commit, and release workflow | -| **SBOM attestation** | the SBOM bound to the binary's digest | +| **SBOM attestation** | the SBOM bound to the `bpfcompat-linux-amd64` digest (the arm64 CLI and the validator carry SLSA provenance only) | Signing is **keyless** (Sigstore Fulcio + Rekor) via GitHub Actions OIDC — there is no long-lived private key. The signer identity is the release workflow itself. ## 1. Verify build provenance (recommended) -Requires the GitHub CLI (`gh`). This is the strongest check — it proves the binary -was built by *this repo's* release workflow for the tag you downloaded. Pin the -workflow and the exact tag, as `scripts/install.sh` does: +Requires the GitHub CLI (`gh`) 2.68.0 or later: `--signer-workflow` was added +in gh 2.51.0 and `--source-ref` in gh 2.68.0, and an older gh stops with +`unknown flag` without verifying anything (upgrade it, or use the cosign check +in section 2). How strictly gh matches these values also depends on its +version: gh 2.97.0 and 2.102.0 fixed published advisories in that matching +(GHSA-mm27-mwq9-fr5g, GHSA-wjmr-j3rp-mh2g, GHSA-4mq3-hpgx-9cx8), so 2.102.0 or +later is recommended. + +This is the strongest check — it proves the binary was built by *this repo's* +release workflow for the tag you downloaded. Pin the workflow and the exact +tag, as `scripts/install.sh` does: ```bash VER=v0.3.7 # the exact release tag you downloaded @@ -39,8 +47,8 @@ attestation from another tag's release run (for example a prerelease) also passes, and `gh attestation verify ./bpfcompat-linux-amd64 --repo Kernel-Guard/bpfcompat` alone accepts an attestation from any workflow in this repository. -From v0.4.0-rc.4 (v0.3.7 has no such asset), the promotion evidence is -independently attested by the same workflow: +From v0.4.0-rc.2 and v0.3.8-rc.1 (v0.3.7 has no such asset), the promotion +evidence is independently attested by the same workflow: ```bash gh attestation verify ./release-candidate-evidence.json \ @@ -57,8 +65,8 @@ bundle first (`gh attestation download`) and verifying with `--bundle`. ```bash VER=v0.3.7 # the exact release tag you downloaded -# 1) integrity -sha256sum -c SHA256SUMS +# 1) integrity of the assets you downloaded +sha256sum -c --ignore-missing SHA256SUMS # 2) authenticity of SHA256SUMS (keyless cosign), bound to the exact tag cosign verify-blob SHA256SUMS \ @@ -117,12 +125,13 @@ reference for those. built them. - **Promotion binding** — one attested record connects the release tag and commit to the tested VM reports and promoted container digest (from - v0.4.0-rc.4). + v0.4.0-rc.2 and v0.3.8-rc.1; not in v0.3.7). -From v0.4.0-rc.4 (the v0.3.7 Action verifies checksums only), the GitHub -Action consumes prebuilt release binaries only after both `sha256sum -c` and -`gh attestation verify` succeed for the CLI and validator, and, from -v0.4.0-rc.5, for the third-party license bundle when `SHA256SUMS` lists it. +From v0.4.0-rc.2 and v0.3.8-rc.1 (the v0.3.7 Action verifies checksums only), +the GitHub Action consumes prebuilt release binaries only after both +`sha256sum -c` and `gh attestation verify` succeed for the CLI and validator, +and, from v0.4.0-rc.5, for the third-party license bundle when `SHA256SUMS` +lists it. Attestation verification is restricted to `.github/workflows/release-artifacts.yml`; a checksum or attestation mismatch is a hard failure. When attestation verification is unavailable, the Action From e1cbd9748e2878033c60bf2cc1e405890d1a4dce Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:45:57 +0300 Subject: [PATCH 19/38] release: refuse schedule keys and cron forms the preflight cannot judge Fourth exact-head review findings: - A schedule entry with any key besides cron (GitHub's timezone, for one) is refused: only cron is evaluated, in UTC, so a timezone would move the real firing time outside the window while the check said VERIFIED. - Cron fields are stricter: names only in their own field, digits only (no +N), steps bounded by the field's range (a huge step crashed the helper), the documented a/n form, day of week 0-6, and a day field starting with * is unrestricted for the day-of-month/day-of-week rule, as in cron. - A campaign-candidate.json that is JSON but not an object aborts the report (exit 3) instead of ending it with a truncated jq error. - CHANGELOG: an empty if is no condition; the incident note's run is named, not checked. Signed-off-by: ErenAri --- CHANGELOG.md | 16 ++++---- internal/tools/rcpreflight/main.go | 52 ++++++++++++++++--------- internal/tools/rcpreflight/main_test.go | 9 ++++- scripts/rc-preflight.sh | 2 +- scripts/rc-preflight_test.sh | 4 ++ 5 files changed, 55 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2d1785e..8dcd0e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,10 +70,10 @@ qualification evidence does not carry over to this candidate. - A read-only RC preflight inventories observed prerequisites without counting pending qualification as a pass. Candidate Action pins and the canary's candidate env are read from actual workflow YAML nodes; comments, shell-script - text and steps or jobs that can never run (an `if` that reads no context, - calls no function and is not `true`, or a job that needs one of those) - cannot establish them. Conditions that call functions are left to the - runner. + text and steps or jobs that can never run (a constant `if` other than + `true`, such as `false` or `0`, or a job that needs one of those; an empty + `if` is no condition) cannot establish them. Conditions that read a context + or call a function are left to the runner. - Releases carry the third-party license material of their binaries as signed assets listed in `SHA256SUMS`: `bpfcompat-third-party-licenses.tar.gz` (verbatim license, notice and patent texts of every linked Go module, the Go @@ -107,10 +107,10 @@ qualification evidence does not carry over to this candidate. image digest and complete after the candidate was built; the operator note must contain the exact candidate-evidence hash; the Falco run must start after the candidate was built (whether it ran the candidate's validator is recorded, - not required); the promotion incident note must link exactly one workflow run - (the rejected `promote-release` run; that it was rejected is asserted by the - operator, not checked), and an earlier candidate's note is accepted with that - fact recorded. + not required); the promotion incident note must name `promote-release` and + link exactly one workflow run (that it is the rejected `promote-release` run + is asserted by the operator, not checked), and an earlier candidate's note is + accepted with that fact recorded. - CI: top-level workflow tokens are read-only; the hosted compatibility gate is required for `pkg/` and `scripts/` changes; untrusted-input parsers have fuzz coverage. diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go index 4829ff2..bb9f6d1 100644 --- a/internal/tools/rcpreflight/main.go +++ b/internal/tools/rcpreflight/main.go @@ -256,21 +256,28 @@ type cronField struct { star bool } -var cronNames = map[string]int{ - "jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6, "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12, - "sun": 0, "mon": 1, "tue": 2, "wed": 3, "thu": 4, "fri": 5, "sat": 6, -} +var ( + monthNames = map[string]int{"jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6, "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12} + dayNames = map[string]int{"sun": 0, "mon": 1, "tue": 2, "wed": 3, "thu": 4, "fri": 5, "sat": 6} +) + +func digits(s string) bool { return s != "" && strings.Trim(s, "0123456789") == "" } // parseCronField accepts what GitHub documents for schedule crons: *, a -// value, a range a-b, a step */n or a-b/n, comma lists, and JAN-DEC/SUN-SAT -// names. Anything else is refused rather than guessed. -func parseCronField(s string, lo, hi int) (cronField, error) { +// value, a range a-b, a step */n, a-b/n or a/n, comma lists, and names +// (JAN-DEC, SUN-SAT) only in their own field. Anything else is refused rather +// than guessed. As in cron, a field that starts with * is unrestricted for +// the day-of-month/day-of-week rule. +func parseCronField(s string, lo, hi int, names map[string]int) (cronField, error) { var f cronField - f.star = s == "*" + f.star = strings.HasPrefix(s, "*") value := func(v string) (int, error) { - if n, ok := cronNames[strings.ToLower(v)]; ok && hi <= 12 { + if n, ok := names[strings.ToLower(v)]; ok { return n, nil } + if !digits(v) { + return 0, fmt.Errorf("cron value %q", v) + } n, err := strconv.Atoi(v) if err != nil || n < lo || n > hi { return 0, fmt.Errorf("cron value %q outside %d-%d", v, lo, hi) @@ -282,7 +289,7 @@ func parseCronField(s string, lo, hi int) (cronField, error) { step := 1 if hasStep { n, err := strconv.Atoi(stepText) - if err != nil || n < 1 { + if !digits(stepText) || err != nil || n < 1 || n > hi { return f, fmt.Errorf("cron step %q", part) } step = n @@ -301,14 +308,15 @@ func parseCronField(s string, lo, hi int) (cronField, error) { if first > last { return f, fmt.Errorf("cron range %q", part) } - case hasStep: - return f, fmt.Errorf("cron step needs * or a range: %q", part) default: n, err := value(base) if err != nil { return f, err } - first, last = n, n + first = n // a/n runs from a to the field's end + if !hasStep { + last = n + } } for n := first; n <= last; n += step { f.match[n] = true @@ -325,17 +333,15 @@ func parseCron(line string) (cronSpec, error) { if len(fields) != 5 { return c, fmt.Errorf("cron %q does not have 5 fields", line) } - bounds := [5][2]int{{0, 59}, {0, 23}, {1, 31}, {1, 12}, {0, 7}} + bounds := [5][2]int{{0, 59}, {0, 23}, {1, 31}, {1, 12}, {0, 6}} + names := [5]map[string]int{3: monthNames, 4: dayNames} for i, field := range fields { - f, err := parseCronField(field, bounds[i][0], bounds[i][1]) + f, err := parseCronField(field, bounds[i][0], bounds[i][1], names[i]) if err != nil { return c, err } c[i] = f } - if c[4].match[7] { - c[4].match[0] = true // 7 is Sunday too - } return c, nil } @@ -372,6 +378,16 @@ func checkSchedule(root *yaml.Node, published string) error { } var crons []cronSpec for _, item := range schedule.Content { + // Only cron is evaluated, in UTC; any other key (timezone, for one) + // would change when GitHub fires it, so it is refused, not ignored. + if item.Kind != yaml.MappingNode { + return fmt.Errorf("schedule entry is not a mapping at line %d", item.Line) + } + for i := 0; i < len(item.Content); i += 2 { + if k := item.Content[i]; k.Value != "cron" { + return fmt.Errorf("schedule entry key %q at line %d is not checked; only cron, in UTC", k.Value, k.Line) + } + } _, line := field(item, "cron") if line == nil || line.Kind != yaml.ScalarNode { return fmt.Errorf("schedule entry without a cron string at line %d", item.Line) diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go index e7bbb6b..c9e97fd 100644 --- a/internal/tools/rcpreflight/main_test.go +++ b/internal/tools/rcpreflight/main_test.go @@ -207,7 +207,14 @@ func TestScheduleWindows(t *testing.T) { {"every-five-minutes", wf("*/5 * * * *"), true}, {"names-and-ranges", wf("14 10 * NOV TUE", "14 10 * nov thu"), true}, {"dom-or-dow", wf("14 10 1 11 TUE", "14 10 1 11 THU"), true}, - {"dow-seven-is-sunday", wf("14 10 * 11 7"), false}, + {"dow-seven-refused", wf("14 10 3 11 *", "14 10 5 11 7"), false}, + {"start-step-form", wf("14 10 3/2 11 *"), true}, + {"star-step-day-is-unrestricted", wf("14 10 */2 11 SUN"), false}, + {"month-name-in-dow", wf("14 10 * 11 NOV"), false}, + {"dow-name-in-month", wf("14 10 3 MON *", "14 10 5 11 *"), false}, + {"plus-sign", wf("14 10 +3 11 *", "14 10 5 11 *"), false}, + {"huge-step", wf("14 10 1-5/9223372036854775807 11 *", "14 10 5 11 *"), false}, + {"timezone-key", "on:\n schedule:\n - cron: \"14 10 3 11 *\"\n timezone: \"Asia/Tokyo\"\n - cron: \"14 10 5 11 *\"\njobs:\n plan:\n steps:\n - run: true\n", false}, {"list-and-step", wf("0,30 8-12/2 3,5 11 *"), true}, {"invalid-field", wf("14 10 3 11 *", "61 10 5 11 *"), false}, {"too-few-fields", wf("14 10 3 11"), false}, diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh index bf979a9..128cd7b 100755 --- a/scripts/rc-preflight.sh +++ b/scripts/rc-preflight.sh @@ -232,7 +232,7 @@ else } ok_designations=0 main_file .github/campaign-candidate.json "$tmp/cc.json" - jq -e . "$tmp/cc.json" >/dev/null 2>&1 || abort "campaign-candidate.json on main is not JSON" + jq -e 'type == "object"' "$tmp/cc.json" >/dev/null 2>&1 || abort "campaign-candidate.json on main is not a JSON object" # The scheduled campaigns also bind the two digests recorded there, so they # must be this release's SHA256SUMS and candidate evidence. cc_problems="" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh index c576952..94f244a 100755 --- a/scripts/rc-preflight_test.sh +++ b/scripts/rc-preflight_test.sh @@ -338,6 +338,10 @@ scenario badev; printf 'garbage' >"$tmp/badev/release-candidate-evidence.json" run badev 200 if [[ $rc -eq 3 ]] && grep -q 'release-candidate-evidence.json of .* is not JSON' "$tmp/out"; then ok "unparsable candidate evidence aborts" else fail "bad evidence (rc=$rc)"; show; fi +scenario ccarray; touch "$tmp/ccarray/cc-fixed"; printf '[]' >"$tmp/ccarray/cc-raw.json"; contents "$tmp/ccarray/cc-raw.json" >"$tmp/ccarray/cc.json" +run ccarray 200 +if [[ $rc -eq 3 ]] && grep -q 'campaign-candidate.json on main is not a JSON object' "$tmp/out"; then ok "a campaign pin that is not a JSON object aborts" +else fail "non-object campaign pin (rc=$rc)"; show; fi # 9. Building the pin parser: no module or toolchain download, and the caller's # toolchain choice is kept (an older Go on PATH may switch to a cached From df47e0fce3e9ceee9b86c5a42e2eb7bf7d21ac65 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:51:31 +0300 Subject: [PATCH 20/38] readiness: ignore unbooted optional profiles when dating the Falco run Review of the previous commit: the runner writes no started_at for a target it never runs, such as an optional profile the default runner cannot boot (status unsupported; fail in v0.3.7), and the shipped expanded matrix has three. Refusing every target without started_at would refuse such a legitimate report. Now every required or passing target must record started_at, and the earliest recorded start is the run's start. The runbook describes the rule, and the test gains a positive case shaped like a real report (targets start before run.started_at) with an unbooted optional profile. Signed-off-by: ErenAri --- docs/production-release-process.md | 7 +++++-- scripts/production-readiness-report.sh | 10 ++++++---- scripts/production-readiness-report_test.sh | 18 ++++++++++++++++-- 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index f15d6bd..ed43d8d 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -166,8 +166,11 @@ aggregator binds every piece of evidence to the attested candidate (`generated_at`, tag, commit, image digest, evidence SHA-256): - **Falco.** `falco.started_at` must equal the report's own `run.started_at`, - and, as for the campaigns, that run must start after the candidate's - `generated_at`: evidence must postdate the candidate it graduates. + and the run must start after the candidate's `generated_at`: evidence must + postdate the candidate it graduates. The runner writes `run.started_at` when + it assembles the report, so the run's start is the earliest of that and every + target's `started_at`; a required or passing target without `started_at` is + refused, while an optional profile the runner never booted does not count. `workflow_run_id` and `commit_sha` are not recorded in the report and remain operator-asserted (`falco.run_identity` in the output). Written policy does not require the Falco lane to run the candidate's own bpfcompat, so this is diff --git a/scripts/production-readiness-report.sh b/scripts/production-readiness-report.sh index 3ee3770..cdb0ea8 100755 --- a/scripts/production-readiness-report.sh +++ b/scripts/production-readiness-report.sh @@ -353,13 +353,15 @@ falco_started_epoch="$(iso_epoch "$falco_report_started_at" "Falco report run.st # run.started_at is written when the report is assembled, after the targets # ran, so it can postdate the candidate while the run itself began before it. # The run started at the earliest of run.started_at and every target's -# started_at, and every target must record one. -jq -e '[.targets[] | select((.started_at // "") == "")] | length == 0' "$falco_report" >/dev/null || - fail "Falco report has a target without started_at, so when the run started cannot be established" +# started_at. Every required or passing target must record one; an optional +# profile the runner never booted (no started_at, not passing) does not count. +jq -e '[.targets[] | select((.required == true or .status == "pass") and (.started_at // "") == "")] | length == 0' \ + "$falco_report" >/dev/null || + fail "Falco report has a target that ran without started_at, so when the run started cannot be established" while IFS= read -r target_started_at; do target_epoch="$(iso_epoch "$target_started_at" "Falco report target started_at")" ((target_epoch < falco_started_epoch)) && falco_started_epoch="$target_epoch" -done < <(jq -r '.targets[].started_at' "$falco_report") +done < <(jq -r '.targets[] | .started_at // empty' "$falco_report") [[ "$falco_started_epoch" -gt "$candidate_generated_epoch" ]] || fail "Falco run started before release candidate ${candidate_tag} was built" # Written policy does not say the Falco run must execute the candidate's own diff --git a/scripts/production-readiness-report_test.sh b/scripts/production-readiness-report_test.sh index 3357881..44c8fba 100755 --- a/scripts/production-readiness-report_test.sh +++ b/scripts/production-readiness-report_test.sh @@ -487,8 +487,22 @@ expect_refused "a Falco run whose first target predates the candidate" "$tmp/bad "Falco run started before release candidate" jq 'del(.targets[2].started_at)' "$falco_report.keep" >"$falco_report" rehash "$falco_report" .falco.report_sha256 "$tmp/bad-falco-target-nostart.json" -expect_refused "a Falco report with a target that has no start time" "$tmp/bad-falco-target-nostart.json" \ - "Falco report has a target without started_at" +expect_refused "a Falco report whose required target has no start time" "$tmp/bad-falco-target-nostart.json" \ + "Falco report has a target that ran without started_at" +# Shaped like a real report: targets start minutes before run.started_at, all +# after the candidate; an optional profile the runner never booted +# ("unsupported", no started_at) does not count. +jq '.targets |= map(.started_at = "2026-08-03T05:50:00Z") | + .targets += [{profile_id: "talos-1.7", required: false, status: "unsupported", duration_ms: 0}]' \ + "$falco_report.keep" >"$falco_report" +rehash "$falco_report" .falco.report_sha256 "$tmp/good-falco-real-shape.json" +if BPFCOMPAT_SKIP_READINESS_ATTESTATION=1 "$script" "$tmp/good-falco-real-shape.json" "$tmp/good.md" >"$tmp/falco-real-shape.log" 2>&1; then + echo "[production-readiness-test] accepted: a real-shaped Falco report with an unbooted optional profile" +else + cat "$tmp/falco-real-shape.log" >&2 + echo "[production-readiness-test] a real-shaped Falco report with an unbooted optional profile was refused" >&2 + exit 1 +fi restore "$falco_report" # Rollback: alias, drilled digest and recovery are this candidate's. From e033fe6be8c3b9e7128b9391e1c5305c328fc4af Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:54:57 +0300 Subject: [PATCH 21/38] readiness: look up the drill and incident runs it is given Written policy requires the rollback evidence to be the note rollback-drill.yml writes and the incident note to link the deliberately rejected promote-release run. The readiness report checked only that each was a well-formed run ID, so (verified by reproduction) a run ID that does not exist, or a successful ci.yml run, was accepted with attestations verified. With attestation verification on, the drill run must now be a successful rollback-drill.yml run on main and the incident run a completed promote-release.yml run that did not succeed; an unreadable run fails. Which rejection qualifies stays the operator's choice; a successful or unrelated run never does. The test-only bypass is unchanged. Signed-off-by: ErenAri --- docs/production-release-process.md | 8 +++-- scripts/production-readiness-report.sh | 18 ++++++++++ scripts/production-readiness-report_test.sh | 39 +++++++++++++++++++++ 3 files changed, 63 insertions(+), 2 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index ed43d8d..3852a77 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -198,8 +198,12 @@ aggregator binds every piece of evidence to the attested candidate - **Incident.** The note must contain `[bpfcompat-promotion-incident:v1]`, name `promote-release`, and link the deliberately rejected run, and only that run, as - `https://github.com/Kernel-Guard/bpfcompat/actions/runs/`. Written - policy does not require the incident drill to target the current candidate, + `https://github.com/Kernel-Guard/bpfcompat/actions/runs/`. With + attestation verification on, that run must be a completed `promote-release.yml` + run that did not succeed, and the rollback evidence's workflow run must be a + successful `rollback-drill.yml` run on `main`; both are looked up on GitHub. + Written policy does not require the incident drill to target the current + candidate, so an earlier candidate's fail-closed drill is accepted; the output records `incident.promotion_run_id` and `incident.candidate_bound`. - **Operator.** The confirmation must contain diff --git a/scripts/production-readiness-report.sh b/scripts/production-readiness-report.sh index cdb0ea8..0ffd9eb 100755 --- a/scripts/production-readiness-report.sh +++ b/scripts/production-readiness-report.sh @@ -545,6 +545,24 @@ incident_run_ids="$( [[ "$(wc -l <<<"$incident_run_ids")" -eq 1 ]] || fail "incident evidence links more than one workflow run; it must link only the rejected promote-release run" incident_run_id="$incident_run_ids" +# Written policy: the rollback evidence is the note rollback-drill.yml writes, +# and the incident links the deliberately rejected promote-release run. With +# attestation verification on, both run IDs must be real runs of those +# workflows here: the drill a successful run on main, the incident a completed +# promote-release run that did not succeed. (Which rejection qualifies stays +# the operator's call; a successful or unrelated run never does.) +if [[ "${BPFCOMPAT_SKIP_READINESS_ATTESTATION:-0}" != "1" ]]; then + rollback_run="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${rollback_run_id}" 2>/dev/null)" || + fail "rollback drill run ${rollback_run_id} cannot be read from GitHub" + jq -e '.path == ".github/workflows/rollback-drill.yml" and .head_branch == "main" and + .status == "completed" and .conclusion == "success"' <<<"$rollback_run" >/dev/null || + fail "rollback drill run ${rollback_run_id} is not a successful rollback-drill.yml run on main" + incident_run="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${incident_run_id}" 2>/dev/null)" || + fail "incident run ${incident_run_id} cannot be read from GitHub" + jq -e '.path == ".github/workflows/promote-release.yml" and + .status == "completed" and .conclusion != "success"' <<<"$incident_run" >/dev/null || + fail "incident run ${incident_run_id} is not a rejected promote-release.yml run" +fi incident_candidate_bound=false if grep -qE "(^|[^[:alnum:].-])${candidate_tag//./\\.}([^[:alnum:]]|$)" "$incident"; then incident_candidate_bound=true diff --git a/scripts/production-readiness-report_test.sh b/scripts/production-readiness-report_test.sh index 44c8fba..8b80bf3 100755 --- a/scripts/production-readiness-report_test.sh +++ b/scripts/production-readiness-report_test.sh @@ -626,6 +626,21 @@ mkdir -p "$tmp/bin" cat >"$tmp/bin/gh" <<'STUB' #!/usr/bin/env bash [[ "$1" == attestation && "$2" == --help ]] && exit 0 +# Workflow run lookups: run 5000 is the rollback drill and 6000 the rejected +# promotion; STUB_* variables override their fields, any other run is a 404. +if [[ "$1" == api && "$2" =~ ^repos/Kernel-Guard/bpfcompat/actions/runs/([0-9]+)$ ]]; then + printf '%s\n' "$*" >>"$STUB_LOG" + case "${BASH_REMATCH[1]}" in + 5000) jq -n --arg p "${STUB_ROLLBACK_PATH:-.github/workflows/rollback-drill.yml}" \ + --arg b "${STUB_ROLLBACK_BRANCH:-main}" --arg c "${STUB_ROLLBACK_CONCLUSION:-success}" \ + '{path: $p, head_branch: $b, status: "completed", conclusion: $c}' ;; + 6000) jq -n --arg p "${STUB_INCIDENT_PATH:-.github/workflows/promote-release.yml}" \ + --arg c "${STUB_INCIDENT_CONCLUSION:-failure}" \ + '{path: $p, head_branch: "v0.4.0-rc.1", status: "completed", conclusion: $c}' ;; + *) echo "gh: Not Found (HTTP 404)" >&2; exit 1 ;; + esac + exit 0 +fi [[ "$1 $2" == "attestation verify" ]] || { echo "unexpected gh $*" >&2; exit 99; } printf '%s\n' "$*" >>"$STUB_LOG" [[ -n "${STUB_FAIL:-}" && "$3" == *"$STUB_FAIL"* ]] && exit 1 @@ -661,6 +676,30 @@ if PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" STUB_EVENT=workflow_dispatch "$s echo "[production-readiness-test] a dispatched run relabelled as scheduled was accepted" >&2; exit 1 fi echo "[production-readiness-test] refused: evidence claiming schedule whose attestation shows workflow_dispatch" +grep -Fq "api repos/Kernel-Guard/bpfcompat/actions/runs/5000" "$tmp/gh.log" && + grep -Fq "api repos/Kernel-Guard/bpfcompat/actions/runs/6000" "$tmp/gh.log" || { + echo "[production-readiness-test] the drill and incident runs were not looked up" >&2; exit 1; } +# Written policy: the rollback note comes from a rollback-drill.yml run and the +# incident links the rejected promote-release run. Both must be real runs. +for case in \ + "STUB_ROLLBACK_PATH=.github/workflows/ci.yml|a rollback drill run that is another workflow" \ + "STUB_ROLLBACK_CONCLUSION=failure|a rollback drill run that did not succeed" \ + "STUB_ROLLBACK_BRANCH=feature|a rollback drill run not on main" \ + "STUB_INCIDENT_PATH=.github/workflows/ci.yml|an incident linking a run of another workflow" \ + "STUB_INCIDENT_CONCLUSION=success|an incident linking a promotion that succeeded"; do + if env "${case%%|*}" PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" "$script" "$tmp/input.json" "$tmp/bad.md" >/dev/null 2>&1; then + echo "[production-readiness-test] ${case#*|} was accepted" >&2; exit 1 + fi + echo "[production-readiness-test] refused: ${case#*|}" +done +sed 's#/actions/runs/6000#/actions/runs/6999#' "$incident_note" >"$tmp/incident-404.md" +cp "$incident_note" "$incident_note.keep"; cp "$tmp/incident-404.md" "$incident_note" +rehash "$incident_note" .incident.evidence_sha256 "$tmp/bad-incident-404.json" +if PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" "$script" "$tmp/bad-incident-404.json" "$tmp/bad.md" >/dev/null 2>&1; then + echo "[production-readiness-test] an incident linking a run that does not exist was accepted" >&2; exit 1 +fi +echo "[production-readiness-test] refused: an incident linking a run that does not exist" +mv "$incident_note.keep" "$incident_note" # --- Readiness output v0.2 and the real stable gate ---------------------------- jq -e --arg c "$(printf 'c%.0s' {1..40})" ' From 227320497f6f19883cdc070dbdaa6682234b2693 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:57:04 +0300 Subject: [PATCH 22/38] release: let SHA-pinned templates keep the previous stable during the stable window DEF-3, found by the independent review of f61c291's simulated stable PR: the docs/integrations templates are pinned by commit SHA (their README makes that policy), so they cannot name vX.Y.Z before its tag exists, and check-doc-pins.sh's one-version rule then refused the stable PR ("docs pin more than one bpfcompat version"). During the stable window -- the stable PR while vX.Y.Z is absent, and the vX.Y.Z tag build itself -- exactly one other documented version may sit beside v: the newest earlier stable release tag on the remote. Every later commit is strict, so the templates are repinned with the consumer canary once the tag exists (now stated in the runbook). check-release-consistency_test.sh no longer hard-codes 0.3.7 as the stable channel's version; it reads stable_version from release.yaml, so the stable PR does not have to edit the test. Signed-off-by: ErenAri --- docs/production-release-process.md | 19 +++++--- scripts/check-doc-pins.sh | 53 ++++++++++++++++++++++- scripts/check-doc-pins_test.sh | 44 +++++++++++++++++++ scripts/check-release-consistency_test.sh | 7 ++- 4 files changed, 114 insertions(+), 9 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index b5a8514..9d145d4 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -251,9 +251,14 @@ Publish no other stable release between the stable pull request's CI and the `vX.Y.Z` tag build: the stable pin may only name the newest published stable release below `X.Y.Z`, so one published in that window (a maintenance release, for example) fails the tag build. Re-run `make check-release-consistency` on -the exact commit immediately before tagging. The stable pull request's docs may -already name `vX.Y.Z`: `scripts/check-doc-pins.sh` accepts that one tag while -it positively does not exist yet. +the exact commit immediately before tagging, and tag the merge commit as soon +as the pull request merges: from then until the tag exists, `main`'s docs name +a release that is not out yet. In the stable pull request, the tag-form action +references in `README.md` and `docs/` name `vX.Y.Z`, which +`scripts/check-doc-pins.sh` accepts while that tag positively does not exist. +The SHA-pinned integration templates in `docs/integrations/` cannot name it +yet; they stay on the most recent earlier stable release during the stable +pull request and the tag build. The stable release pull request and the `vX.Y.Z` tag build cannot pin a tag that does not exist yet or that names their own commit, so they keep the @@ -263,9 +268,11 @@ accepts that pin only while `vX.Y.Z` is absent or resolves to the commit under check; a tag or GitHub release lookup it cannot complete fails the check. Once the new stable tag exists, repin the `# bpfcompat-pin: stable` step in -`.github/workflows/consumer-canary.yml` to its commit SHA and update the -`# vX.Y.Z` comment beside it. `scripts/check-canary-pins.sh` runs inside the -release-consistency gate and fails every later commit until that is done, +`.github/workflows/consumer-canary.yml` and the SHA-pinned templates in +`docs/integrations/` to its commit SHA, and update the `# vX.Y.Z` comments +beside them. `scripts/check-canary-pins.sh` (inside the release-consistency +gate) and `scripts/check-doc-pins.sh` fail every later commit until that is +done, because a canary left on the previous release stops testing the path the documentation now sends every reader down. The `# bpfcompat-pin: candidate` step is not enforced while `release_channel` is `stable`; repin it with the diff --git a/scripts/check-doc-pins.sh b/scripts/check-doc-pins.sh index 589201a..f0584be 100755 --- a/scripts/check-doc-pins.sh +++ b/scripts/check-doc-pins.sh @@ -25,6 +25,13 @@ # release_channel `stable`, a tag-form pin on exactly v is # accepted when the remote positively has no such tag; an unreachable remote # still fails. Commit-SHA pins never get this exemption. +# +# The integration templates are pinned by commit SHA (docs/integrations), so +# they cannot name vX.Y.Z until its tag exists. During the same stable window +# -- the stable PR while vX.Y.Z is absent, and the vX.Y.Z tag build itself -- +# rule 3 allows exactly one other version beside v: the most +# recent earlier stable release tag (vA.B.C < X.Y.Z). Every later commit is +# strict again, so the templates are repinned once the tag exists. set -euo pipefail cd "$(dirname "${BASH_SOURCE[0]}")/.." @@ -58,6 +65,35 @@ preparing="" [[ "$(release_value release_channel)" == stable ]] && preparing="v$(release_value stable_version)" prepared="" +# version_lt A B: X.Y.Z version A sorts strictly before X.Y.Z version B. +version_lt() { + local -a a b + local i + IFS=. read -r -a a <<<"$1" + IFS=. read -r -a b <<<"$2" + for i in 0 1 2; do + if (( 10#${a[i]} != 10#${b[i]} )); then + (( 10#${a[i]} < 10#${b[i]} )) + return + fi + done + return 1 +} + +# previous_stable : the newest vA.B.C tag on the remote below it, or +# nothing when the remote cannot be listed. +previous_stable() { + local tags tag newest="" + tags="$(git ls-remote --tags --refs origin 'refs/tags/v*' 2>/dev/null)" || return 0 + while read -r _ tag; do + tag="${tag#refs/tags/}" + [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || continue + version_lt "${tag#v}" "${1#v}" || continue + if [[ -z "$newest" ]] || version_lt "${newest#v}" "${tag#v}"; then newest="$tag"; fi + done <<<"$tags" + printf '%s' "$newest" +} + # tag_absent : the remote answered and has no such tag (git ls-remote # --exit-code returns 2); a remote that cannot be queried is not "absent". tag_absent() { @@ -120,7 +156,18 @@ if (( ${#versions[@]} == 0 )); then note_fail "found no 'uses: Kernel-Guard/bpfcompat@...' pins in README.md or docs -- the docs moved, or this check no longer matches them" fi -if (( ${#versions[@]} > 1 )); then +# The stable window: v is being prepared (positively absent) +# or this is its tag build (it resolves to HEAD). +window_previous="" +if [[ -n "$preparing" && ${#versions[@]} -eq 2 && -n "${versions[$preparing]:-}" ]]; then + head_commit="$(git rev-parse -q --verify 'HEAD^{commit}' 2>/dev/null || true)" + if [[ -n "$prepared" ]] || [[ -n "$head_commit" && "$(tag_commit "$preparing")" == "$head_commit" ]]; then + window_previous="$(previous_stable "$preparing")" + fi +fi +if (( ${#versions[@]} == 2 )) && [[ -n "$window_previous" && -n "${versions[$window_previous]:-}" ]]; then + echo "[doc-pins] stable window for ${preparing}: SHA-pinned templates may stay on ${window_previous} until the tag exists (${versions[$window_previous]% })" +elif (( ${#versions[@]} > 1 )); then note_fail "docs pin more than one bpfcompat version:" for v in "${!versions[@]}"; do echo " $v -> ${versions[$v]}" >&2 @@ -132,7 +179,9 @@ if (( fail )); then exit 1 fi -if [[ -n "$prepared" ]]; then +if [[ -n "$window_previous" ]]; then + echo "[doc-pins] ok: documented pins name ${preparing}, the stable release being prepared or built; SHA-pinned templates stay on ${window_previous} until its tag exists" +elif [[ -n "$prepared" ]]; then echo "[doc-pins] ok: every documented action pin names ${prepared}, the stable release being prepared (its tag does not exist yet)" else echo "[doc-pins] ok: every documented action pin resolves to ${!versions[*]}" diff --git a/scripts/check-doc-pins_test.sh b/scripts/check-doc-pins_test.sh index 5113d3c..ade9c52 100755 --- a/scripts/check-doc-pins_test.sh +++ b/scripts/check-doc-pins_test.sh @@ -30,6 +30,21 @@ fixture() { for t in "$@"; do gitq -C "$d/work" tag -d "$t"; done W="$d/work" } +# template [file]: an integration template SHA-pinned to , +# committed on top of the fixture (the version's tag is on the base commit). +template() { + mkdir -p "$W/docs/integrations" + printf 'jobs:\n t:\n steps:\n - uses: Kernel-Guard/bpfcompat@%s # %s\n' \ + "$(git -C "$W" rev-list --max-parents=0 HEAD)" "$1" >"$W/docs/integrations/${2:-t}.yml" + gitq -C "$W" add -A + gitq -C "$W" commit -m "template $1" +} +# tag_head : tag the current HEAD on the remote (the tag build of it). +tag_head() { + gitq -C "$W" tag -a "$1" -m "$1" + gitq -C "$W" push origin "$1" + gitq -C "$W" tag -d "$1" +} expect() { # [stderr-substring] local want="$1" desc="$2" needle="${3:-}" rc=0 bash "$W/scripts/check-doc-pins.sh" >"$tmp/out" 2>&1 || rc=$? @@ -62,5 +77,34 @@ fixture offline stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 git -C "$W" remote set-url origin "$tmp/offline/does-not-exist.git" expect fail "stable PR: an unreachable remote fails closed" "pins v0.4.0, which is not a released tag" +# The integration templates are SHA-pinned (docs/integrations/README.md), so +# they cannot name vX.Y.Z before its tag exists: during the stable release +# window they stay on the most recent earlier stable release. +fixture tplpr stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.6 v0.3.7 +template v0.3.7 +expect pass "stable PR: templates keep their SHA pin on the newest earlier stable" + +fixture tplold stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.6 v0.3.7 +template v0.3.6 +expect fail "stable PR: a template on an older stable than the newest fails" "docs pin more than one bpfcompat version" + +fixture tplthree stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.6 v0.3.7 +template v0.3.7 a +template v0.3.6 b +expect fail "stable PR: three documented versions fail" "docs pin more than one bpfcompat version" + +fixture tplbuild stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 +template v0.3.7 +tag_head v0.4.0 +expect pass "tag build: the new tag is HEAD, templates still on the previous stable" + +fixture tplafter stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 v0.4.0 +template v0.3.7 +expect fail "after the tag: templates must be repinned to the new release" "docs pin more than one bpfcompat version" + +fixture tplrc prerelease 0.3.7 'uses: Kernel-Guard/bpfcompat@v0.3.7' v0.3.6 v0.3.7 +template v0.3.6 +expect fail "prerelease channel: two documented versions fail as before" "docs pin more than one bpfcompat version" + if [[ $failures -ne 0 ]]; then echo "[doc-pins-test] $failures failure(s)" >&2; exit 1; fi echo "[doc-pins-test] PASS" diff --git a/scripts/check-release-consistency_test.sh b/scripts/check-release-consistency_test.sh index b0c907d..b3576ac 100755 --- a/scripts/check-release-consistency_test.sh +++ b/scripts/check-release-consistency_test.sh @@ -26,6 +26,9 @@ restore() { rm -rf "$tmp" } trap restore EXIT +current_stable="$( + awk -F': ' '$1 == "stable_version" {print $2; exit}' release.yaml +)" current_release="$( awk -F': ' '$1 == "release_version" {print $2; exit}' release.yaml )" @@ -53,7 +56,9 @@ EOF BPFCOMPAT_RELEASE_METADATA=release.yaml "$script" >"$tmp/current.log" grep -Fq 'channel=prerelease' "$tmp/current.log" -write_metadata 0.3.7 0.3.7 stable +# The current stable release as a stable channel: docs and canary pins must +# already agree with it (the stable PR changes stable_version, so this follows). +write_metadata "$current_stable" "$current_stable" stable BPFCOMPAT_RELEASE_METADATA="$tmp/release.yaml" "$script" >"$tmp/stable.log" grep -Fq 'channel=stable' "$tmp/stable.log" From f8809a9df266cf0df21c54641547afb8e1a792b8 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Sun, 18 Oct 2026 23:58:14 +0300 Subject: [PATCH 23/38] install.sh: report a gh that cannot run; test that every gh verify call is made Review follow-ups for the gh capability probe: - A gh that cannot run at all (every command fails, e.g. an unreadable ~/.config/gh/config.yml) was told it "lacks the 'gh attestation' command" and to upgrade. When `gh attestation --help` fails, the installer now also runs `gh --help`; if that fails too it refuses with "installed gh could not run 'gh --help' (); fix gh or install cosign". Still fail-closed; nothing else changes. - Comment: `gh attestation` exists, hidden, from gh 2.47.0 (public from 2.49.0). Behaviour is unchanged: on 2.47/2.48 the flag probe already names --signer-workflow and --source-ref as missing. install_test.sh: removing or neutralising one gh verify call used to go unnoticed (only the whole gh block and the bundle call were guarded). The gh stub now records every verify call; the gh-path install cases assert that exactly the installed assets (CLI, validator, and the license bundle when listed) were verified, each with the release workflow and --source-ref refs/tags/, and the cosign cases that gh verified nothing. New cases swap only the CLI, or only the validator, for bytes attested by another tag's run (SHA256SUMS regenerated to match), so each call must refuse on its own. Also: a gh that cannot run, with and without cosign, and a root `--help` in the stub as real gh has. Mutation check (scratch copies of install.sh): replacing a verify call with `true`, turning its `die` into a warning, or dropping its --source-ref, for each of the three calls, fails this suite (9/9); the previous suite caught 2/9. Signed-off-by: ErenAri --- scripts/install.sh | 14 ++++-- scripts/install_test.sh | 101 +++++++++++++++++++++++++++++++++------- 2 files changed, 93 insertions(+), 22 deletions(-) diff --git a/scripts/install.sh b/scripts/install.sh index 43b4731..2021b38 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -149,12 +149,18 @@ if have cosign; then die "cosign signature verification FAILED for SHA256SUMS - refusing to install" fi elif have gh; then - # The tag-bound check below needs `gh attestation` (gh 2.49.0), - # --signer-workflow (2.51.0) and --source-ref (2.68.0). An older gh would - # fail it like a forged release would, so ask gh's own help what it offers - # (not its version: distro builds vary) and name what is missing instead. + # The tag-bound check below needs `gh attestation` (gh 2.47.0, hidden until + # 2.49.0), --signer-workflow (2.51.0) and --source-ref (2.68.0). An older gh + # would fail it like a forged release would, so ask gh's own help what it + # offers (not its version: distro builds vary) and name what is missing. gh_lacks="" if ! gh attestation --help >/dev/null 2>&1; then + # A gh that cannot run at all (say, an unreadable config) fails every + # command; say so with gh's own error rather than blame its version. + if ! gh --help >/dev/null 2>&1; then + gh_err="$( { gh --help >/dev/null; } 2>&1 | sed -n '1p')" + die "installed gh could not run 'gh --help'${gh_err:+ ($gh_err)}; fix gh or install cosign - refusing checksum-only installation" + fi gh_lacks="the 'gh attestation' command" else gh_help="$(gh attestation verify --help 2>&1 || true)" diff --git a/scripts/install_test.sh b/scripts/install_test.sh index 6fb1fb3..919828d 100644 --- a/scripts/install_test.sh +++ b/scripts/install_test.sh @@ -11,9 +11,12 @@ # the identity baked into SHA256SUMS.crt (the release tag that signed it) # and the signature covers this exact SHA256SUMS; # * gh accepts an attestation only for a binary the fake release pipeline -# attested, and honours --source-ref like the real CLI does; with -# FAKE_GH_LACKS it plays an older gh that has no `attestation` command or -# no --signer-workflow / --source-ref flag, failing like cobra does; +# attested, and honours --source-ref like the real CLI does. It records +# every verify call, so the gh-path cases check that each installed asset +# was verified, bound to the release workflow and tag. With FAKE_GH_LACKS +# it plays an older gh that has no `attestation` command or no +# --signer-workflow / --source-ref flag, failing like cobra does; with +# FAKE_GH_BROKEN=1, a gh that cannot run at all (say, a bad config); # * sudo pretends to be root by unlocking the test's read-only tree. # # Cases: latest -> stable, arch mapping (incl. extra SHA256SUMS lines, the @@ -24,7 +27,8 @@ # SHA256SUMS lists it, verified like the binaries, refused when # corrupt/missing/unattested, absent in old releases), and a gh too old for # the tag-bound check (refused naming what it lacks, never reported as a -# failed verification or a missing gh; cosign still preferred when present). +# failed verification or a missing gh; a gh that cannot run is reported as +# such; cosign still preferred when present). set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -48,6 +52,7 @@ TOOLS="$tmp/tools" ATTEST_DB="$tmp/attestations" CURL_LOG="$tmp/curl.log" GH_LOG="$tmp/gh.log" +GH_VERIFY_LOG="$tmp/gh-verify.log" mkdir -p "$WWW" "$SHIM" "$TOOLS" "$tmp/home" : >"$ATTEST_DB" @@ -140,11 +145,21 @@ EOF cat >"$SHIM/gh" <>"$GH_LOG" +if [[ -n "\${FAKE_GH_BROKEN:-}" ]]; then + echo 'failed to create root command: failed to read configuration: invalid config file /home/u/.config/gh/config.yml: invalid yaml' >&2 + exit 1 +fi lacks=" \${FAKE_GH_LACKS:-} " +if [[ "\${1:-}" == "--help" ]]; then + echo 'Work seamlessly with GitHub from the command line.' + exit 0 +fi if [[ "\${1:-}" == "--version" ]]; then printf 'gh version %s (2026-01-01)\nhttps://github.com/cli/cli/releases/latest\n' "\${FAKE_GH_VERSION:-2.102.0}" exit 0 @@ -182,6 +197,7 @@ while [[ \$# -gt 0 ]]; do *) file="\$1"; shift;; esac done +printf 'verify %s repo=%s signer=%s ref=%s\n' "\$file" "\$repo" "\$signer" "\$ref" >>"$GH_VERIFY_LOG" [[ "\$repo" == "$REPO" ]] || exit 1 [[ "\$signer" == "$REPO/.github/workflows/release-artifacts.yml" ]] || exit 1 sum="\$("$REAL_SHA" <"\$file" | cut -d' ' -f1)" @@ -242,7 +258,7 @@ done OUT="$tmp/out" run() { local scen="$1" path="$2"; shift 2 - : >"$CURL_LOG"; : >"$GH_LOG"; rm -f "$tmp/sudo.log" + : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; rm -f "$tmp/sudo.log" set +e env -i HOME="$tmp/home" PATH="$path" FAKE_PORT="$FAKE_PORT" SCENARIO="$scen" "$@" \ sh "$INSTALL" >"$OUT" 2>&1 @@ -252,9 +268,27 @@ run() { WITH_COSIGN="$SHIM:$TOOLS" GH_ONLY="$tmp/ghonly:$TOOLS" show() { sed 's/^/ | /' "$OUT" >&2; } +# verified : on the gh-only PATH, gh verified exactly +# these assets, once each, bound to the release workflow and refs/tags/; +# with cosign on PATH, gh verified nothing. So a dropped, unbound or extra +# verify call fails the case. +verified() { + local path="$1" tag="$2" a want=""; shift 2 + if [[ "$path" == "$GH_ONLY" ]]; then + for a in "$@"; do + want+="verify $a repo=$REPO signer=$REPO/.github/workflows/release-artifacts.yml ref=refs/tags/$tag"$'\n' + done + fi + [[ "$(sort "$GH_VERIFY_LOG")" == "$(printf '%s' "$want" | sort)" ]] && return 0 + printf '%s' "$want" | sort | sed 's/^/ want gh | /' >&2 + sort "$GH_VERIFY_LOG" | sed 's/^/ got gh | /' >&2 + return 1 +} +CLI_AMD64=bpfcompat-linux-amd64 +VAL_AMD64=bpfcompat-validator-static-linux-amd64 # ---------------------------------------------------------------- fixtures -for s in stable arm64 nosha noverify mismatch bindcosign bindgh trunc dirs badver badlatest ghcap; do +for s in stable arm64 nosha noverify mismatch bindcosign bindgh bindghcli bindghval trunc dirs badver badlatest ghcap; do release "$s" v0.3.7 amd64 release "$s" v0.4.0-rc.4 amd64 latest "$s" v0.3.7 @@ -274,7 +308,8 @@ for path in "$WITH_COSIGN" "$GH_ONLY"; do run stable "$path" BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" if [[ $rc -eq 0 ]] && grep -q 'fake bpfcompat v0.3.7 amd64' "$OUT" && grep -qx 'validator v0.3.7 amd64' "$d/libexec/bpfcompat-validator" && - [[ ! -e "$d/share" ]] && grep -q 'publishes no third-party license bundle' "$OUT"; then + [[ ! -e "$d/share" ]] && grep -q 'publishes no third-party license bundle' "$OUT" && + verified "$path" v0.3.7 "$CLI_AMD64" "$VAL_AMD64"; then ok "latest -> v0.3.7 (PATH=${path%%:*}), no license bundle in that release" else fail "latest did not install v0.3.7 (rc=$rc)"; show; fi done @@ -283,7 +318,8 @@ done for path in "$WITH_COSIGN" "$GH_ONLY"; do d="$tmp/i-rc-${#path}" run stable "$path" BPFCOMPAT_VERSION=v0.4.0-rc.4 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" - if [[ $rc -eq 0 ]] && grep -q 'fake bpfcompat v0.4.0-rc.4 amd64' "$OUT"; then ok "pinned v0.4.0-rc.4" + if [[ $rc -eq 0 ]] && grep -q 'fake bpfcompat v0.4.0-rc.4 amd64' "$OUT" && + verified "$path" v0.4.0-rc.4 "$CLI_AMD64" "$VAL_AMD64"; then ok "pinned v0.4.0-rc.4" else fail "pinned prerelease did not install (rc=$rc)"; show; fi done @@ -295,7 +331,8 @@ if [[ $rc -eq 0 ]] && grep -q 'fake bpfcompat v0.3.7 arm64' "$OUT" && [[ ! -e "$ else fail "aarch64 mapping (rc=$rc)"; show; fi d="$tmp/i-amd64" run arm64 "$GH_ONLY" FAKE_ARCH=amd64 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" -if [[ $rc -eq 0 ]] && grep -q 'fake bpfcompat v0.3.7 amd64' "$OUT" && [[ -x "$d/libexec/bpfcompat-validator" ]]; then +if [[ $rc -eq 0 ]] && grep -q 'fake bpfcompat v0.3.7 amd64' "$OUT" && [[ -x "$d/libexec/bpfcompat-validator" ]] && + verified "$GH_ONLY" v0.3.7 "$CLI_AMD64" "$VAL_AMD64"; then ok "amd64 with extra SHA256SUMS entries" else fail "amd64 with extra SHA256SUMS entries (rc=$rc)"; show; fi d="$tmp/i-mips" @@ -332,6 +369,21 @@ run bindgh "$GH_ONLY" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOM if [[ $rc -ne 0 && ! -e "$d" ]] && grep -q 'attestation verification FAILED' "$OUT"; then ok "gh: rc.4 attestation refused for v0.3.7" else fail "gh accepted a binary attested for another tag (rc=$rc)"; show; fi +# 4b. Each gh verify call stands on its own: swap only the CLI, or only the +# validator, for rc.4's attested bytes and regenerate SHA256SUMS to match, +# so the attestation is the only thing that can catch it. Dropping or +# neutralising that asset's verify call makes this install succeed. (The +# license bundle's call is covered by the unattested-bundle case in 8.) +for c in "bindghcli $CLI_AMD64" "bindghval $VAL_AMD64"; do + scen="${c%% *}" asset="${c#* }" + cp "$(rel_dir "$scen" v0.4.0-rc.4)/$asset" "$(rel_dir "$scen" v0.3.7)/$asset" + (cd "$(rel_dir "$scen" v0.3.7)" && "$REAL_SHA" bpfcompat-* >SHA256SUMS) + d="$tmp/i-$scen" + run "$scen" "$GH_ONLY" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" + if [[ $rc -ne 0 && ! -e "$d" ]] && grep -q "attestation verification FAILED for $asset\$" "$OUT"; then + ok "gh: only $asset attested for rc.4 -> refused for v0.3.7" + else fail "gh accepted $asset attested for another tag (rc=$rc)"; show; fi +done # 5. INST-3: a truncated `curl | sh` stream must not run any part of the install. total="$(wc -l <"$INSTALL")" @@ -406,7 +458,8 @@ for path in "$WITH_COSIGN" "$GH_ONLY"; do run lic "$path" BPFCOMPAT_VERSION=v0.4.0-rc.5 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" if [[ $rc -eq 0 ]] && grep -q '"version": "v0.4.0-rc.5"' "$d/share/doc/bpfcompat/v0.4.0-rc.5/INVENTORY.json" && [[ -s "$d/share/doc/bpfcompat/v0.4.0-rc.5/go/toolchain/LICENSE" ]] && - grep -q 'bpfcompat-third-party-licenses.tar.gz' "$OUT"; then + grep -q 'bpfcompat-third-party-licenses.tar.gz' "$OUT" && + verified "$path" v0.4.0-rc.5 "$CLI_AMD64" "$VAL_AMD64" bpfcompat-third-party-licenses.tar.gz; then ok "license bundle verified and installed next to the CLI (PATH=${path%%:*})" else fail "license bundle install (rc=$rc)"; show; fi done @@ -469,14 +522,26 @@ gh_lacking attestation 2.45.0 "installed gh 2.45.0 lacks the 'gh attestation' co gh_lacking --source-ref 2.67.0 "installed gh 2.67.0 lacks --source-ref in 'gh attestation verify', needed for" gh_lacking "--signer-workflow --source-ref" 2.50.0 "installed gh 2.50.0 lacks --signer-workflow and --source-ref in 'gh attestation verify'" gh_lacking --source-ref "custom-build" "installed gh lacks --source-ref in 'gh attestation verify'" -for lacks in attestation --source-ref; do - d="$tmp/i-ghcap-cosign-${lacks#--}" - run ghcap "$WITH_COSIGN" FAKE_GH_LACKS="$lacks" FAKE_GH_VERSION=2.45.0 \ +# A gh that cannot run at all (every command fails, as with an unreadable +# config) is reported as such, with gh's own error, not as lacking a command. +d="$tmp/i-ghbroken" bad="" +run ghcap "$GH_ONLY" FAKE_GH_BROKEN=1 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" +[[ $rc -ne 0 && ! -e "$d" ]] || bad="$bad installed-or-rc0" +grep -qF "installed gh could not run 'gh --help' (failed to create root command: failed to read configuration: invalid config file" "$OUT" || + bad="$bad reason-not-named" +grep -qF 'fix gh or install cosign - refusing checksum-only installation' "$OUT" || bad="$bad no-remedy" +! grep -qE 'lacks|FAILED|requires cosign or GitHub CLI' "$OUT" || bad="$bad wrong-reason" +! grep -qE '^attestation verify [^-]' "$GH_LOG" || bad="$bad verify-attempted" +if [[ -z "$bad" ]]; then ok "gh that cannot run: refused, naming gh's own error" +else fail "gh that cannot run (rc=$rc):$bad"; show; fi +for ghenv in FAKE_GH_LACKS=attestation FAKE_GH_LACKS=--source-ref FAKE_GH_BROKEN=1; do + d="$tmp/i-ghcap-cosign-${ghenv//[^A-Za-z]/}" + run ghcap "$WITH_COSIGN" "$ghenv" FAKE_GH_VERSION=2.45.0 \ BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" if [[ $rc -eq 0 && -x "$d/bin/bpfcompat" && -x "$d/libexec/bpfcompat-validator" ]] && grep -q 'cosign signature over SHA256SUMS verified' "$OUT"; then - ok "gh lacking '$lacks' + cosign: verified with cosign and installed" - else fail "gh lacking '$lacks' with cosign present (rc=$rc)"; show; fi + ok "gh with $ghenv + cosign: verified with cosign and installed" + else fail "gh with $ghenv and cosign present (rc=$rc)"; show; fi done if [[ $failures -ne 0 ]]; then From 7f7a4effe2860cc4ee4219b5641d14635afc7b8d Mon Sep 17 00:00:00 2001 From: ErenAri Date: Mon, 19 Oct 2026 00:15:39 +0300 Subject: [PATCH 24/38] readiness: bind the looked-up drill run to this repository, dispatch and time Review of e033fe6: the drill lookup accepted a fork's pull-request run from a branch named main, any old rollback-drill run, and a note whose Completed time was unrelated to the run (2099). The drill run must now be a dispatched run of this repository that started after the candidate was built, with the note's Completed time inside it; the incident run must be a dispatched promote-release run of this repository. The runbook says the lookups need the network and an authenticated gh. Signed-off-by: ErenAri --- docs/production-release-process.md | 15 +++++++++------ scripts/production-readiness-report.sh | 14 ++++++++++++-- scripts/production-readiness-report_test.sh | 16 +++++++++++++--- 3 files changed, 34 insertions(+), 11 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index 3852a77..2c924dc 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -194,15 +194,17 @@ aggregator binds every piece of evidence to the attested candidate `ghcr.io/kernel-guard/bpfcompat:rollback-drill-vX.Y.Z-rc.N`, `- Candidate digest:` is the candidate image digest, `- Final digest:` equals a distinct `- Known-good digest:`, and `- Completed:` is after - `generated_at`. A drill of an earlier candidate is refused. + `generated_at`. A drill of an earlier candidate is refused. With attestation + verification on, `- Workflow run:` must be a successful, dispatched + `rollback-drill.yml` run on this repository's `main` that started after + `generated_at`, and `- Completed:` must fall inside that run. - **Incident.** The note must contain `[bpfcompat-promotion-incident:v1]`, name `promote-release`, and link the deliberately rejected run, and only that run, as `https://github.com/Kernel-Guard/bpfcompat/actions/runs/`. With - attestation verification on, that run must be a completed `promote-release.yml` - run that did not succeed, and the rollback evidence's workflow run must be a - successful `rollback-drill.yml` run on `main`; both are looked up on GitHub. - Written policy does not require the incident drill to target the current + attestation verification on, that run must be a completed, dispatched + `promote-release.yml` run on this repository that did not succeed. Written + policy does not require the incident drill to target the current candidate, so an earlier candidate's fail-closed drill is accepted; the output records `incident.promotion_run_id` and `incident.candidate_bound`. @@ -226,7 +228,8 @@ The command writes both Markdown and JSON. Add the reviewed outputs to request. The stable release workflow validates, checksums, attests, and publishes those files; a stable tag fails if they are absent or not `ready`. The online candidate check requires an authenticated GitHub CLI version that -provides `gh attestation`; an older CLI fails closed. +provides `gh attestation`, and the drill and incident runs are read through the +GitHub API; an older or unauthenticated CLI, or no network, fails closed. Do not use `BPFCOMPAT_SKIP_READINESS_ATTESTATION=1` outside the regression test; production evidence must verify the candidate attestation online. diff --git a/scripts/production-readiness-report.sh b/scripts/production-readiness-report.sh index 0ffd9eb..6b47f08 100755 --- a/scripts/production-readiness-report.sh +++ b/scripts/production-readiness-report.sh @@ -555,13 +555,23 @@ if [[ "${BPFCOMPAT_SKIP_READINESS_ATTESTATION:-0}" != "1" ]]; then rollback_run="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${rollback_run_id}" 2>/dev/null)" || fail "rollback drill run ${rollback_run_id} cannot be read from GitHub" jq -e '.path == ".github/workflows/rollback-drill.yml" and .head_branch == "main" and + .event == "workflow_dispatch" and .head_repository.full_name == "Kernel-Guard/bpfcompat" and .status == "completed" and .conclusion == "success"' <<<"$rollback_run" >/dev/null || - fail "rollback drill run ${rollback_run_id} is not a successful rollback-drill.yml run on main" + fail "rollback drill run ${rollback_run_id} is not a successful, dispatched rollback-drill.yml run on this repository's main" + # The run itself started after the candidate was built, and the note's + # Completed time falls inside it. + rollback_run_started="$(iso_epoch "$(jq -r '.run_started_at // ""' <<<"$rollback_run")" "rollback drill run start")" + rollback_run_updated="$(iso_epoch "$(jq -r '.updated_at // ""' <<<"$rollback_run")" "rollback drill run end")" + [[ "$rollback_run_started" -gt "$candidate_generated_epoch" ]] || + fail "rollback drill run ${rollback_run_id} started before release candidate ${candidate_tag} was built" + [[ "$rollback_completed_epoch" -ge "$rollback_run_started" && "$rollback_completed_epoch" -le "$rollback_run_updated" ]] || + fail "rollback drill note's Completed time is outside run ${rollback_run_id}" incident_run="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${incident_run_id}" 2>/dev/null)" || fail "incident run ${incident_run_id} cannot be read from GitHub" jq -e '.path == ".github/workflows/promote-release.yml" and + .event == "workflow_dispatch" and .head_repository.full_name == "Kernel-Guard/bpfcompat" and .status == "completed" and .conclusion != "success"' <<<"$incident_run" >/dev/null || - fail "incident run ${incident_run_id} is not a rejected promote-release.yml run" + fail "incident run ${incident_run_id} is not a rejected, dispatched promote-release.yml run on this repository" fi incident_candidate_bound=false if grep -qE "(^|[^[:alnum:].-])${candidate_tag//./\\.}([^[:alnum:]]|$)" "$incident"; then diff --git a/scripts/production-readiness-report_test.sh b/scripts/production-readiness-report_test.sh index 8b80bf3..013e698 100755 --- a/scripts/production-readiness-report_test.sh +++ b/scripts/production-readiness-report_test.sh @@ -633,10 +633,15 @@ if [[ "$1" == api && "$2" =~ ^repos/Kernel-Guard/bpfcompat/actions/runs/([0-9]+) case "${BASH_REMATCH[1]}" in 5000) jq -n --arg p "${STUB_ROLLBACK_PATH:-.github/workflows/rollback-drill.yml}" \ --arg b "${STUB_ROLLBACK_BRANCH:-main}" --arg c "${STUB_ROLLBACK_CONCLUSION:-success}" \ - '{path: $p, head_branch: $b, status: "completed", conclusion: $c}' ;; + --arg e "${STUB_ROLLBACK_EVENT:-workflow_dispatch}" --arg r "${STUB_ROLLBACK_REPO:-Kernel-Guard/bpfcompat}" \ + --arg s "${STUB_ROLLBACK_STARTED:-2026-07-30T15:55:00Z}" --arg u "${STUB_ROLLBACK_UPDATED:-2026-07-30T16:01:00Z}" \ + '{path: $p, head_branch: $b, event: $e, head_repository: {full_name: $r}, + run_started_at: $s, updated_at: $u, status: "completed", conclusion: $c}' ;; 6000) jq -n --arg p "${STUB_INCIDENT_PATH:-.github/workflows/promote-release.yml}" \ - --arg c "${STUB_INCIDENT_CONCLUSION:-failure}" \ - '{path: $p, head_branch: "v0.4.0-rc.1", status: "completed", conclusion: $c}' ;; + --arg c "${STUB_INCIDENT_CONCLUSION:-failure}" --arg e "${STUB_INCIDENT_EVENT:-workflow_dispatch}" \ + --arg r "${STUB_INCIDENT_REPO:-Kernel-Guard/bpfcompat}" \ + '{path: $p, head_branch: "v0.4.0-rc.1", event: $e, head_repository: {full_name: $r}, + status: "completed", conclusion: $c}' ;; *) echo "gh: Not Found (HTTP 404)" >&2; exit 1 ;; esac exit 0 @@ -685,6 +690,11 @@ for case in \ "STUB_ROLLBACK_PATH=.github/workflows/ci.yml|a rollback drill run that is another workflow" \ "STUB_ROLLBACK_CONCLUSION=failure|a rollback drill run that did not succeed" \ "STUB_ROLLBACK_BRANCH=feature|a rollback drill run not on main" \ + "STUB_ROLLBACK_REPO=someone/bpfcompat|a rollback drill run from a fork" \ + "STUB_ROLLBACK_EVENT=pull_request|a rollback drill run that was not dispatched" \ + "STUB_ROLLBACK_STARTED=2026-07-30T13:00:00Z|a rollback drill run that started before the candidate" \ + "STUB_ROLLBACK_UPDATED=2026-07-30T15:59:00Z|a rollback note whose completion is outside its run" \ + "STUB_INCIDENT_REPO=someone/bpfcompat|an incident linking a run from a fork" \ "STUB_INCIDENT_PATH=.github/workflows/ci.yml|an incident linking a run of another workflow" \ "STUB_INCIDENT_CONCLUSION=success|an incident linking a promotion that succeeded"; do if env "${case%%|*}" PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" "$script" "$tmp/input.json" "$tmp/bad.md" >/dev/null 2>&1; then From 6de1bb7a0e4c911daaa3e835d67f8165a04df087 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Mon, 19 Oct 2026 02:33:44 +0300 Subject: [PATCH 25/38] docs: the v0.3.8 candidates' Action fails without attestation support Review of 35d6ebe: the Action falls back to a source build when gh cannot verify attestations only in the v0.4.0 release candidates; the v0.3.8 candidates' Action fails the job instead. Also name the error an older gh prints before 2.49.0 (unknown command "attestation"). Signed-off-by: ErenAri --- docs/verifying-releases.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/verifying-releases.md b/docs/verifying-releases.md index a051228..ca12468 100644 --- a/docs/verifying-releases.md +++ b/docs/verifying-releases.md @@ -22,9 +22,10 @@ long-lived private key. The signer identity is the release workflow itself. ## 1. Verify build provenance (recommended) Requires the GitHub CLI (`gh`) 2.68.0 or later: `--signer-workflow` was added -in gh 2.51.0 and `--source-ref` in gh 2.68.0, and an older gh stops with -`unknown flag` without verifying anything (upgrade it, or use the cosign check -in section 2). How strictly gh matches these values also depends on its +in gh 2.51.0 and `--source-ref` in gh 2.68.0. An older gh stops without +verifying anything, with `unknown flag` (before 2.68.0) or +`unknown command "attestation"` (before 2.49.0); upgrade it, or use the cosign +check in section 2. How strictly gh matches these values also depends on its version: gh 2.97.0 and 2.102.0 fixed published advisories in that matching (GHSA-mm27-mwq9-fr5g, GHSA-wjmr-j3rp-mh2g, GHSA-4mq3-hpgx-9cx8), so 2.102.0 or later is recommended. @@ -134,8 +135,10 @@ and, from v0.4.0-rc.5, for the third-party license bundle when `SHA256SUMS` lists it. Attestation verification is restricted to `.github/workflows/release-artifacts.yml`; a checksum or attestation mismatch -is a hard failure. When attestation verification is unavailable, the Action -builds from its pinned source instead of trusting unverifiable release bytes. +is a hard failure. When attestation verification is unavailable (no `gh` with +attestation support), the v0.4.0 release candidates' Action builds from its +pinned source instead of trusting unverifiable release bytes; the v0.3.8 +release candidates' Action fails the job instead. From v0.4.0-rc.5, after verification the Action runs that validator by path and pinned digest (`BPFCOMPAT_VALIDATOR_BIN` / `BPFCOMPAT_VALIDATOR_SHA256`), From 2269647bdb10954a603ea62e2b8e41fa0be91815 Mon Sep 17 00:00:00 2001 From: ErenAri Date: Mon, 19 Oct 2026 02:37:26 +0300 Subject: [PATCH 26/38] docs: gh reports unknown command "attestation" only before 2.47.0 gh 2.47.0 and 2.48.0 already run the hidden attestation command, so they fail with unknown flag, like every gh before 2.68.0. Signed-off-by: ErenAri --- docs/verifying-releases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/verifying-releases.md b/docs/verifying-releases.md index ca12468..8f6e11f 100644 --- a/docs/verifying-releases.md +++ b/docs/verifying-releases.md @@ -24,7 +24,7 @@ long-lived private key. The signer identity is the release workflow itself. Requires the GitHub CLI (`gh`) 2.68.0 or later: `--signer-workflow` was added in gh 2.51.0 and `--source-ref` in gh 2.68.0. An older gh stops without verifying anything, with `unknown flag` (before 2.68.0) or -`unknown command "attestation"` (before 2.49.0); upgrade it, or use the cosign +`unknown command "attestation"` (before 2.47.0); upgrade it, or use the cosign check in section 2. How strictly gh matches these values also depends on its version: gh 2.97.0 and 2.102.0 fixed published advisories in that matching (GHSA-mm27-mwq9-fr5g, GHSA-wjmr-j3rp-mh2g, GHSA-4mq3-hpgx-9cx8), so 2.102.0 or From b9149e7188df116b553ad3664f6103ed38f4519e Mon Sep 17 00:00:00 2001 From: ErenAri Date: Mon, 19 Oct 2026 02:37:40 +0300 Subject: [PATCH 27/38] release: bind the stable window to the canary pin; fix the channel test Review of 2273204 (full stable-sequence simulation): - DEF-4: check-release-consistency_test.sh still asserted that the current tree is on the prerelease channel, so the stable PR's required CI, the push to main and the v0.4.0 tag build's quality gates would fail. It now reads release_channel from release.yaml. - The stable window's second documented version is now the release the consumer canary's '# bpfcompat-pin: stable' step names, which check-canary-pins.sh verifies is the newest published stable release, so the two checks cannot disagree (a stable tag without a published release no longer qualifies), and no remote tag listing is involved. - Only commit-SHA pins count toward that second version; a tag-form pin on the previous release is refused as before. - The ci.yml step comment and the runbook describe the window precisely; docs name an unreleased version until promotion, not only until the tag. Signed-off-by: ErenAri --- .github/workflows/ci.yml | 7 ++-- docs/production-release-process.md | 9 ++--- scripts/check-doc-pins.sh | 40 ++++++++++++++--------- scripts/check-doc-pins_test.sh | 24 ++++++++++++++ scripts/check-release-consistency_test.sh | 3 +- 5 files changed, 59 insertions(+), 24 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7bf7091..cae4cb8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -62,9 +62,10 @@ jobs: # weeks because nothing checked the pins a reader copy-pastes. run: bash scripts/check-doc-pins.sh - name: Documented action pins regression - # The stable release PR documents vX.Y.Z before its tag exists; only - # that tag, only in the stable channel, and only when the remote - # positively lacks it. + # The stable release window: docs may name vX.Y.Z before its tag + # exists (only that tag, only in the stable channel, only when the + # remote positively lacks it), and SHA-pinned templates may stay on + # the release the consumer canary's stable pin names until the tag. run: bash scripts/check-doc-pins_test.sh - name: Action OCI-artifact detection regression # Guards the fix that lets the action accept remote OCI references diff --git a/docs/production-release-process.md b/docs/production-release-process.md index 9d145d4..5f4fa37 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -252,13 +252,14 @@ Publish no other stable release between the stable pull request's CI and the release below `X.Y.Z`, so one published in that window (a maintenance release, for example) fails the tag build. Re-run `make check-release-consistency` on the exact commit immediately before tagging, and tag the merge commit as soon -as the pull request merges: from then until the tag exists, `main`'s docs name -a release that is not out yet. In the stable pull request, the tag-form action +as the pull request merges: from then until promotion, `main`'s docs name a +release that is not published yet. In the stable pull request, the tag-form action references in `README.md` and `docs/` name `vX.Y.Z`, which `scripts/check-doc-pins.sh` accepts while that tag positively does not exist. The SHA-pinned integration templates in `docs/integrations/` cannot name it -yet; they stay on the most recent earlier stable release during the stable -pull request and the tag build. +yet; during the stable pull request and the tag build they stay, pinned by +SHA, on the release the consumer canary's `# bpfcompat-pin: stable` step names +(the most recent published stable release before `X.Y.Z`). The stable release pull request and the `vX.Y.Z` tag build cannot pin a tag that does not exist yet or that names their own commit, so they keep the diff --git a/scripts/check-doc-pins.sh b/scripts/check-doc-pins.sh index f0584be..4189989 100755 --- a/scripts/check-doc-pins.sh +++ b/scripts/check-doc-pins.sh @@ -29,9 +29,11 @@ # The integration templates are pinned by commit SHA (docs/integrations), so # they cannot name vX.Y.Z until its tag exists. During the same stable window # -- the stable PR while vX.Y.Z is absent, and the vX.Y.Z tag build itself -- -# rule 3 allows exactly one other version beside v: the most -# recent earlier stable release tag (vA.B.C < X.Y.Z). Every later commit is -# strict again, so the templates are repinned once the tag exists. +# rule 3 allows exactly one other version beside v, and only +# as commit-SHA pins: the earlier release the consumer canary's stable pin +# names (scripts/check-canary-pins.sh verifies that pin is the newest published +# stable release before X.Y.Z). Every later commit is strict again, so the +# templates are repinned with the canary once the tag exists. set -euo pipefail cd "$(dirname "${BASH_SOURCE[0]}")/.." @@ -64,6 +66,7 @@ release_value() { awk -F': *' -v k="$1" '$1 == k { gsub(/["\r]/, "", $2); print preparing="" [[ "$(release_value release_channel)" == stable ]] && preparing="v$(release_value stable_version)" prepared="" +declare -A tag_form=() # version_lt A B: X.Y.Z version A sorts strictly before X.Y.Z version B. version_lt() { @@ -80,18 +83,18 @@ version_lt() { return 1 } -# previous_stable : the newest vA.B.C tag on the remote below it, or -# nothing when the remote cannot be listed. -previous_stable() { - local tags tag newest="" - tags="$(git ls-remote --tags --refs origin 'refs/tags/v*' 2>/dev/null)" || return 0 - while read -r _ tag; do - tag="${tag#refs/tags/}" - [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || continue - version_lt "${tag#v}" "${1#v}" || continue - if [[ -z "$newest" ]] || version_lt "${newest#v}" "${tag#v}"; then newest="$tag"; fi - done <<<"$tags" - printf '%s' "$newest" +# canary_stable_version: the `# vA.B.C` comment on the action pin that follows +# the `# bpfcompat-pin: stable` marker in the consumer canary, or nothing. +canary_stable_version() { + awk ' + /#[[:space:]]*bpfcompat-pin:[[:space:]]*stable[[:space:]]*$/ { want = 1; next } + want && /uses:[[:space:]]*Kernel-Guard\/bpfcompat@/ { + if (match($0, /#[[:space:]]*v[0-9]+\.[0-9]+\.[0-9]+[[:space:]]*$/)) { + v = substr($0, RSTART, RLENGTH); sub(/^#[[:space:]]*/, "", v); sub(/[[:space:]]*$/, "", v); print v + } + exit + } + ' .github/workflows/consumer-canary.yml 2>/dev/null || true } # tag_absent : the remote answered and has no such tag (git ls-remote @@ -134,6 +137,7 @@ while IFS= read -r hit; do fi elif [[ "$ref" =~ ^v[0-9]+\.[0-9]+\.[0-9]+ ]]; then version="$ref" + tag_form["$version"]=1 if [[ -z "$(tag_commit "$version")" ]]; then if [[ -n "$preparing" && "$version" == "$preparing" ]] && tag_absent "$version"; then prepared="$version" @@ -162,7 +166,11 @@ window_previous="" if [[ -n "$preparing" && ${#versions[@]} -eq 2 && -n "${versions[$preparing]:-}" ]]; then head_commit="$(git rev-parse -q --verify 'HEAD^{commit}' 2>/dev/null || true)" if [[ -n "$prepared" ]] || [[ -n "$head_commit" && "$(tag_commit "$preparing")" == "$head_commit" ]]; then - window_previous="$(previous_stable "$preparing")" + candidate_previous="$(canary_stable_version)" + if [[ "$candidate_previous" =~ ^v([0-9]+\.[0-9]+\.[0-9]+)$ ]] && + version_lt "${BASH_REMATCH[1]}" "${preparing#v}" && [[ -z "${tag_form[$candidate_previous]:-}" ]]; then + window_previous="$candidate_previous" + fi fi fi if (( ${#versions[@]} == 2 )) && [[ -n "$window_previous" && -n "${versions[$window_previous]:-}" ]]; then diff --git a/scripts/check-doc-pins_test.sh b/scripts/check-doc-pins_test.sh index ade9c52..173e5ea 100755 --- a/scripts/check-doc-pins_test.sh +++ b/scripts/check-doc-pins_test.sh @@ -21,6 +21,13 @@ fixture() { cp "$ROOT_DIR/scripts/check-doc-pins.sh" "$d/work/scripts/" printf 'stable_version: %s\nrelease_version: %s\nrelease_channel: %s\n' "$stable" "$stable" "$channel" >"$d/work/release.yaml" printf '# x\n```yaml\n - %s\n```\n' "$pin" >"$d/work/README.md" + # The consumer canary's stable pin names the release the stable window keeps + # (CANARY, default v0.3.7); NO_CANARY leaves the file out. + if [[ -z "${NO_CANARY:-}" ]]; then + mkdir -p "$d/work/.github/workflows" + printf 'jobs:\n s:\n steps:\n # bpfcompat-pin: stable\n - uses: Kernel-Guard/bpfcompat@%s # %s\n' \ + "$(printf 'a%.0s' {1..40})" "${CANARY:-v0.3.7}" >"$d/work/.github/workflows/consumer-canary.yml" + fi gitq -C "$d/work" add -A gitq -C "$d/work" commit -m base for t in "$@"; do gitq -C "$d/work" tag -a "$t" -m "$t"; done @@ -102,6 +109,23 @@ fixture tplafter stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 v0.4. template v0.3.7 expect fail "after the tag: templates must be repinned to the new release" "docs pin more than one bpfcompat version" +fixture tplform stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 +mkdir -p "$W/docs"; printf 'jobs:\n t:\n steps:\n - uses: "Kernel-Guard/bpfcompat@v0.3.7"\n' >"$W/docs/stale.yml" +expect fail "stable PR: a tag-form pin on the previous stable is not a SHA-pinned template" "docs pin more than one bpfcompat version" + +CANARY=v0.3.6 fixture tplcanary stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.6 v0.3.7 +template v0.3.7 +expect fail "stable PR: templates must stay on the release the canary pins" "docs pin more than one bpfcompat version" + +fixture tplbuildwrong stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.6 v0.3.7 +template v0.3.6 +tag_head v0.4.0 +expect fail "tag build: templates on another release than the canary's fail" "docs pin more than one bpfcompat version" + +NO_CANARY=1 fixture tplnocanary stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 +template v0.3.7 +expect fail "stable PR: without a canary stable pin there is no window" "docs pin more than one bpfcompat version" + fixture tplrc prerelease 0.3.7 'uses: Kernel-Guard/bpfcompat@v0.3.7' v0.3.6 v0.3.7 template v0.3.6 expect fail "prerelease channel: two documented versions fail as before" "docs pin more than one bpfcompat version" diff --git a/scripts/check-release-consistency_test.sh b/scripts/check-release-consistency_test.sh index b3576ac..495f9f6 100755 --- a/scripts/check-release-consistency_test.sh +++ b/scripts/check-release-consistency_test.sh @@ -54,7 +54,8 @@ EOF } BPFCOMPAT_RELEASE_METADATA=release.yaml "$script" >"$tmp/current.log" -grep -Fq 'channel=prerelease' "$tmp/current.log" +current_channel="$(awk -F': ' '$1 == "release_channel" {print $2; exit}' release.yaml)" +grep -Fq "channel=${current_channel}" "$tmp/current.log" # The current stable release as a stable channel: docs and canary pins must # already agree with it (the stable PR changes stable_version, so this follows). From 4cb4516bda42cf4877397518c5f780ded8dfe54a Mon Sep 17 00:00:00 2001 From: ErenAri Date: Mon, 19 Oct 2026 02:55:00 +0300 Subject: [PATCH 28/38] release: read the canary's first version comment, as check-canary-pins does Review of b9149e7: on a canary line with two version comments ("# v0.3.7 # v0.3.6"), check-canary-pins.sh read the first and check-doc-pins.sh the last, so the stable window could admit templates on a different release than the verified canary pin. Both now read the first, and a test pins it. Signed-off-by: ErenAri --- scripts/check-doc-pins.sh | 6 ++++-- scripts/check-doc-pins_test.sh | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/scripts/check-doc-pins.sh b/scripts/check-doc-pins.sh index 4189989..8c8eebe 100755 --- a/scripts/check-doc-pins.sh +++ b/scripts/check-doc-pins.sh @@ -89,8 +89,10 @@ canary_stable_version() { awk ' /#[[:space:]]*bpfcompat-pin:[[:space:]]*stable[[:space:]]*$/ { want = 1; next } want && /uses:[[:space:]]*Kernel-Guard\/bpfcompat@/ { - if (match($0, /#[[:space:]]*v[0-9]+\.[0-9]+\.[0-9]+[[:space:]]*$/)) { - v = substr($0, RSTART, RLENGTH); sub(/^#[[:space:]]*/, "", v); sub(/[[:space:]]*$/, "", v); print v + # The first "# v..." comment, exactly as scripts/check-canary-pins.sh + # reads it, so the two checks always see the same version. + if (match($0, /#[[:space:]]*v[0-9][0-9A-Za-z._-]*/)) { + v = substr($0, RSTART, RLENGTH); sub(/^#[[:space:]]*/, "", v); print v } exit } diff --git a/scripts/check-doc-pins_test.sh b/scripts/check-doc-pins_test.sh index 173e5ea..f64f6fb 100755 --- a/scripts/check-doc-pins_test.sh +++ b/scripts/check-doc-pins_test.sh @@ -122,6 +122,10 @@ template v0.3.6 tag_head v0.4.0 expect fail "tag build: templates on another release than the canary's fail" "docs pin more than one bpfcompat version" +CANARY='v0.3.7 # v0.3.6' fixture tpltwocomments stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.6 v0.3.7 +template v0.3.6 +expect fail "stable PR: the canary's first version comment counts, as in check-canary-pins" "docs pin more than one bpfcompat version" + NO_CANARY=1 fixture tplnocanary stable 0.4.0 'uses: Kernel-Guard/bpfcompat@v0.4.0' v0.3.7 template v0.3.7 expect fail "stable PR: without a canary stable pin there is no window" "docs pin more than one bpfcompat version" From 723b5e2b4a04ab740978044c8b133894c13e65ca Mon Sep 17 00:00:00 2001 From: ErenAri Date: Mon, 19 Oct 2026 08:40:21 +0300 Subject: [PATCH 29/38] install.sh: report gh auth and API failures apart from a refused verification On the gh path every `gh attestation verify` failure was reported as "GitHub attestation verification FAILED for ", with gh's output sent to /dev/null. A capable gh that is not logged in, holds a rejected token, or cannot reach the GitHub API got the same tampering-like words as a release whose attestation does not match (review S2). The installer now keeps four outcomes apart, and every one still refuses (exit 1, nothing installed, temp dir removed, no checksum-only fallback, no retry): 1. gh lacks the attestation command or flags: the existing help probe, "installed gh lacks ...", unchanged. 2. No usable credentials: "GitHub authentication needed for attestation verification: gh is not logged in ()", or "GitHub rejected gh's credentials (HTTP 401)". 3. GitHub API out of reach: "GitHub API unavailable for attestation verification: no HTTP response | HTTP 5xx | HTTP 429, rate limited | HTTP 403, rate limited ()". 4. Anything else, including no attestation for the digest (404), another signer workflow or source ref, or a digest mismatch: the existing "GitHub attestation verification FAILED for ", unchanged. Classes 2 and 3 rest only on positive evidence from one read-only request made before the first verify call: `gh api --method GET --include repos//attestations/sha256:?per_page=1`, the endpoint that call queries, through the same gh, host, token and proxy. gh exits 4 only from its root auth check (internal/ghcmd/cmd.go, pkg/cmd/root/root.go at v2.68.0 and v2.102.0), which also guards `attestation verify` (only --bundle lifts it). With --include the first stdout line is the status line gh builds from the response, and a transport failure exits 1 with empty stdout. A 403 counts only with GitHub's X-Ratelimit-Remaining: 0 or Retry-After header. Any other answer leaves the decision to the verify calls, unchanged (same assets, --repo, --signer-workflow, --source-ref refs/tags/), whose failures other than exit code 4 always read "verification FAILED". gh's verify text is never parsed: with stdout not a terminal it prints only its final error, worded differently by 2.68 and 2.102 for the same 401 or 404, and on 2.102 a dead proxy fails before the API fetch with a Sigstore message. Probing before verifying, not after a failure, means a refusal can never be relabelled as an auth or network problem when the verify itself spends a nearly exhausted rate limit or a token expires in between. Cost: one extra GET on the gh path (about 240 KB for v0.3.7's CLI, next to about 22 MB of downloads). The cosign path is unchanged and still runs no gh command. gh's words are quoted as one line with control characters removed, also in the "could not run 'gh --help'" refusal (review R2-N2), and the gh_err and gh_ver captures tolerate an imported pipefail (R2-N1). install_test.sh: the gh stub answers the probe from the attestation DB and can play a gh that is not logged in (exit 4), holds a rejected token (401), gets no response, a 503, a 429, a rate-limited 403 (also with coloured headers) or a plain 403, a verify that exits 4, and a refusing verify whose stderr claims a network or auth error. 35 new cases check the class named, the absence of the other classes' wording, nothing installed, the installer's temp dir removed, no control bytes, exactly one probe made before any verify call, the cosign path untouched in every mode, and imported pipefail. The 45 existing cases pass unchanged; against f8809a9's install.sh the new suite fails 24 cases. All 34 mutants of the change are caught (a refusal reworded as network or auth, a die removed on the auth or network path, a 404 taken for an outage, the probe ignored or removed, S1 regressions on the new calls). Real gh 2.68.0 and 2.102.0 (official tarballs) with the real v0.3.7 assets: no token and an invalid token give class 2, a dead HTTPS proxy class 3, a modified CLI or rc.4's assets served as v0.3.7 class 4, and the real release installs. f8809a9 said "FAILED" for the first three. Signed-off-by: ErenAri --- scripts/install.sh | 71 +++++++++-- scripts/install_test.sh | 273 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 319 insertions(+), 25 deletions(-) diff --git a/scripts/install.sh b/scripts/install.sh index 2021b38..041f27a 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -35,6 +35,23 @@ ALT_LIBEXEC_DIR="/usr/libexec/bpfcompat" err() { printf 'bpfcompat-install: %s\n' "$*" >&2; } die() { err "$*"; exit 1; } have() { command -v "$1" >/dev/null 2>&1; } +# first_line FILE: its first non-blank line, without control characters, to +# quote gh's own words in a refusal (never more, never terminal escapes). +first_line() { sed -n '/[^[:space:]]/{p;q;}' "$1" | tr -d '[:cntrl:]'; } +# gh_no_auth REASON FILE: gh cannot authenticate to GitHub; quote FILE's +# first line (gh's own words) and refuse. +gh_no_auth() { + gh_err="$(first_line "$2")" || : + die "GitHub authentication needed for attestation verification: $1${gh_err:+ ($gh_err)}; run 'gh auth login' or set a valid GH_TOKEN, or install cosign - refusing checksum-only installation" +} +# gh_refused ASSET STATUS: a `gh attestation verify` call failed. gh exits +# 4 only when it has no credentials at all (its root check, which runs +# before the asset is read), so that is no refusal; any other failure is +# reported as a failed verification, whatever gh printed. +gh_refused() { + [ "$2" != 4 ] || gh_no_auth "gh is not logged in" .gh-verify.out + die "GitHub attestation verification FAILED for $1" +} # needs_root DIR: true when DIR (or, if it does not exist yet, its nearest # existing ancestor, which `install -d` would create it under) is not writable. @@ -55,9 +72,10 @@ DOC_DIR="${BPFCOMPAT_DOC_DIR:-$(dirname "$BIN_DIR")/share/doc/bpfcompat}" LICENSE_ASSET="bpfcompat-third-party-licenses.tar.gz" have curl || die "curl is required" -if have sha256sum; then SHACHK="sha256sum -c"; elif have shasum; then SHACHK="shasum -a 256 -c"; else +if have sha256sum; then SHA256="sha256sum"; elif have shasum; then SHA256="shasum -a 256"; else die "need sha256sum or shasum to verify downloads" fi +SHACHK="$SHA256 -c" os="$(uname -s)"; arch="$(uname -m)" [ "$os" = "Linux" ] || die "unsupported OS '$os' (only Linux is published today)" @@ -158,7 +176,7 @@ elif have gh; then # A gh that cannot run at all (say, an unreadable config) fails every # command; say so with gh's own error rather than blame its version. if ! gh --help >/dev/null 2>&1; then - gh_err="$( { gh --help >/dev/null; } 2>&1 | sed -n '1p')" + gh_err="$( { gh --help >/dev/null; } 2>&1 | sed -n '1p' | tr -d '[:cntrl:]')" || : die "installed gh could not run 'gh --help'${gh_err:+ ($gh_err)}; fix gh or install cosign - refusing checksum-only installation" fi gh_lacks="the 'gh attestation' command" @@ -171,25 +189,60 @@ elif have gh; then [ -z "$gh_lacks" ] || gh_lacks="$gh_lacks in 'gh attestation verify'" fi if [ -n "$gh_lacks" ]; then - gh_ver="$(gh --version 2>/dev/null | sed -n '1s/^gh version \([0-9][0-9.]*\).*/\1/p')" + gh_ver="$(gh --version 2>/dev/null | sed -n '1s/^gh version \([0-9][0-9.]*\).*/\1/p')" || : die "installed gh${gh_ver:+ $gh_ver} lacks $gh_lacks, needed for tag-bound attestation verification; upgrade to gh >= 2.68.0 or install cosign - refusing checksum-only installation" fi err "cosign not found; verifying GitHub build attestations" + # A capable gh can still be unable to check anything: not logged in, its + # token rejected, or the GitHub API out of reach. Tell those apart from a + # refused verification without trusting gh's wording, which changes + # between releases and is all gh prints when stdout is not a terminal: + # before the first verify call, make one read-only request to the + # attestation API that call uses (same gh, host, token and proxy), and + # refuse here only on positive evidence - gh's documented auth exit code + # (4), an HTTP 401, no HTTP response at all, a 5xx, a 429 or a + # rate-limited 403. Any other answer (200, 404 = no attestation for this + # digest, ...) leaves the decision to the verify calls, whose failures are + # always reported as a failed verification. Probing first, not after a + # failure, means a refusal can never be relabelled as an auth or network + # problem, however gh words it or whatever the API does next. + gh_digest="$($SHA256 <"$CLI_ASSET" | sed -n '1s/^\([0-9a-fA-F]\{64\}\).*/\1/p')" || : + gh_rc=0 + gh api --method GET --include "repos/$REPO/attestations/sha256:$gh_digest?per_page=1" \ + >.gh-api.out 2>.gh-api.err || gh_rc=$? + gh_status="$(tr -d '\r' <.gh-api.out | sed -n '1s/^HTTP\/[0-9.]* \([0-9][0-9][0-9]\).*/\1/p')" || : + gh_down="" + case "$gh_rc:$gh_status" in + 4:) gh_no_auth "gh is not logged in" .gh-api.err ;; + *:401) gh_no_auth "GitHub rejected gh's credentials (HTTP 401)" .gh-api.err ;; + *:5[0-9][0-9]) gh_down="HTTP $gh_status" ;; + *:429) gh_down="HTTP 429, rate limited" ;; + *:403) + # A 403 is an outage only with GitHub's own rate-limit headers. + gh_rl="$(tr -d '\r\033' <.gh-api.out | sed -n '2,/^$/{s/\[[0-9;]*m//g;p;}' | + grep -ciE '^(x-ratelimit-remaining: *0|retry-after: *[0-9]+)$')" || : + [ "${gh_rl:-0}" = 0 ] || gh_down="HTTP 403, rate limited" ;; + 1:) [ -s .gh-api.out ] || gh_down="no HTTP response" ;; + esac + if [ -n "$gh_down" ]; then + gh_err="$(first_line .gh-api.err)" || : + die "GitHub API unavailable for attestation verification: $gh_down${gh_err:+ ($gh_err)}; check the network or proxy and https://www.githubstatus.com, then retry, or install cosign - refusing checksum-only installation" + fi GH_REPO="$REPO" gh attestation verify "$CLI_ASSET" --repo "$REPO" \ --signer-workflow "$REPO/.github/workflows/release-artifacts.yml" \ - --source-ref "refs/tags/$VERSION" >/dev/null 2>&1 || - die "GitHub attestation verification FAILED for $CLI_ASSET" + --source-ref "refs/tags/$VERSION" >.gh-verify.out 2>&1 || + gh_refused "$CLI_ASSET" "$?" if [ "$install_validator" = "1" ]; then GH_REPO="$REPO" gh attestation verify "$VALIDATOR_ASSET" --repo "$REPO" \ --signer-workflow "$REPO/.github/workflows/release-artifacts.yml" \ - --source-ref "refs/tags/$VERSION" >/dev/null 2>&1 || - die "GitHub attestation verification FAILED for $VALIDATOR_ASSET" + --source-ref "refs/tags/$VERSION" >.gh-verify.out 2>&1 || + gh_refused "$VALIDATOR_ASSET" "$?" fi if [ "$install_licenses" = "1" ]; then GH_REPO="$REPO" gh attestation verify "$LICENSE_ASSET" --repo "$REPO" \ --signer-workflow "$REPO/.github/workflows/release-artifacts.yml" \ - --source-ref "refs/tags/$VERSION" >/dev/null 2>&1 || - die "GitHub attestation verification FAILED for $LICENSE_ASSET" + --source-ref "refs/tags/$VERSION" >.gh-verify.out 2>&1 || + gh_refused "$LICENSE_ASSET" "$?" fi err "GitHub build attestations verified" else diff --git a/scripts/install_test.sh b/scripts/install_test.sh index 919828d..c7b5f14 100644 --- a/scripts/install_test.sh +++ b/scripts/install_test.sh @@ -16,7 +16,11 @@ # was verified, bound to the release workflow and tag. With FAKE_GH_LACKS # it plays an older gh that has no `attestation` command or no # --signer-workflow / --source-ref flag, failing like cobra does; with -# FAKE_GH_BROKEN=1, a gh that cannot run at all (say, a bad config); +# FAKE_GH_BROKEN=1, a gh that cannot run at all (say, a bad config). It +# also answers the installer's read-only API probe (`gh api --include`) +# from the attestation DB and records it, and can play a capable gh that +# is not logged in, holds a rejected token, cannot reach the API or gets +# a 5xx / 429 / rate-limited 403, modelled on real gh 2.68.0 / 2.102.0; # * sudo pretends to be root by unlocking the test's read-only tree. # # Cases: latest -> stable, arch mapping (incl. extra SHA256SUMS lines, the @@ -25,10 +29,15 @@ # `curl | sh` streams, custom install dirs, tag validation (the whole value, # embedded newlines included), the third-party license bundle (installed when # SHA256SUMS lists it, verified like the binaries, refused when -# corrupt/missing/unattested, absent in old releases), and a gh too old for +# corrupt/missing/unattested, absent in old releases), a gh too old for # the tag-bound check (refused naming what it lacks, never reported as a # failed verification or a missing gh; a gh that cannot run is reported as -# such; cosign still preferred when present). +# such; cosign still preferred when present), and a capable gh that cannot +# complete the check (not logged in / credentials rejected / GitHub API +# unavailable, each refused under its own name, never as a failed +# verification, while every refusal - including one whose stderr claims a +# network or auth problem - still reads "verification FAILED"), with gh's +# words stripped of control bytes and imported pipefail tolerated. set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -53,6 +62,8 @@ ATTEST_DB="$tmp/attestations" CURL_LOG="$tmp/curl.log" GH_LOG="$tmp/gh.log" GH_VERIFY_LOG="$tmp/gh-verify.log" +GH_API_LOG="$tmp/gh-api.log" +ITMP="$tmp/itmp" mkdir -p "$WWW" "$SHIM" "$TOOLS" "$tmp/home" : >"$ATTEST_DB" @@ -144,15 +155,33 @@ exit 0 EOF cat >"$SHIM/gh" <), answered from the same DB: 200 when the digest is +# attested, 404 when it is not, as the real API does. # FAKE_GH_LACKS plays an older gh (feature history from gh's source): # "attestation" = gh < 2.47.0 (hidden in 2.47-2.48, public from 2.49.0), # "--signer-workflow" = gh < 2.51.0, "--source-ref" = gh < 2.68.0. -# FAKE_GH_VERSION is what --version reports. FAKE_GH_BROKEN=1 fails every -# command the way gh does with an unreadable config. +# FAKE_GH_VERSION is what --version reports ("fail": --version fails). +# FAKE_GH_BROKEN=1 fails every command the way gh does with an unreadable +# config. A capable gh that cannot finish the check, modelled on real gh +# 2.68.0/2.102.0 runs (exit codes, --include output; help and --version +# are exempt, like gh's root auth check): +# FAKE_GH_AUTH=none no credentials: gh's login hint on stderr, exit 4 +# FAKE_GH_AUTH=bad token rejected: HTTP 401 +# FAKE_GH_NET=down no HTTP response (proxy/DNS/connect failure), exit 1 +# FAKE_GH_NET=503|429|403|403rl|403ra that HTTP status; 403rl carries +# X-Ratelimit-Remaining: 0, 403ra a Retry-After header +# FAKE_GH_COLOR=1 header names coloured, as with CLICOLOR_FORCE +# FAKE_GH_ERR_CTRL=1 terminal control bytes in gh's error lines +# FAKE_GH_VERIFY_RC=4 verify exits 4 even though the API probe answered +# FAKE_GH_VERIFY_ERR text a failing verify prints instead of its error +# FAKE_GH_ATTEST_SIGNER workflow the stored attestations were signed by echo "\$*" >>"$GH_LOG" +ctl="" cr="" +if [[ -n "\${FAKE_GH_ERR_CTRL:-}" ]]; then ctl=\$'\e]0;PWNED\a\e[31m' cr=\$'\r'; fi if [[ -n "\${FAKE_GH_BROKEN:-}" ]]; then - echo 'failed to create root command: failed to read configuration: invalid config file /home/u/.config/gh/config.yml: invalid yaml' >&2 + echo "\${ctl}failed to create root command: failed to read configuration: invalid config file /home/u/.config/gh/config.yml: invalid yaml\$cr" >&2 exit 1 fi lacks=" \${FAKE_GH_LACKS:-} " @@ -161,9 +190,66 @@ if [[ "\${1:-}" == "--help" ]]; then exit 0 fi if [[ "\${1:-}" == "--version" ]]; then + [[ "\${FAKE_GH_VERSION:-}" != fail ]] || exit 1 printf 'gh version %s (2026-01-01)\nhttps://github.com/cli/cli/releases/latest\n' "\${FAKE_GH_VERSION:-2.102.0}" exit 0 fi +login_hint() { + printf 'To get started with GitHub CLI, please run: gh auth login\nAlternatively, populate the GH_TOKEN environment variable with a GitHub API authentication token.\n' >&2 + exit 4 +} +# hdrs [header...]: what gh api --include prints before the body. +hdrs() { + local h c="" r="" + if [[ -n "\${FAKE_GH_COLOR:-}" ]]; then c=\$'\e[1;34m' r=\$'\e[m'; fi + printf '%s\n' "\$1"; shift + for h in "Content-Type: application/json; charset=utf-8" "\$@"; do + printf '%s%s%s:%s\r\n' "\$c" "\${h%%:*}" "\$r" "\${h#*:}" + done + printf '\r\n' +} +if [[ "\${1:-}" == api ]]; then + [[ \$# -eq 5 && "\$2 \$3 \$4" == "--method GET --include" ]] || { echo "gh shim: unexpected api call: \$*" >&2; exit 2; } + path="\$5" st="" + if [[ "\${FAKE_GH_AUTH:-}" == none ]]; then echo "\$path noauth" >>"$GH_API_LOG"; login_hint; fi + if [[ "\${FAKE_GH_AUTH:-}" == bad ]]; then st=401 + else + case "\${FAKE_GH_NET:-}" in + down) + echo "\$path -" >>"$GH_API_LOG" + echo "\${ctl}Get \"https://api.github.com/\$path\": proxyconnect tcp: dial tcp 127.0.0.1:9: connect: connection refused\$cr" >&2 + exit 1;; + 503|429|403|403rl|403ra) st="\$FAKE_GH_NET";; + "") + d="\${path#repos/$REPO/attestations/sha256:}"; d="\${d%%\?*}" + if [[ "\$path" == "repos/$REPO/attestations/sha256:\$d?per_page=1" && "\$d" =~ ^[0-9a-f]{64}\$ ]] && + grep -q "^\$d " "$ATTEST_DB"; then st=200; else st=404; fi;; + *) echo "gh shim: unknown FAKE_GH_NET" >&2; exit 2;; + esac + fi + echo "\$path \$st" >>"$GH_API_LOG" + case "\$st" in + 200) hdrs "HTTP/2.0 200 OK" "X-Ratelimit-Remaining: 4999"; echo '{"attestations":[{"bundle":{}}]}'; exit 0;; + 404) hdrs "HTTP/2.0 404 Not Found" "X-Ratelimit-Remaining: 4998" + echo '{"message":"Not Found","status":"404"}'; echo "gh: Not Found (HTTP 404)" >&2;; + 401) hdrs "HTTP/2.0 401 Unauthorized" + echo '{"message":"Bad credentials","status":"401"}'; echo "gh: Bad credentials (HTTP 401)" >&2;; + 503) hdrs "HTTP/2.0 503 Service Unavailable"; echo 'unicorn'; echo "\${ctl}gh: HTTP 503\$cr" >&2;; + 429) hdrs "HTTP/2.0 429 Too Many Requests" "Retry-After: 60" + echo '{"message":"You have exceeded a secondary rate limit."}' + echo "gh: You have exceeded a secondary rate limit. (HTTP 429)" >&2;; + 403rl) hdrs "HTTP/2.0 403 Forbidden" "X-Ratelimit-Remaining: 0" "X-Ratelimit-Reset: 1760000000" + echo '{"message":"API rate limit exceeded for user ID 1."}' + echo "gh: API rate limit exceeded for user ID 1. (HTTP 403)" >&2;; + 403ra) hdrs "HTTP/2.0 403 Forbidden" "Retry-After: 60" "X-Ratelimit-Remaining: 4000" + echo '{"message":"You have exceeded a secondary rate limit."}' + echo "gh: You have exceeded a secondary rate limit. (HTTP 403)" >&2;; + 403) hdrs "HTTP/2.0 403 Forbidden" "X-Ratelimit-Remaining: 4000" + echo '{"message":"Resource not accessible by personal access token"}' + echo "gh: Resource not accessible by personal access token (HTTP 403)" >&2;; + esac + exit 1 +fi if [[ "\${1:-}" == attestation && "\$lacks" == *" attestation "* ]]; then printf 'unknown command "attestation" for "gh"\n\nUsage: gh [flags]\n' >&2 exit 1 @@ -198,11 +284,27 @@ while [[ \$# -gt 0 ]]; do esac done printf 'verify %s repo=%s signer=%s ref=%s\n' "\$file" "\$repo" "\$signer" "\$ref" >>"$GH_VERIFY_LOG" -[[ "\$repo" == "$REPO" ]] || exit 1 -[[ "\$signer" == "$REPO/.github/workflows/release-artifacts.yml" ]] || exit 1 +# refuse : how gh reports a failed verify when stdout is not a TTY. +refuse() { + if [[ -n "\${FAKE_GH_VERIFY_ERR:-}" ]]; then printf '%s\n' "\$FAKE_GH_VERIFY_ERR" >&2 + else printf '\nError: %s\n' "\$1" >&2; fi + exit 1 +} +[[ "\${FAKE_GH_AUTH:-}" != none && "\${FAKE_GH_VERIFY_RC:-}" != 4 ]] || login_hint +url="https://api.github.com/repos/\$repo/attestations/sha256:...?per_page=30" +[[ "\${FAKE_GH_AUTH:-}" != bad ]] || refuse "failed to fetch attestations from \$repo: HTTP 401: Bad credentials (\$url)" +case "\${FAKE_GH_NET:-}" in + "") ;; + down) refuse "failed to fetch attestations from \$repo: Get \"\$url\": proxyconnect tcp: dial tcp 127.0.0.1:9: connect: connection refused";; + *) refuse "failed to fetch attestations from \$repo: HTTP \${FAKE_GH_NET:0:3} (\$url)";; +esac +[[ "\$repo" == "$REPO" ]] || refuse "no attestations found" +[[ "\$signer" == "$REPO/.github/workflows/release-artifacts.yml" && + "\$signer" == "\${FAKE_GH_ATTEST_SIGNER:-$REPO/.github/workflows/release-artifacts.yml}" ]] || + refuse "expected SAN to match signer workflow \$signer" sum="\$("$REAL_SHA" <"\$file" | cut -d' ' -f1)" -if [[ -n "\$ref" ]]; then grep -qx "\$sum \$ref" "$ATTEST_DB" -else grep -q "^\$sum " "$ATTEST_DB"; fi +if [[ -n "\$ref" ]]; then grep -qx "\$sum \$ref" "$ATTEST_DB" || refuse "expected SourceRepositoryRef to be \$ref" +else grep -q "^\$sum " "$ATTEST_DB" || refuse "no attestations found"; fi EOF cat >"$SHIM/uname" <<'EOF' #!/bin/sh @@ -254,14 +356,18 @@ for _ in $(seq 100); do done [[ -n "$FAKE_PORT" ]] || { echo "[install-test] fake server did not start" >&2; exit 1; } -# run [VAR=value...] : run install.sh from a file. +# run [VAR=value...] : run install.sh from a file, with +# "${RUN_SH[@]}" (default: sh) and TMPDIR pointing at an empty $ITMP, so a +# case can check that the installer removed its temp dir. OUT="$tmp/out" +RUN_SH=(sh) run() { local scen="$1" path="$2"; shift 2 - : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; rm -f "$tmp/sudo.log" + : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; : >"$GH_API_LOG"; rm -f "$tmp/sudo.log" + rm -rf "$ITMP"; mkdir -p "$ITMP" set +e - env -i HOME="$tmp/home" PATH="$path" FAKE_PORT="$FAKE_PORT" SCENARIO="$scen" "$@" \ - sh "$INSTALL" >"$OUT" 2>&1 + env -i HOME="$tmp/home" TMPDIR="$ITMP" PATH="$path" FAKE_PORT="$FAKE_PORT" SCENARIO="$scen" "$@" \ + "${RUN_SH[@]}" "$INSTALL" >"$OUT" 2>&1 rc=$? set -e } @@ -288,7 +394,8 @@ CLI_AMD64=bpfcompat-linux-amd64 VAL_AMD64=bpfcompat-validator-static-linux-amd64 # ---------------------------------------------------------------- fixtures -for s in stable arm64 nosha noverify mismatch bindcosign bindgh bindghcli bindghval trunc dirs badver badlatest ghcap; do +for s in stable arm64 nosha noverify mismatch bindcosign bindgh bindghcli bindghval trunc dirs badver badlatest ghcap \ + ghapi ghrefuse ghnoatt; do release "$s" v0.3.7 amd64 release "$s" v0.4.0-rc.4 amd64 latest "$s" v0.3.7 @@ -515,6 +622,7 @@ gh_lacking() { ! grep -q 'requires cosign or GitHub CLI' "$OUT" || bad="$bad says-gh-missing" ! grep -q 'FAILED' "$OUT" || bad="$bad says-verification-failed" ! grep -qE '^attestation verify [^-]' "$GH_LOG" || bad="$bad verify-attempted" + ! grep -q '^api ' "$GH_LOG" || bad="$bad api-probed" if [[ -z "$bad" ]]; then ok "gh $ver lacking '$lacks': refused, naming what it lacks" else fail "gh $ver lacking '$lacks' (rc=$rc):$bad"; show; fi } @@ -532,6 +640,7 @@ grep -qF "installed gh could not run 'gh --help' (failed to create root command: grep -qF 'fix gh or install cosign - refusing checksum-only installation' "$OUT" || bad="$bad no-remedy" ! grep -qE 'lacks|FAILED|requires cosign or GitHub CLI' "$OUT" || bad="$bad wrong-reason" ! grep -qE '^attestation verify [^-]' "$GH_LOG" || bad="$bad verify-attempted" +! grep -q '^api ' "$GH_LOG" || bad="$bad api-probed" if [[ -z "$bad" ]]; then ok "gh that cannot run: refused, naming gh's own error" else fail "gh that cannot run (rc=$rc):$bad"; show; fi for ghenv in FAKE_GH_LACKS=attestation FAKE_GH_LACKS=--source-ref FAKE_GH_BROKEN=1; do @@ -544,6 +653,138 @@ for ghenv in FAKE_GH_LACKS=attestation FAKE_GH_LACKS=--source-ref FAKE_GH_BROKEN else fail "gh with $ghenv and cosign present (rc=$rc)"; show; fi done +# 10. S2: a capable gh that cannot complete the check is told apart from a +# refused verification, and every outcome refuses. Before the first +# verify call, one read-only probe of the attestation API the verify +# uses decides "not authenticated" (gh's auth exit code 4, or HTTP 401) +# and "GitHub API unavailable" (no HTTP response, 5xx, 429, rate-limited +# 403); any other answer hands over to the verify calls, whose failures +# (other than exit code 4) always read "verification FAILED", whatever +# gh prints. Each case: nonzero exit, nothing installed, the installer's +# temp dir gone, only its own class named, no terminal control bytes. +C2='GitHub authentication needed for attestation verification' +C3='GitHub API unavailable for attestation verification' +C4='GitHub attestation verification FAILED for' +# ghrefuse: the v0.3.7 CLI is rc.4's attested binary (attested, other tag). +# ghnoatt: the v0.3.7 CLI has no attestation at all. SHA256SUMS matches both. +cp "$(rel_dir ghrefuse v0.4.0-rc.4)/$CLI_AMD64" "$(rel_dir ghrefuse v0.3.7)/$CLI_AMD64" +printf '#!/bin/sh\necho "unattested build"\n' >"$(rel_dir ghnoatt v0.3.7)/$CLI_AMD64" +for s in ghrefuse ghnoatt; do (cd "$(rel_dir "$s" v0.3.7)" && "$REAL_SHA" bpfcompat-* >SHA256SUMS); done +# probe_path : the probe the installer must make for v0.3.7's CLI. +probe_path() { + echo "repos/$REPO/attestations/sha256:$("$REAL_SHA" <"$(rel_dir "$1" v0.3.7)/$CLI_AMD64" | cut -d' ' -f1)?per_page=1" +} +itmp_clean() { [[ -d "$ITMP" && -z "$(ls -A "$ITMP")" ]]; } +no_ctl() { [[ -z "$(LC_ALL=C tr -d '\n[:print:]' <"$OUT")" ]]; } +# gh_class <scenario> <class 2|3|4> <text> <verify calls> <probe status> [VAR=value...] +# verify calls: how many `gh attestation verify` calls may have been made; +# probe status: what the one probe got ("-" = no HTTP response). +gh_class() { + local title="$1" scen="$2" cls="$3" want="$4" nver="$5" pst="$6" d="$tmp/i-ghclass" bad="" n + shift 6 + run "$scen" "$GH_ONLY" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" "$@" + [[ $rc -ne 0 && ! -e "$d" ]] || bad+=" installed-or-rc0" + itmp_clean || bad+=" tmpdir-left" + grep -qF -- "$want" "$OUT" || bad+=" reason-not-named" + case "$cls" in + 2) grep -qF 'or install cosign - refusing checksum-only installation' "$OUT" || bad+=" no-remedy" + ! grep -qE "$C3|FAILED|lacks|requires cosign or GitHub CLI" "$OUT" || bad+=" other-class-named";; + 3) grep -qF 'then retry, or install cosign - refusing checksum-only installation' "$OUT" || bad+=" no-remedy" + ! grep -qE "$C2|FAILED|lacks|requires cosign or GitHub CLI" "$OUT" || bad+=" other-class-named";; + 4) ! grep -qiE 'authenticat|logged in|credential|GH_TOKEN|network|proxy|unavailable|unreachable|rate.limit|refused|retry|HTTP [0-9]|lacks' "$OUT" || + bad+=" auth-or-network-wording" + ! grep -qF -- "$C2" "$OUT" && ! grep -qF -- "$C3" "$OUT" || bad+=" other-class-named";; + esac + no_ctl || bad+=" control-bytes" + [[ "$(cat "$GH_API_LOG")" == "$(probe_path "$scen") $pst" ]] || bad+=" probe($(tr '\n' '|' <"$GH_API_LOG"))" + n="$(grep -c . "$GH_VERIFY_LOG" || true)" + [[ "$n" == "$nver" ]] || bad+=" verify-calls=$n" + if [[ -z "$bad" ]]; then ok "gh $title: class $cls, refused" + else fail "gh $title (rc=$rc):$bad"; show; fi +} +gh_class "not logged in" ghapi 2 \ + "$C2: gh is not logged in (To get started with GitHub CLI, please run: gh auth login); run 'gh auth login' or set a valid GH_TOKEN" \ + 0 noauth FAKE_GH_AUTH=none +gh_class "token rejected (HTTP 401)" ghapi 2 \ + "$C2: GitHub rejected gh's credentials (HTTP 401) (gh: Bad credentials (HTTP 401)); run 'gh auth login' or set a valid GH_TOKEN" \ + 0 401 FAKE_GH_AUTH=bad +gh_class "API unreachable" ghapi 3 \ + "$C3: no HTTP response (Get \"https://api.github.com/$(probe_path ghapi)\": proxyconnect tcp: dial tcp 127.0.0.1:9: connect: connection refused); check the network" \ + 0 - FAKE_GH_NET=down +gh_class "API HTTP 503" ghapi 3 "$C3: HTTP 503 (gh: HTTP 503)" 0 503 FAKE_GH_NET=503 +gh_class "API HTTP 429" ghapi 3 "$C3: HTTP 429, rate limited (gh: You have exceeded a secondary rate limit. (HTTP 429))" 0 429 FAKE_GH_NET=429 +gh_class "API rate limit (403, X-Ratelimit-Remaining: 0, coloured headers)" ghapi 3 \ + "$C3: HTTP 403, rate limited (gh: API rate limit exceeded for user ID 1. (HTTP 403))" 0 403rl FAKE_GH_NET=403rl FAKE_GH_COLOR=1 +gh_class "API secondary rate limit (403, Retry-After)" ghapi 3 "$C3: HTTP 403, rate limited" 0 403ra FAKE_GH_NET=403ra +gh_class "API 403 without rate-limit headers" ghapi 4 "$C4 $CLI_AMD64" 1 403 FAKE_GH_NET=403 +gh_class "no attestation for the CLI digest (API 404)" ghnoatt 4 "$C4 $CLI_AMD64" 1 404 +gh_class "CLI attested for another tag (API answers)" ghrefuse 4 "$C4 $CLI_AMD64" 1 200 +gh_class "attestation signed by another workflow (API answers)" ghapi 4 "$C4 $CLI_AMD64" 1 200 \ + FAKE_GH_ATTEST_SIGNER="$REPO/.github/workflows/ci.yml" +gh_class "refusal whose stderr claims a network error" ghrefuse 4 "$C4 $CLI_AMD64" 1 200 \ + FAKE_GH_VERIFY_ERR=$'error connecting to api.github.com\ncheck your internet connection or https://githubstatus.com' +gh_class "refusal whose stderr claims missing auth" ghrefuse 4 "$C4 $CLI_AMD64" 1 200 \ + FAKE_GH_VERIFY_ERR=$'To get started with GitHub CLI, please run: gh auth login\nHTTP 401: Bad credentials' +gh_class "verify exits 4 after the API answered" ghapi 2 \ + "$C2: gh is not logged in (To get started with GitHub CLI, please run: gh auth login)" 1 200 FAKE_GH_VERIFY_RC=4 +gh_class "API unreachable, control bytes in gh's error" ghapi 3 "$C3: no HTTP response (]0;PWNED[31mGet \"https://api.github.com/" \ + 0 - FAKE_GH_NET=down FAKE_GH_ERR_CTRL=1 +# R2-N2: the "could not run 'gh --help'" line is stripped of control bytes too. +d="$tmp/i-ghbroken-ctl" bad="" +run ghcap "$GH_ONLY" FAKE_GH_BROKEN=1 FAKE_GH_ERR_CTRL=1 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" +[[ $rc -ne 0 && ! -e "$d" ]] && itmp_clean || bad+=" installed-or-rc0-or-tmpdir" +grep -qF "installed gh could not run 'gh --help' (]0;PWNED[31mfailed to create root command: failed to read configuration: invalid config file /home/u/.config/gh/config.yml: invalid yaml); fix gh" "$OUT" || + bad+=" reason-not-named" +no_ctl || bad+=" control-bytes" +if [[ -z "$bad" ]]; then ok "gh that cannot run, control bytes in its error: stripped, refused" +else fail "gh that cannot run with control bytes (rc=$rc):$bad"; show; fi + +# 10b. The probe on a good release: exactly one read-only GET of the CLI's +# attestations, before the first verify call; with cosign present, gh +# is never run. The cosign path is the same for every gh mode above. +d="$tmp/i-ghprobe" bad="" +run ghapi "$GH_ONLY" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" +[[ $rc -eq 0 && -x "$d/bin/bpfcompat" ]] && itmp_clean || bad+=" not-installed-or-tmpdir" +verified "$GH_ONLY" v0.3.7 "$CLI_AMD64" "$VAL_AMD64" || bad+=" verify-calls" +[[ "$(cat "$GH_API_LOG")" == "$(probe_path ghapi) 200" ]] || bad+=" probe" +[[ "$(grep -nE '^(api |attestation verify [^-])' "$GH_LOG" | head -1)" == *":api --method GET --include $(probe_path ghapi)" ]] || + bad+=" probe-not-first" +if [[ -z "$bad" ]]; then ok "gh path: one read-only API probe of the CLI digest, before the verify calls" +else fail "gh path probe (rc=$rc):$bad"; show; sed 's/^/ gh | /' "$GH_LOG" >&2; fi +for ghenv in FAKE_GH_AUTH=none FAKE_GH_AUTH=bad FAKE_GH_NET=down FAKE_GH_NET=503 FAKE_GH_NET=429 FAKE_GH_NET=403rl \ + FAKE_GH_NET=403 FAKE_GH_VERIFY_RC=4 FAKE_GH_VERIFY_ERR=x FAKE_GH_ERR_CTRL=1; do + d="$tmp/i-ghclass-cosign" + run ghapi "$WITH_COSIGN" "$ghenv" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" + if [[ $rc -eq 0 && -x "$d/bin/bpfcompat" && -x "$d/libexec/bpfcompat-validator" && ! -s "$GH_LOG" ]] && itmp_clean && + grep -q 'cosign signature over SHA256SUMS verified' "$OUT"; then + ok "cosign path with $ghenv: gh never run, cosign verified, installed" + else fail "cosign path with $ghenv (rc=$rc)"; show; fi + rm -rf "$d" +done + +# 10c. R2-N1: with pipefail imported (bash as sh, SHELLOPTS exported) every +# refusal still prints its reason, and a good release still installs. +RUN_SH=(bash --posix) +PF=SHELLOPTS=braceexpand:hashall:interactive-comments:pipefail +for c in "FAKE_GH_BROKEN=1|installed gh could not run 'gh --help' (failed to create root command" \ + "FAKE_GH_VERSION=fail FAKE_GH_LACKS=attestation|installed gh lacks the 'gh attestation' command" \ + "FAKE_GH_NET=down|$C3: no HTTP response (Get " "FAKE_GH_NET=403rl|$C3: HTTP 403, rate limited" \ + "FAKE_GH_NET=403|$C4 $CLI_AMD64" "FAKE_GH_AUTH=none|$C2: gh is not logged in (To get started" \ + "FAKE_GH_AUTH=bad|$C2: GitHub rejected gh's credentials (HTTP 401) (gh: Bad credentials"; do + d="$tmp/i-pipefail" + read -r -a envs <<<"${c%%|*}" + run ghapi "$GH_ONLY" "$PF" "${envs[@]}" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" + if [[ $rc -ne 0 && ! -e "$d" ]] && itmp_clean && grep -qF -- "${c#*|}" "$OUT"; then + ok "pipefail imported, ${c%%|*}: refused with its reason" + else fail "pipefail imported, ${c%%|*} (rc=$rc)"; show; fi +done +d="$tmp/i-pipefail-ok" +run ghapi "$GH_ONLY" "$PF" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" +if [[ $rc -eq 0 && -x "$d/bin/bpfcompat" ]] && verified "$GH_ONLY" v0.3.7 "$CLI_AMD64" "$VAL_AMD64" && + [[ "$(cat "$GH_API_LOG")" == "$(probe_path ghapi) 200" ]]; then ok "pipefail imported: gh path installs a good release" +else fail "pipefail imported, good release (rc=$rc)"; show; fi +RUN_SH=(sh) + if [[ $failures -ne 0 ]]; then echo "[install-test] $failures failure(s)" >&2 exit 1 From 17adeab9ebe8957bd534bd32ac021b6dbdd8267c Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 09:27:48 +0300 Subject: [PATCH 30/38] security(release): bind release provenance to the exact tag and candidate A tag's own copies of these checks are what run for that release (the release build, its promotion, and the Action pinned to it), so they must be in the commit that is tagged. - promote-release: compare SHA256SUMS.sig, SHA256SUMS.crt and the SBOM json/sig/crt in the draft with the candidate run's copies, and verify the cosign signatures (both blobs and the image) with the exact release-artifacts.yml@refs/tags/<tag> identity instead of any v* tag. The identity lines are the ones 4a51550 ships on the v0.3.8 line (v0.3.8-rc.1, v0.3.8-rc.2). The evidence attestation and verify-release-assets.sh are bound to the tag ref and expected commit. - verify-release-assets.sh: require BPFCOMPAT_EXPECTED_SOURCE_SHA and BPFCOMPAT_EXPECTED_SOURCE_REF and pass them as --source-digest and --source-ref (as in 4a51550). The tag build's two gates pass the run's commit and ref; maintenance-evidence already did. - action.yml: before verifying prebuilt assets, check that gh offers --signer-workflow, --source-ref and --source-digest, and resolve the pinned commit (the SHA pin itself, or git ls-remote of the tag, peeled). If either is unavailable, take the existing warned source build; never an unbound prebuilt install and never a job failure. - release-artifacts stable gate: require a completed rollback drill of the release's own candidate (drill alias and image digest), next to the incident term it already checks. - tests: new release-workflow-gates_test.sh (stable gate cases and the promotion steps' run bodies against stub cosign/gh/docker, plus caller wiring), bound-identity cases in verify-release-assets_test.sh, and tag/SHA-pin, ls-remote failure, gh capability and real-verifier cases in the Action tests. Signed-off-by: ErenAri <erenari27@gmail.com> --- .github/workflows/ci.yml | 7 + .github/workflows/promote-release.yml | 21 +- .github/workflows/release-artifacts.yml | 7 + action.yml | 43 +- scripts/action-license-bundle_test.sh | 25 +- scripts/action-validator-pin_test.sh | 200 +++++++++- scripts/release-workflow-gates_test.sh | 497 ++++++++++++++++++++++++ scripts/verify-release-assets.sh | 13 +- scripts/verify-release-assets_test.sh | 83 +++- 9 files changed, 872 insertions(+), 24 deletions(-) create mode 100755 scripts/release-workflow-gates_test.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3d64282..db7cfd9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -156,6 +156,13 @@ jobs: run: bash scripts/check-production-code-scanning_test.sh - name: Production-readiness evidence regression run: bash scripts/production-readiness-report_test.sh + - name: Release-workflow gate regression + # Run by a tag's own copy, so they cannot be fixed for a release after + # it is tagged: the stable gate requires a completed rollback drill of + # the release's own candidate; promotion requires the candidate run's + # signature files and SBOM, this tag's exact signer identity, and + # tag- and commit-bound attestations. + run: bash scripts/release-workflow-gates_test.sh - name: Campaign-to-candidate binding regression # A scheduled campaign must test the designated immutable release # candidate (tag, commit, signed assets, attestations), not whatever diff --git a/.github/workflows/promote-release.yml b/.github/workflows/promote-release.yml index d2a0f36..a348d53 100644 --- a/.github/workflows/promote-release.yml +++ b/.github/workflows/promote-release.yml @@ -151,6 +151,12 @@ jobs: cmp candidate/run-assets/SHA256SUMS candidate/draft-assets/SHA256SUMS cmp "$evidence" candidate/draft-assets/release-candidate-evidence.json + # The SBOM and the signature files are not listed in SHA256SUMS: + # the draft must carry the exact candidate run's copies. + for signed in SHA256SUMS.sig SHA256SUMS.crt \ + bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt; do + cmp "candidate/run-assets/${signed}" "candidate/draft-assets/${signed}" + done assets=( bpfcompat-linux-amd64 @@ -170,23 +176,28 @@ jobs: if [[ "$channel" == "stable" ]]; then assets+=(production-readiness.json production-readiness.md) fi + # Attestations and signatures must come from this tag's release + # run at the expected commit, not from any v* tag's run. + export BPFCOMPAT_EXPECTED_SOURCE_SHA="$EXPECTED_COMMIT" + export BPFCOMPAT_EXPECTED_SOURCE_REF="refs/tags/$RELEASE_TAG" bash scripts/verify-release-assets.sh \ candidate/draft-assets "$GITHUB_REPOSITORY" "${assets[@]}" gh attestation verify "$evidence" \ --repo "$GITHUB_REPOSITORY" \ - --signer-workflow "${GITHUB_REPOSITORY}/.github/workflows/release-artifacts.yml" + --signer-workflow "${GITHUB_REPOSITORY}/.github/workflows/release-artifacts.yml" \ + --source-digest "$EXPECTED_COMMIT" --source-ref "refs/tags/$RELEASE_TAG" - identity_regexp="^https://github.com/${GITHUB_REPOSITORY}/.github/workflows/release-artifacts.yml@refs/tags/v" + identity="https://github.com/${GITHUB_REPOSITORY}/.github/workflows/release-artifacts.yml@refs/tags/${RELEASE_TAG}" cosign verify-blob \ --certificate candidate/draft-assets/SHA256SUMS.crt \ --signature candidate/draft-assets/SHA256SUMS.sig \ - --certificate-identity-regexp "$identity_regexp" \ + --certificate-identity "$identity" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ candidate/draft-assets/SHA256SUMS cosign verify-blob \ --certificate candidate/draft-assets/bpfcompat.sbom.cdx.crt \ --signature candidate/draft-assets/bpfcompat.sbom.cdx.sig \ - --certificate-identity-regexp "$identity_regexp" \ + --certificate-identity "$identity" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ candidate/draft-assets/bpfcompat.sbom.cdx.json @@ -219,7 +230,7 @@ jobs: version --json | jq -r .version)" test "$actual_version" = "$expected_version" cosign verify \ - --certificate-identity-regexp "^https://github.com/${GITHUB_REPOSITORY}/.github/workflows/release-artifacts.yml@refs/tags/v" \ + --certificate-identity "https://github.com/${GITHUB_REPOSITORY}/.github/workflows/release-artifacts.yml@refs/tags/${RELEASE_TAG}" \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ "${IMAGE}@${EXPECTED_DIGEST}" diff --git a/.github/workflows/release-artifacts.yml b/.github/workflows/release-artifacts.yml index 35638f9..ab9aec0 100644 --- a/.github/workflows/release-artifacts.yml +++ b/.github/workflows/release-artifacts.yml @@ -201,6 +201,9 @@ jobs: .slo.campaign_count == 4 and .slo.infrastructure_errors == 0 and (.canaries | length == 3) and + .rollback.completed == true and + .rollback.alias == (($c.image | sub("@[^@]*$"; "")) + ":rollback-drill-" + $c.tag) and + .rollback.candidate_digest == ($c.image | sub("^.*@"; "")) and .incident.completed == true and .operator.approval_mode == "solo-maintainer" and .operator.confirmed == true @@ -329,6 +332,8 @@ jobs: - name: Verify candidate attestations env: GH_TOKEN: ${{ github.token }} + BPFCOMPAT_EXPECTED_SOURCE_SHA: ${{ github.sha }} + BPFCOMPAT_EXPECTED_SOURCE_REF: ${{ github.ref }} run: > bash scripts/verify-release-assets.sh dist Kernel-Guard/bpfcompat @@ -427,6 +432,8 @@ jobs: - name: Verify ARM64 candidate attestation env: GH_TOKEN: ${{ github.token }} + BPFCOMPAT_EXPECTED_SOURCE_SHA: ${{ github.sha }} + BPFCOMPAT_EXPECTED_SOURCE_REF: ${{ github.ref }} run: > bash scripts/verify-release-assets.sh dist Kernel-Guard/bpfcompat diff --git a/action.yml b/action.yml index a12b288..8528203 100644 --- a/action.yml +++ b/action.yml @@ -216,10 +216,47 @@ runs: echo "::warning::GitHub CLI attestation verification is unavailable; building from the pinned Action source instead of using unverifiable prebuilt binaries. This run does not use the attested ${ref} validator." exit 0 fi + # Verification is bound to this exact release (tag ref and commit), + # which needs --signer-workflow (gh 2.51.0) and --source-ref and + # --source-digest (gh 2.68.0). Ask gh's own help, as scripts/install.sh + # does, rather than its version. + gh_help="$(gh attestation verify --help 2>&1 || true)" + gh_lacks="" + for flag in --signer-workflow --source-ref --source-digest; do + grep -Eq -- "(^|[^[:alnum:]-])${flag}([^[:alnum:]-]|\$)" <<<"$gh_help" || + gh_lacks="${gh_lacks:+$gh_lacks }$flag" + done + if [[ -n "$gh_lacks" ]]; then + echo "::warning::GitHub CLI cannot bind attestations to the release ('gh attestation verify' lacks ${gh_lacks}; needs gh >= 2.68.0); building from the pinned Action source instead of using unverifiable prebuilt binaries. This run does not use the attested ${ref} validator." + exit 0 + fi + # The commit the release was built from: a SHA pin names it; a tag + # pin is resolved to the commit the tag points at (peeled when + # annotated). If that fails, build from source rather than install + # binaries that cannot be bound to this release. + source_sha="${ACTION_REF:-}" + if [[ ! "$source_sha" =~ ^[0-9a-f]{40}$ ]]; then + source_sha="" + if tag_refs="$(GIT_TERMINAL_PROMPT=0 git ls-remote https://github.com/Kernel-Guard/bpfcompat.git \ + "refs/tags/${ref}" "refs/tags/${ref}^{}" 2>/dev/null)"; then + source_sha="$(awk -v tag="refs/tags/${ref}" ' + $2 == tag && !peeled { sha = $1 } + $2 == (tag "^{}") { sha = $1; peeled = 1 } + END { print sha } + ' <<<"$tag_refs")" + fi + fi + if [[ ! "$source_sha" =~ ^[0-9a-f]{40}$ ]]; then + echo "::warning::Could not resolve release ${ref} to its commit; building from the pinned Action source instead of using prebuilt binaries that cannot be bound to the release. This run does not use the attested ${ref} binaries or validator." + exit 0 + fi # A failed verification is a hard error, not a fallback: assets exist - # but do not match what the release workflow published or attested. - GH_TOKEN="$GITHUB_API_TOKEN" bash \ - "$GITHUB_ACTION_PATH/scripts/verify-release-assets.sh" \ + # but do not match what this release's workflow run published or + # attested. + GH_TOKEN="$GITHUB_API_TOKEN" \ + BPFCOMPAT_EXPECTED_SOURCE_SHA="$source_sha" \ + BPFCOMPAT_EXPECTED_SOURCE_REF="refs/tags/${ref}" \ + bash "$GITHUB_ACTION_PATH/scripts/verify-release-assets.sh" \ "$tmp" Kernel-Guard/bpfcompat "${verify_assets[@]}" mkdir -p "$GITHUB_ACTION_PATH/bin" "$GITHUB_ACTION_PATH/validator/c-libbpf/bin" diff --git a/scripts/action-license-bundle_test.sh b/scripts/action-license-bundle_test.sh index 07c3055..3d892dc 100755 --- a/scripts/action-license-bundle_test.sh +++ b/scripts/action-license-bundle_test.sh @@ -2,8 +2,10 @@ # Regression for the composite action's third-party license bundle delivery. # # Renders the real action.yml "Download prebuilt binaries" step the way the -# Actions runner does and runs it with bash against stubs (curl, gh, uname, -# verify-release-assets.sh). Hermetic: no network, no real release assets. +# Actions runner does and runs it with bash against stubs (curl, gh, git, +# uname, verify-release-assets.sh). Hermetic: no network, no real release +# assets. The gh and git stubs only let the step bind verification to the +# pinned release (scripts/action-validator-pin_test.sh covers that binding). # # Contract checked: # - A release whose SHA256SUMS lists bpfcompat-third-party-licenses.tar.gz: @@ -26,6 +28,7 @@ import hashlib import io import os import re +import shutil import subprocess import sys import tarfile @@ -102,7 +105,23 @@ f='{assets}'/"${{url##*/}}" [[ -f "$f" ]] || exit 22 cp "$f" "$out" """) - write_exec(os.path.join(stubs, "gh"), "#!/usr/bin/env bash\nexit 0\n") + # A gh that can bind attestations to a source ref and commit (>= 2.68.0). + write_exec(os.path.join(stubs, "gh"), """#!/usr/bin/env bash +if [[ "$1 ${2:-} ${3:-}" == "attestation verify --help" ]]; then + echo ' --signer-workflow string Enforce that the workflow that signed the attestation matches the provided value' + echo ' --source-digest string Enforce that the digest associated with the source repository matches the provided value' + echo ' --source-ref string Enforce that the git ref associated with the source repository matches the provided value' +fi +exit 0 +""") + # git ls-remote resolves the pinned tag v9.9.9 to its commit. + write_exec(os.path.join(stubs, "git"), f"""#!/usr/bin/env bash +if [[ "${{1:-}}" == ls-remote ]]; then + printf '%s\\trefs/tags/v9.9.9\\n%s\\trefs/tags/v9.9.9^{{}}\\n' {"d" * 40} {"a" * 40} + exit 0 +fi +exec '{shutil.which("git") or "/usr/bin/git"}' "$@" +""") write_exec(os.path.join(stubs, "uname"), "#!/usr/bin/env bash\nif [[ \"$1\" == -m ]]; then echo x86_64; else /bin/uname \"$@\"; fi\n") outputs = os.path.join(d, "outputs") diff --git a/scripts/action-validator-pin_test.sh b/scripts/action-validator-pin_test.sh index ec579cd..2f671b7 100755 --- a/scripts/action-validator-pin_test.sh +++ b/scripts/action-validator-pin_test.sh @@ -4,8 +4,8 @@ # # It renders the real action.yml "Download prebuilt binaries" and # "Run bpfcompat" steps the way the Actions runner does and executes them with -# bash against stubs (curl, gh, uname, verify-release-assets.sh, bpfcompat). -# Hermetic: no network, no VM, no real release assets. +# bash against stubs (curl, gh, git, uname, verify-release-assets.sh, +# bpfcompat). Hermetic: no network, no VM, no real release assets. # # Contract checked: # - When the prebuilt release assets were downloaded and verified, the run @@ -20,6 +20,13 @@ # fallback is a ::warning:: annotation, not a plain log line. A ref that # is not a release keeps its documented, unannotated source build. # - Without resolved prebuilt binaries the run step does not invent a pin. +# - Prebuilt assets are verified for exactly the pinned release: its tag ref +# and the commit it was built from (a SHA pin is that commit; a tag pin is +# resolved with git ls-remote, peeled when annotated). Assets attested by +# another release's run are a hard error. When the commit cannot be +# resolved, or gh cannot bind attestations to a ref and commit (gh older +# than 2.68.0), the run takes the warned source build, never an unbound +# prebuilt install and never a job failure. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -31,6 +38,7 @@ python3 - "$ACTION" "$tmp" <<'PY' import hashlib import os import re +import shutil import subprocess import sys @@ -67,15 +75,56 @@ def write_exec(path, body): VALIDATOR_BYTES = b"release-validator-bytes\n" CLI_BYTES = b"#!/usr/bin/env bash\nexit 0\n" VALIDATOR_SHA = hashlib.sha256(VALIDATOR_BYTES).hexdigest() +CLI_SHA = hashlib.sha256(CLI_BYTES).hexdigest() TAG_COMMIT = "a" * 40 - - -def run_prebuilt(case, ref, curl_fail="", gh_ok=True, machine="x86_64", sums_sha=VALIDATOR_SHA): - """Execute the prebuilt step; return (rc, stdout+stderr, outputs dict, action_root).""" +TAG_OBJECT = "d" * 40 +OTHER_COMMIT = "e" * 40 +REAL_GIT = shutil.which("git") or "/usr/bin/git" +REAL_VERIFIER = os.path.join(os.path.dirname(os.path.abspath(action_path)), "scripts", "verify-release-assets.sh") +# git ls-remote of an annotated release tag: the tag object, then the commit. +ANNOTATED = f"{TAG_OBJECT}\trefs/tags/v9.9.9\n{TAG_COMMIT}\trefs/tags/v9.9.9^{{}}\n" +GH_FLAGS = { + "--signer-workflow": " --signer-workflow string Enforce that the workflow that signed the attestation matches the provided value ([host/]<owner>/<repo>/<path>/<to>/<workflow>)", + "--source-digest": " --source-digest string Enforce that the digest associated with the source repository matches the provided value", + "--source-ref": " --source-ref string Enforce that the git ref associated with the source repository matches the provided value", +} +# What the last run_prebuilt saw: the verifier's environment (None if it was +# not called), and every git invocation. +LAST = {} + + +def gh_help(lacks, extra=""): + """`gh attestation verify --help` FLAGS, after gh 2.102.0, minus `lacks`.""" + lines = ["Verify the integrity and provenance of an artifact using its associated", + "cryptographically signed attestations.", "", "FLAGS", + " -o, --owner string GitHub organization to scope attestation lookup by", + " -R, --repo string Repository name in the format <owner>/<repo>", + " --signer-digest string Enforce that the digest associated with the signer workflow matches the provided value"] + lines += [line for flag, line in GH_FLAGS.items() if flag not in lacks] + if extra: + lines.append(extra) + return "\n".join(lines) + "\n" + + +def run_prebuilt(case, ref, curl_fail="", gh_ok=True, machine="x86_64", sums_sha=VALIDATOR_SHA, + gh_lacks=(), gh_help_extra="", gh_help_fails=False, ls_remote=ANNOTATED, + verifier="stub", attested=("refs/tags/v9.9.9", TAG_COMMIT)): + """Execute the prebuilt step; return (rc, stdout+stderr, outputs dict, action_root). + + ls_remote is what `git ls-remote` prints (None: it fails like a network + error). verifier "stub" records the identity the step passes; "real" runs + this checkout's scripts/verify-release-assets.sh against a gh whose assets + carry the release-workflow attestation `attested` (source ref, commit).""" root = os.path.join(case, "action") os.makedirs(os.path.join(root, "scripts")) - write_exec(os.path.join(root, "scripts", "verify-release-assets.sh"), - "#!/usr/bin/env bash\nexit 0\n") + verifier_log = os.path.join(case, "verifier_env") + if verifier == "real": + shutil.copy(REAL_VERIFIER, os.path.join(root, "scripts", "verify-release-assets.sh")) + else: + write_exec(os.path.join(root, "scripts", "verify-release-assets.sh"), + "#!/usr/bin/env bash\n" + "printf 'SHA=%s\\nREF=%s\\nARGS=%s\\n' \"${BPFCOMPAT_EXPECTED_SOURCE_SHA-<unset>}\" " + f"\"${{BPFCOMPAT_EXPECTED_SOURCE_REF-<unset>}}\" \"$*\" >'{verifier_log}'\nexit 0\n") stubs = os.path.join(case, "stubs") os.makedirs(stubs) # curl: serve release assets and the tags API from fixtures. @@ -95,12 +144,54 @@ case "$url" in */tags\\?per_page=100) printf '[{{"name":"v9.9.9","commit":{{"sha":"{TAG_COMMIT}"}}}}]' ;; */bpfcompat-linux-amd64) printf '%s' '{CLI_BYTES.decode().replace(chr(10), "\\n")}' | sed 's/\\\\n/\\n/g' >"$out" ;; */bpfcompat-validator-static-linux-amd64) printf 'release-validator-bytes\\n' >"$out" ;; - */SHA256SUMS) printf '%s bpfcompat-validator-static-linux-amd64\\n0000 bpfcompat-linux-amd64\\n' '{sums_sha}' >"$out" ;; + */SHA256SUMS) printf '%s bpfcompat-validator-static-linux-amd64\\n%s bpfcompat-linux-amd64\\n' '{sums_sha}' '{CLI_SHA}' >"$out" ;; *) exit 22 ;; esac """) - write_exec(os.path.join(stubs, "gh"), - "#!/usr/bin/env bash\nexit %d\n" % (0 if gh_ok else 1)) + # gh: `gh attestation verify --help` lists the flags this gh has; verify + # checks the assets' attestation `attested` like gh does (an absent or + # empty --source-ref / --source-digest accepts any value). + help_file = os.path.join(case, "gh_help") + with open(help_file, "w") as fh: + fh.write(gh_help(gh_lacks, gh_help_extra)) + write_exec(os.path.join(stubs, "gh"), f"""#!/usr/bin/env bash +{"exit 1" if not gh_ok else ""} +if [[ "$1 ${{2:-}} ${{3:-}}" == "attestation verify --help" ]]; then + {"exit 1" if gh_help_fails else ""} + cat '{help_file}' + exit 0 +fi +[[ "$1 ${{2:-}}" == "attestation --help" ]] && exit 0 +[[ "$1 ${{2:-}}" == "attestation verify" ]] || exit 0 +shift 2 +ref="" src="" +while [[ $# -gt 0 ]]; do + case "$1" in + --repo) [[ "$2" == Kernel-Guard/bpfcompat ]] || exit 1; shift 2 ;; + --signer-workflow) [[ "$2" == Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml ]] || exit 1; shift 2 ;; + --source-ref) ref="$2"; shift 2 ;; + --source-digest) src="$2"; shift 2 ;; + --*) echo "unknown flag: $1" >&2; exit 1 ;; + *) shift ;; + esac +done +[[ -z "$ref" || "$ref" == '{attested[0]}' ]] || {{ echo "stub gh: expected SourceRepositoryRef $ref, got {attested[0]}" >&2; exit 1; }} +[[ -z "$src" || "$src" == '{attested[1]}' ]] || {{ echo "stub gh: expected SourceRepositoryDigest $src, got {attested[1]}" >&2; exit 1; }} +exit 0 +""") + git_log = os.path.join(case, "git_calls") + ls_file = os.path.join(case, "ls_remote") + with open(ls_file, "w") as fh: + fh.write(ls_remote or "") + write_exec(os.path.join(stubs, "git"), f"""#!/usr/bin/env bash +printf '%s\\n' "$*" >>'{git_log}' +if [[ "${{1:-}}" == ls-remote ]]; then + {"echo 'fatal: unable to access https://github.com/Kernel-Guard/bpfcompat.git/: Could not resolve host: github.com' >&2; exit 128" if ls_remote is None else ""} + cat '{ls_file}' + exit 0 +fi +exec '{REAL_GIT}' "$@" +""") write_exec(os.path.join(stubs, "uname"), f"#!/usr/bin/env bash\nif [[ \"$1\" == -m ]]; then echo {machine}; else /bin/uname \"$@\"; fi\n") outputs_file = os.path.join(case, "prebuilt_output") @@ -121,6 +212,12 @@ esac if "=" in line: k, v = line.rstrip("\n").split("=", 1) outputs[k] = v + LAST["verifier"] = None + if os.path.exists(verifier_log): + with open(verifier_log) as fh: + LAST["verifier"] = dict(l.rstrip("\n").split("=", 1) for l in fh if "=" in l) + LAST["git"] = open(git_log).read().splitlines() if os.path.exists(git_log) else [] + LAST["installed"] = os.path.exists(os.path.join(root, "bin", "bpfcompat")) return res.returncode, res.stdout + res.stderr, outputs, root @@ -258,6 +355,87 @@ rc, log, outs, root = run_prebuilt(case, "main") check(rc == 0 and outs.get("resolved") == "false" and "::warning::" not in log, "branch ref: documented source build, no warning") +# 8. Tag pin: verification is bound to the tag ref and the commit the +# annotated tag points at (the peeled ^{} line, not the tag object). +case = new_case() +rc, log, outs, root = run_prebuilt(case, "v9.9.9") +got = LAST["verifier"] or {} +check(rc == 0 and outs.get("resolved") == "true", "tag pin: prebuilt resolved") +check(got.get("REF") == "refs/tags/v9.9.9", "tag pin: verifier bound to refs/tags/v9.9.9") +check(got.get("SHA") == TAG_COMMIT, "tag pin: verifier bound to the tag's peeled commit") +check(any(c.startswith("ls-remote ") and "refs/tags/v9.9.9" in c for c in LAST["git"]), + "tag pin: the tag is resolved with git ls-remote") + +# 8b. A lightweight tag has no ^{} line: its own line is the commit. +case = new_case() +rc, log, outs, root = run_prebuilt(case, "v9.9.9", ls_remote=f"{TAG_COMMIT}\trefs/tags/v9.9.9\n") +check(rc == 0 and (LAST["verifier"] or {}).get("SHA") == TAG_COMMIT, + "lightweight tag pin: verifier bound to the tag's commit") + +# 8c. ls-remote patterns match ref tails, so other refs ending in the tag name +# may be listed too (sorted after it); only refs/tags/<tag> counts. +case = new_case() +rc, log, outs, root = run_prebuilt( + case, "v9.9.9", + ls_remote=ANNOTATED + f"{OTHER_COMMIT}\trefs/tags/x/refs/tags/v9.9.9\n{OTHER_COMMIT}\trefs/tags/x/refs/tags/v9.9.9^{{}}\n") +check(rc == 0 and (LAST["verifier"] or {}).get("SHA") == TAG_COMMIT, + "tag pin: refs that merely end in the tag name are ignored") + +# 8d. SHA pin: the pinned commit itself, and the release tag it maps to. +case = new_case() +rc, log, outs, root = run_prebuilt(case, TAG_COMMIT) +got = LAST["verifier"] or {} +check(rc == 0 and outs.get("resolved") == "true" and got.get("SHA") == TAG_COMMIT + and got.get("REF") == "refs/tags/v9.9.9", "SHA pin: verifier bound to the pinned commit and its release tag") +check(not any(c.startswith("ls-remote") for c in LAST["git"]), "SHA pin: no git ls-remote") + +# 9. The tag cannot be resolved to a commit: warned source build, not a job +# failure and not an unbound prebuilt install. +for label, listing in [("git ls-remote fails", None), + ("git ls-remote lists no such tag", ""), + ("git ls-remote lists only other refs", f"{OTHER_COMMIT}\trefs/tags/x/refs/tags/v9.9.9\n")]: + case = new_case() + rc, log, outs, root = run_prebuilt(case, "v9.9.9", ls_remote=listing) + check(rc == 0 and outs.get("resolved") == "false" and "::warning::" in log, + f"tag pin, {label}: warned source build") + check(LAST["verifier"] is None and not LAST["installed"] and "validator-sha256" not in outs, + f"tag pin, {label}: nothing verified or installed") + +# 10. A gh that cannot bind attestations to a ref and commit (older than +# 2.68.0), or whose help cannot be read: warned source build that names +# what is missing, never an unbound verification. +for label, kwargs, needle in [ + ("gh 2.51-2.67 (no --source-ref/--source-digest)", {"gh_lacks": ("--source-ref", "--source-digest")}, "--source-ref"), + ("gh 2.49-2.50 (no --signer-workflow either)", + {"gh_lacks": ("--signer-workflow", "--source-ref", "--source-digest")}, "--signer-workflow"), + ("gh without --source-digest", {"gh_lacks": ("--source-digest",)}, "--source-digest"), + ("help names only a longer flag", {"gh_lacks": ("--source-ref",), + "gh_help_extra": " --source-ref-name string (not the flag)"}, "--source-ref"), + ("gh attestation verify --help fails", {"gh_help_fails": True}, "gh >= 2.68.0"), +]: + case = new_case() + rc, log, outs, root = run_prebuilt(case, "v9.9.9", **kwargs) + check(rc == 0 and outs.get("resolved") == "false" and "::warning::" in log and needle in log, + f"{label}: warned source build naming {needle}") + check(LAST["verifier"] is None and not LAST["installed"], f"{label}: nothing verified or installed") + +# 11. The real verifier: assets attested for exactly this release are used; +# another release's genuine, attested assets are a hard error. +case = new_case() +rc, log, outs, root = run_prebuilt(case, "v9.9.9", verifier="real") +check(rc == 0 and outs.get("resolved") == "true" and LAST["installed"], + "real verifier: assets attested for v9.9.9 at its commit are used") +for label, ref, attested in [ + ("tag pin, assets of another tag's release run", "v9.9.9", ("refs/tags/v9.9.8", OTHER_COMMIT)), + ("tag pin, another tag's run at the same commit", "v9.9.9", ("refs/tags/v9.9.8", TAG_COMMIT)), + ("SHA pin, the tag's ref built from another commit", TAG_COMMIT, ("refs/tags/v9.9.9", OTHER_COMMIT)), +]: + case = new_case() + rc, log, outs, root = run_prebuilt(case, ref, verifier="real", attested=attested) + check(rc != 0 and outs.get("resolved") != "true" and not LAST["installed"] + and "stub gh: expected SourceRepository" in log, + f"real verifier, {label}: hard error, nothing installed") + if failures: print(f"action validator-pin regression: {len(failures)} check(s) FAILED", file=sys.stderr) sys.exit(1) diff --git a/scripts/release-workflow-gates_test.sh b/scripts/release-workflow-gates_test.sh new file mode 100755 index 0000000..18ebc0c --- /dev/null +++ b/scripts/release-workflow-gates_test.sh @@ -0,0 +1,497 @@ +#!/usr/bin/env bash +# Regression for the evidence and identity checks that run inside the release +# workflows themselves. A tag's own copy of these workflows is what executes +# for that release, so a check missing here cannot be added for a release +# after it is tagged. +# +# 1. Stable gate (release-artifacts.yml, "Stage production readiness +# evidence"): a ready readiness record must carry a completed rollback +# drill of this release's own candidate (drill alias and image digest), +# like the incident section. The gate's jq program is extracted from the +# workflow and run against real production-readiness-report.sh output. +# 2. Promotion (promote-release.yml): the exact run: bodies of the +# re-verification and image steps run against stub cosign/gh/docker. The +# draft's signature files and SBOM must be the candidate run's own +# copies; every cosign identity is the exact +# release-artifacts.yml@refs/tags/<tag> (a signature made by another v* +# tag's run is refused); attestation checks are bound to the tag ref and +# the expected commit. +# 3. Every workflow call of scripts/verify-release-assets.sh supplies the +# source commit and tag ref that script requires. +# +# Hermetic: no network, no real release assets. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +RELEASE_WF="${RELEASE_WF:-$ROOT/.github/workflows/release-artifacts.yml}" +PROMOTE_WF="${PROMOTE_WF:-$ROOT/.github/workflows/promote-release.yml}" +MAINTENANCE_WF="${MAINTENANCE_WF:-$ROOT/.github/workflows/maintenance-evidence.yml}" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +failures=0 +ok() { echo "[release-workflow-gates-test] ok: $*"; } +bad() { + echo "[release-workflow-gates-test] FAIL: $*" >&2 + failures=$((failures + 1)) +} + +# ------------------------------------------------------------------ 1. stable gate +# The same extraction production-readiness-report_test.sh uses, so both tests +# judge the program the workflow actually runs. +gate="$(awk ' + /jq -e --arg version "\$version" .$/ {f=1; next} + f && /^ *. --argjson c / {exit} + f {print} +' "$RELEASE_WF")" +[[ "$gate" == *'.gate_status == "ready"'* ]] || { + echo "[release-workflow-gates-test] could not extract the stable gate from ${RELEASE_WF}" >&2 + exit 1 +} +run_gate() { + jq -e --arg version 0.4.0 --argjson c "$(jq -c '.candidate' "$1")" "$gate" "$1" >/dev/null 2>&1 +} + +# Unedited production-readiness-report.sh output (attested mode) for the +# fixture input of production-readiness-report_test.sh, which also runs the +# extracted gate against freshly generated output. +cat >"$tmp/attested.json" <<'JSON' +{ + "schema_version": "v0.2", + "campaign_binding": "candidate-bound-v2", + "attestations_verified": true, + "gate_status": "ready", + "release_version": "0.4.0", + "supported_boundary": "CLI + GitHub Action + disposable QEMU/KVM validation", + "generated_at": "2026-10-19T06:18:49Z", + "campaigns": [ + {"workflow_run_id": 1000, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000001", "started_at": "2026-08-03T03:21:00Z", "report_sha256": "bbbfcb6775f7cc9aeb67c38de6344bf05fe55ba048b64565c6a808460063748a", "target_count": 1}, + {"workflow_run_id": 1001, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000002", "started_at": "2026-08-10T03:21:00Z", "report_sha256": "b453d76863cda6c5b8f908e61ec16a11343caebbb6b06632efcdb229f0e72e3b", "target_count": 1}, + {"workflow_run_id": 1002, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000003", "started_at": "2026-08-17T03:21:00Z", "report_sha256": "62aef96eef32fc2e74fb0c45d6f40b7d048a9a66c8e26f80ba0a9207fd99dda9", "target_count": 1}, + {"workflow_run_id": 1003, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000004", "started_at": "2026-08-24T03:21:00Z", "report_sha256": "34c6d19b431fc19ca3af155fb90a5afcc91da5611e3f98a50045f66e85804c41", "target_count": 1} + ], + "slo": { + "campaign_count": 4, + "target_executions": 4, + "infrastructure_errors": 0, + "target_duration_p95_ms": 1000, + "target_timeout_ms": 720000 + }, + "falco": { + "workflow_run_id": 2000, + "commit_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "started_at": "2026-08-03T06:00:00Z", + "report_sha256": "b066de461debfb7a52511e99a0e9f6209bbb0b2e25da441c1563659030c6e5a7", + "report_run_id": "falco-run", + "run_identity": "operator-asserted", + "started_after_candidate": true, + "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777", + "candidate_bound": true, + "required_profiles": ["ubuntu-22.04-5.15", "debian-12-6.1", "ubuntu-24.04-6.8", "almalinux-8-4.18", "almalinux-9-5.14"] + }, + "candidate": { + "tag": "v0.4.0-rc.1", + "commit_sha": "cccccccccccccccccccccccccccccccccccccccc", + "image": "ghcr.io/kernel-guard/bpfcompat@sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "evidence_sha256": "01463694049c7ae9040e4957ae63b2109a3a689ee53f5e1f7355bfd7825cd94b" + }, + "canaries": [ + {"milestone": "manual", "workflow_run_id": 3000, "completed_at": "2026-07-30T15:05:34Z", "minutes_after_candidate_generated": 65, "components_verified": true, "evidence_sha256": "9c600cc50ac1ad95d5b2eac72cd7e9bd64209e02527df5f9e23f6366db49e813"}, + {"milestone": "t-plus-24h", "workflow_run_id": 3001, "completed_at": "2026-07-31T15:05:34Z", "minutes_after_candidate_generated": 1505, "components_verified": true, "evidence_sha256": "ba0b104f1f0d6715fb4948d3109431426a00c1725b439f370068d94ab6b99323"}, + {"milestone": "t-plus-72h", "workflow_run_id": 3002, "completed_at": "2026-08-02T15:05:34Z", "minutes_after_candidate_generated": 4385, "components_verified": true, "evidence_sha256": "3f7fdf3d7c1a45e6f7dba2a644575d7bb3776737d3e438640878ed0a8695b53e"} + ], + "rollback": { + "completed": true, + "workflow_run_id": 5000, + "completed_at": "2026-07-30T16:00:00Z", + "alias": "ghcr.io/kernel-guard/bpfcompat:rollback-drill-v0.4.0-rc.1", + "known_good_digest": "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "candidate_digest": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "candidate_bound": true, + "evidence_sha256": "5ef1ac927828ee6b35d8fd048b2e0b5d6666fdee48f42dc205691b8d1b670d66" + }, + "incident": { + "completed": true, + "promotion_run_id": 6000, + "candidate_bound": true, + "evidence_sha256": "a72acc9b473bfb1d12f9aec7127f384a2d0b1cec7e9dbed7ee734bbd47bed6ba" + }, + "operator": { + "login": "ErenAri", + "approval_mode": "solo-maintainer", + "confirmed": true, + "candidate_evidence_sha256": "01463694049c7ae9040e4957ae63b2109a3a689ee53f5e1f7355bfd7825cd94b", + "evidence_sha256": "f54e8b7556deb6c44501acd61b384ee0c3447b6bcf7d7d3ba1bd78adda6f8fb0" + }, + "excluded_surfaces": ["runtime-loading", "agent", "api", "registry", "saas", "firecracker", "virtme-ng"] +} +JSON + +if run_gate "$tmp/attested.json"; then + ok "stable gate accepts unedited attested readiness output" +else + bad "stable gate refused unedited attested readiness output" +fi + +# A genuine drill of another candidate: the v0.3.8-rc.2 drill alias and that +# candidate's image digest (docs/engineering/workstream-status.md). +other_alias="ghcr.io/kernel-guard/bpfcompat:rollback-drill-v0.3.8-rc.2" +other_digest="sha256:046effcfe3575c89663cc925042278e0da0804afe4199fff1abf9e13fea26f1d" +# <label>|<jq edit of the attested output>; the edit itself may contain '|'. +while IFS='|' read -r label edit; do + jq --arg alias "$other_alias" --arg digest "$other_digest" "$edit" \ + "$tmp/attested.json" >"$tmp/gate-case.json" + if run_gate "$tmp/gate-case.json"; then + bad "stable gate accepted readiness with ${label}" + else + ok "stable gate refuses readiness with ${label}" + fi +done <<'CASES' +no rollback section|del(.rollback) +a null rollback section|.rollback = null +an empty rollback section|.rollback = {} +rollback.completed false|.rollback.completed = false +rollback.completed as the string "true"|.rollback.completed = "true" +a rollback section holding only completed true|.rollback = {completed: true} +the drill of another candidate (v0.3.8-rc.2 alias and digest)|.rollback.alias = $alias | .rollback.candidate_digest = $digest +another candidate's drill alias|.rollback.alias = $alias +another candidate's image digest|.rollback.candidate_digest = $digest +this tag's drill alias in another image repository|.rollback.alias = "ghcr.io/example/bpfcompat:rollback-drill-v0.4.0-rc.1" +no incident section (control)|del(.incident) +incident.completed false (control)|.incident.completed = false +CASES + +# ------------------------------------------------------------------ 2. promotion +repo=Kernel-Guard/bpfcompat +tag=v0.4.0-rc.5 +commit="$(printf 'c%.0s' {1..40})" +other_commit="$(printf 'e%.0s' {1..40})" +digest="sha256:$(printf 'd%.0s' {1..64})" +image=ghcr.io/kernel-guard/bpfcompat +san="https://github.com/${repo}/.github/workflows/release-artifacts.yml@refs/tags/" +work="$tmp/promote" +mkdir -p "$work/stubs" + +# The exact run: bodies of the two steps, and where the draft's signature +# files and SBOM are compared with the candidate run's copies. +python3 - "$PROMOTE_WF" "$work" <<'PY' +import re +import sys + +import yaml + +wf = yaml.safe_load(open(sys.argv[1], encoding="utf-8")) +steps = {s.get("name", ""): s for s in wf["jobs"]["promote"]["steps"]} +bodies = { + "Re-verify evidence, draft assets, signatures, and channel": "verify-step.sh", + "Verify candidate image immediately before promotion": "image-step.sh", +} +for name, out in bodies.items(): + if name not in steps: + sys.exit(f"promote-release.yml has no step {name!r}") + with open(f"{sys.argv[2]}/{out}", "w", encoding="utf-8") as fh: + fh.write(steps[name]["run"]) +body = steps["Re-verify evidence, draft assets, signatures, and channel"]["run"] +compared = set(re.findall(r'cmp "?candidate/run-assets/([^"\s/]+)"? "?candidate/draft-assets/\1"?', body)) +for var, names in re.findall(r'for (\w+) in ([^;]*); do', body): + if re.search(r'cmp "candidate/run-assets/\$\{%s\}" "candidate/draft-assets/\$\{%s\}"' % (var, var), body): + compared.update(names.replace("\\\n", " ").split()) +with open(f"{sys.argv[2]}/compared.txt", "w", encoding="utf-8") as fh: + fh.write("\n".join(sorted(compared)) + "\n") +PY + +for signed in SHA256SUMS.sig SHA256SUMS.crt \ + bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt; do + if grep -Fxq "$signed" "$work/compared.txt"; then + ok "promotion compares the draft's ${signed} with the candidate run's copy" + else + bad "promotion does not compare the draft's ${signed} with the candidate run's copy" + fi +done + +cat >"$work/stubs/cosign" <<'EOF' +#!/usr/bin/env bash +# Certificate file: SAN=<identity>, KEY=<signing key>, SERIAL=<n>. Signature +# file: SIG:<sha256 of the blob>:<key>:<nonce>. A signature verifies only for +# the same bytes and a certificate of the same key; SERIAL and the nonce model +# a re-issued certificate and a fresh signature, which both still verify. +# Image signatures: "<image@digest> <identity> <key>" lines in $IMAGE_SIGS. +set -euo pipefail +sub="$1" +shift +cert="" sig="" ident="" identre="" issuer="" pos=() +while [[ $# -gt 0 ]]; do + case "$1" in + --certificate) cert="$2"; shift 2 ;; + --signature) sig="$2"; shift 2 ;; + --certificate-identity) ident="$2"; shift 2 ;; + --certificate-identity-regexp) identre="$2"; shift 2 ;; + --certificate-oidc-issuer) issuer="$2"; shift 2 ;; + --*) echo "stub cosign: unexpected flag $1" >&2; exit 64 ;; + *) pos+=("$1"); shift ;; + esac +done +[[ "$issuer" == https://token.actions.githubusercontent.com ]] || + { echo "stub cosign: wrong issuer" >&2; exit 1; } +matches() { + if [[ -n "$ident" ]]; then + [[ "$1" == "$ident" ]] + elif [[ -n "$identre" ]]; then + grep -Eq -- "$identre" <<<"$1" + else + return 1 + fi +} +case "$sub" in + verify-blob) + san="$(sed -n 's/^SAN=//p' "$cert")" + key="$(sed -n 's/^KEY=//p' "$cert")" + IFS=: read -r kind blob_sha sig_key _ <"$sig" || true + [[ "$kind" == SIG && "$sig_key" == "$key" && + "$blob_sha" == "$(sha256sum "${pos[0]}" | awk '{print $1}')" ]] || + { echo "stub cosign: invalid signature" >&2; exit 1; } + matches "$san" || { echo "stub cosign: none of the expected identities matched: $san" >&2; exit 1; } + echo "Verified OK" >&2 + ;; + verify) + while read -r ref id _; do + if [[ "$ref" == "${pos[0]}" ]] && matches "$id"; then exit 0; fi + done <"$IMAGE_SIGS" + echo "stub cosign: no matching signatures" >&2 + exit 1 + ;; + *) echo "stub cosign: unexpected subcommand $sub" >&2; exit 64 ;; +esac +EOF +cat >"$work/stubs/gh" <<'EOF' +#!/usr/bin/env bash +# gh attestation verify backed by $ATTEST_DB: one "<sha256> <source ref> +# <source commit>" line per release-workflow attestation. Like gh, a check +# whose flag is absent accepts any value. +set -euo pipefail +case "$1 ${2:-}" in + "attestation verify") + shift 2 + file="" repo="" signer="" ref="" source="" + while [[ $# -gt 0 ]]; do + case "$1" in + --repo) repo="$2"; shift 2 ;; + --signer-workflow) signer="$2"; shift 2 ;; + --source-ref) ref="$2"; shift 2 ;; + --source-digest) source="$2"; shift 2 ;; + --*) echo "stub gh: unexpected flag $1" >&2; exit 64 ;; + *) file="$1"; shift ;; + esac + done + [[ "$repo" == Kernel-Guard/bpfcompat && + "$signer" == Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml ]] || + { echo "stub gh: wrong repository or signer workflow" >&2; exit 1; } + sha="$(sha256sum "$file" | awk '{print $1}')" + while read -r s r c; do + if [[ "$s" == "$sha" && ( -z "$ref" || "$ref" == "$r" ) && ( -z "$source" || "$source" == "$c" ) ]]; then + exit 0 + fi + done <"$ATTEST_DB" + echo "stub gh: no attestation of ${file##*/} matched" >&2 + exit 1 + ;; + "release view") + printf '{"tagName":"%s","isDraft":true,"isPrerelease":true}\n' "$RELEASE_TAG" + ;; + *) echo "stub gh: unexpected $*" >&2; exit 64 ;; +esac +EOF +cat >"$work/stubs/docker" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail +[[ "$1 $2" == "run --rm" ]] || { echo "stub docker: unexpected $*" >&2; exit 64; } +printf '{"version":"%s"}\n' "$STUB_IMAGE_VERSION" +EOF +chmod +x "$work/stubs/"* + +mkcert() { printf 'SAN=%s\nKEY=%s\nSERIAL=%s\n' "$2" "$3" "${4:-1}" >"$1"; } +mksig() { printf 'SIG:%s:%s:%s\n' "$(sha256sum "$2" | awk '{print $1}')" "$3" "${4:-1}" >"$1"; } +sbom() { printf '{"bomFormat":"CycloneDX","serialNumber":"%s"}\n' "$2" >"$1"; } + +fixture() { # $1 = case directory: an untouched rc.5 candidate and draft + local d="$1" r="$1/candidate/run-assets" evidence_sha + mkdir -p "$r" "$d/candidate/run-evidence" "$d/scripts" + printf 'release_version: 0.4.0-rc.5\nrelease_channel: prerelease\n' >"$d/release.yaml" + # Both scripts have their own regressions. Here they are neutral, and the + # asset verifier records the identity it is given (asserted below). + printf '#!/usr/bin/env bash\nexit 0\n' >"$d/scripts/verify-promotion-inputs.sh" + cat >"$d/scripts/verify-release-assets.sh" <<'EOF' +#!/usr/bin/env bash +printf 'SHA=%s REF=%s\n' "${BPFCOMPAT_EXPECTED_SOURCE_SHA-<unset>}" \ + "${BPFCOMPAT_EXPECTED_SOURCE_REF-<unset>}" >>"$VRA_LOG" +EOF + printf '%s bpfcompat-linux-amd64\n' "$(printf 'a%.0s' {1..64})" >"$r/SHA256SUMS" + sbom "$r/bpfcompat.sbom.cdx.json" candidate-run + mkcert "$r/SHA256SUMS.crt" "${san}${tag}" rc5-sums + mksig "$r/SHA256SUMS.sig" "$r/SHA256SUMS" rc5-sums + mkcert "$r/bpfcompat.sbom.cdx.crt" "${san}${tag}" rc5-sbom + mksig "$r/bpfcompat.sbom.cdx.sig" "$r/bpfcompat.sbom.cdx.json" rc5-sbom + printf '{"tag":"%s","commit_sha":"%s"}\n' "$tag" "$commit" \ + >"$d/candidate/run-evidence/release-candidate-evidence.json" + # stage-draft-release uploads the same run's files. + cp -R "$r" "$d/candidate/draft-assets" + cp "$d/candidate/run-evidence/release-candidate-evidence.json" "$d/candidate/draft-assets/" + printf '%s@%s %s%s rc5-image\n' "$image" "$digest" "$san" "$tag" >"$d/image-sigs" + evidence_sha="$(sha256sum "$d/candidate/run-evidence/release-candidate-evidence.json" | awk '{print $1}')" + printf '%s refs/tags/%s %s\n' "$evidence_sha" "$tag" "$commit" >"$d/attest-db" +} + +tamper() { # $1 = case, $2 = case directory + local draft="$2/candidate/draft-assets" run="$2/candidate/run-assets" f + case "$1" in + untouched-draft) ;; + sbom-from-another-tag-run) + # The public v0.4.0-rc.4 SBOM, signature and certificate. + sbom "$draft/bpfcompat.sbom.cdx.json" rc4-run + mkcert "$draft/bpfcompat.sbom.cdx.crt" "${san}v0.4.0-rc.4" rc4-sbom + mksig "$draft/bpfcompat.sbom.cdx.sig" "$draft/bpfcompat.sbom.cdx.json" rc4-sbom ;; + checksums-signed-by-a-new-tag) + # The same SHA256SUMS bytes signed by a run on a newly created v* tag. + mkcert "$draft/SHA256SUMS.crt" "${san}v9.9.9-rogue" rogue + mksig "$draft/SHA256SUMS.sig" "$draft/SHA256SUMS" rogue ;; + same-tag-sbom-not-from-candidate-run) + sbom "$draft/bpfcompat.sbom.cdx.json" other-run + mkcert "$draft/bpfcompat.sbom.cdx.crt" "${san}${tag}" other-run-sbom + mksig "$draft/bpfcompat.sbom.cdx.sig" "$draft/bpfcompat.sbom.cdx.json" other-run-sbom ;; + image-signed-only-by-another-tag) + printf '%s@%s %sv0.4.0-rc.4 rc4-image\n' "$image" "$digest" "$san" >"$2/image-sigs" ;; + checksums-signature-not-from-run) + mksig "$draft/SHA256SUMS.sig" "$draft/SHA256SUMS" rc5-sums 2 ;; + checksums-certificate-not-from-run) + mkcert "$draft/SHA256SUMS.crt" "${san}${tag}" rc5-sums 2 ;; + sbom-signature-not-from-run) + mksig "$draft/bpfcompat.sbom.cdx.sig" "$draft/bpfcompat.sbom.cdx.json" rc5-sbom 2 ;; + sbom-certificate-not-from-run) + mkcert "$draft/bpfcompat.sbom.cdx.crt" "${san}${tag}" rc5-sbom 2 ;; + run-checksums-signed-by-another-tag) + # Identity is the only barrier: run artifact and draft agree. + for f in "$run" "$draft"; do + mkcert "$f/SHA256SUMS.crt" "${san}v0.4.0-rc.4" rc4-sums + mksig "$f/SHA256SUMS.sig" "$f/SHA256SUMS" rc4-sums + done ;; + run-sbom-signed-by-another-tag) + for f in "$run" "$draft"; do + mkcert "$f/bpfcompat.sbom.cdx.crt" "${san}v0.4.0-rc.4" rc4-sbom + mksig "$f/bpfcompat.sbom.cdx.sig" "$f/bpfcompat.sbom.cdx.json" rc4-sbom + done ;; + evidence-attested-by-another-tag-at-the-commit) + awk -v c="$commit" '{print $1, "refs/tags/v0.4.0-rc.4", c}' "$2/attest-db" >"$2/attest-db.new" + mv "$2/attest-db.new" "$2/attest-db" ;; + evidence-attested-for-the-tag-at-another-commit) + awk -v c="$other_commit" '{print $1, $2, c}' "$2/attest-db" >"$2/attest-db.new" + mv "$2/attest-db.new" "$2/attest-db" ;; + tampered-sbom-bytes-control) + sbom "$draft/bpfcompat.sbom.cdx.json" tampered ;; + *) echo "unknown case $1" >&2; exit 1 ;; + esac +} + +promote() { # $1 = case directory; the two steps in job order + local d="$1" + ( + cd "$d" + export PATH="$work/stubs:$PATH" GITHUB_REPOSITORY="$repo" RELEASE_TAG="$tag" \ + CANDIDATE_RUN_ID=38600000000 CANDIDATE_RUN_ATTEMPT=1 EXPECTED_COMMIT="$commit" \ + EXPECTED_DIGEST="$digest" GH_TOKEN=stub IMAGE="$image" IMAGE_SIGS="$d/image-sigs" \ + ATTEST_DB="$d/attest-db" VRA_LOG="$d/vra.log" STUB_IMAGE_VERSION="$tag" + # The runner's shell for run: steps; && because an if suspends set -e. + bash --noprofile --norc -eo pipefail "$work/verify-step.sh" >"$d/verify.log" 2>&1 && + bash --noprofile --norc -eo pipefail "$work/image-step.sh" >"$d/image.log" 2>&1 + ) +} + +# A refusal counts only for the reason the case is about (an ERE matched +# against the step logs): the draft differs from the candidate run's copy +# (cmp), the signer identity is not this tag's run, the evidence attestation +# is not this tag at this commit, or the signature does not cover the bytes. +declare -A REASON=( + [CMP]='differ' + [ID]='none of the expected identities matched|no matching signatures' + [ATT]='no attestation of release-candidate-evidence.json matched' + [SIG]='invalid signature' +) +REASON[CMP_OR_ID]="${REASON[CMP]}|${REASON[ID]}" +REASON[CMP_OR_SIG]="${REASON[CMP]}|${REASON[SIG]}" +while read -r name expect reason; do + d="$work/$name" + fixture "$d" + tamper "$name" "$d" + if promote "$d" </dev/null; then got=pass; else got=refuse; fi + if [[ "$got" == "$expect" && "$got" == pass ]] || + { [[ "$got" == "$expect" ]] && grep -Eqs -- "${REASON[$reason]}" "$d/verify.log" "$d/image.log"; }; then + ok "promotion of ${tag}, ${name}: ${got}${reason:+ (${reason})}" + else + bad "promotion of ${tag}, ${name}: expected ${expect}${reason:+ (${reason})}, got ${got}" + cat "$d/verify.log" "$d/image.log" 2>/dev/null >&2 || true + fi +done <<'CASES' +untouched-draft pass +sbom-from-another-tag-run refuse CMP_OR_ID +checksums-signed-by-a-new-tag refuse CMP_OR_ID +same-tag-sbom-not-from-candidate-run refuse CMP +image-signed-only-by-another-tag refuse ID +checksums-signature-not-from-run refuse CMP +checksums-certificate-not-from-run refuse CMP +sbom-signature-not-from-run refuse CMP +sbom-certificate-not-from-run refuse CMP +run-checksums-signed-by-another-tag refuse ID +run-sbom-signed-by-another-tag refuse ID +evidence-attested-by-another-tag-at-the-commit refuse ATT +evidence-attested-for-the-tag-at-another-commit refuse ATT +tampered-sbom-bytes-control refuse CMP_OR_SIG +CASES + +want="SHA=${commit} REF=refs/tags/${tag}" +if [[ "$(cat "$work/untouched-draft/vra.log" 2>/dev/null)" == "$want" ]]; then + ok "promotion verifies the draft assets for ${tag} at the expected commit" +else + bad "promotion passed verify-release-assets.sh '$(cat "$work/untouched-draft/vra.log" 2>/dev/null)', not '${want}'" +fi + +# ------------------------------------------------------------------ 3. wiring +# The tag build's gates verify the run's own artifact; maintenance evidence +# verifies the published prerelease of the dispatched rc tag. Each passes the +# commit and tag ref it runs on, which verify-release-assets.sh requires. +wiring="$(python3 - "$RELEASE_WF" "$MAINTENANCE_WF" <<'PY' +import sys + +import yaml + +want = { + "BPFCOMPAT_EXPECTED_SOURCE_SHA": "${{ github.sha }}", + "BPFCOMPAT_EXPECTED_SOURCE_REF": "${{ github.ref }}", +} +expected_calls = {sys.argv[1]: 2, sys.argv[2]: 1} +for path, count in expected_calls.items(): + wf = yaml.safe_load(open(path, encoding="utf-8")) + name = path.rsplit("/", 1)[-1] + calls = 0 + for job_id, job in wf["jobs"].items(): + for step in job.get("steps", []): + if "scripts/verify-release-assets.sh" not in (step.get("run") or ""): + continue + calls += 1 + env = step.get("env") or {} + good = all(env.get(k) == v for k, v in want.items()) + print(("ok " if good else "FAIL ") + + f"{name} {job_id} / {step.get('name')}: verify-release-assets.sh gets the run's commit and tag ref") + print(("ok " if calls == count else "FAIL ") + f"{name}: {calls} verify-release-assets.sh call(s), expected {count}") +PY +)" +while IFS= read -r line; do + case "$line" in + ok\ *) ok "${line#ok }" ;; + *) bad "${line#FAIL }" ;; + esac +done <<<"$wiring" + +if [[ "$failures" -ne 0 ]]; then + echo "[release-workflow-gates-test] ${failures} check(s) FAILED" >&2 + exit 1 +fi +echo "[release-workflow-gates-test] PASS" diff --git a/scripts/verify-release-assets.sh b/scripts/verify-release-assets.sh index a9c0b19..1e46657 100755 --- a/scripts/verify-release-assets.sh +++ b/scripts/verify-release-assets.sh @@ -29,6 +29,15 @@ gh attestation --help >/dev/null 2>&1 || { exit 1 } +# Provenance is bound to exactly one release: the tag ref and the commit its +# release run built. Without them any v* tag's attestation would pass. +[[ "${BPFCOMPAT_EXPECTED_SOURCE_SHA:-}" =~ ^[0-9a-f]{40}$ ]] || { + echo '[release-assets] exact source commit is required' >&2; exit 2; +} +[[ "${BPFCOMPAT_EXPECTED_SOURCE_REF:-}" =~ ^refs/tags/v[0-9A-Za-z.-]+$ ]] || { + echo '[release-assets] exact release tag ref is required' >&2; exit 2; +} + checklist="$(mktemp "${asset_dir}/.bpfcompat-checksums.XXXXXX")" trap 'rm -f "$checklist"' EXIT @@ -70,7 +79,9 @@ done for asset in "${assets[@]}"; do gh attestation verify "${asset_dir}/${asset}" \ --repo "$repo" \ - --signer-workflow "${repo}/.github/workflows/release-artifacts.yml" + --signer-workflow "${repo}/.github/workflows/release-artifacts.yml" \ + --source-digest "$BPFCOMPAT_EXPECTED_SOURCE_SHA" \ + --source-ref "$BPFCOMPAT_EXPECTED_SOURCE_REF" done echo "[release-assets] checksums and release-workflow attestations verified" diff --git a/scripts/verify-release-assets_test.sh b/scripts/verify-release-assets_test.sh index c897486..302a1e7 100755 --- a/scripts/verify-release-assets_test.sh +++ b/scripts/verify-release-assets_test.sh @@ -10,6 +10,10 @@ # took the action down on every amd64 runner before a VM had booted. Verifying # exactly the downloaded assets fixed it -- without loosening anything else, # which is what the remaining cases pin down. +# +# Provenance is bound to one release: the caller names the tag ref and the +# commit it was built from, and an attestation made by any other release run +# (another tag, or the same tag at another commit) does not vouch for it. set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -21,10 +25,18 @@ CLI=bpfcompat-linux-amd64 VALIDATOR=bpfcompat-validator-static-linux-amd64 ARM64=bpfcompat-linux-arm64 +RELEASE_COMMIT="$(printf 'c%.0s' {1..40})" +OTHER_COMMIT="$(printf 'e%.0s' {1..40})" +export BPFCOMPAT_EXPECTED_SOURCE_SHA="$RELEASE_COMMIT" +export BPFCOMPAT_EXPECTED_SOURCE_REF=refs/tags/v0.4.0-rc.5 + mkdir -p "$tmp/bin" # A stub gh that accepts, but only for the exact repo and signer workflow the # real verification demands -- so a call that drops either argument fails here. +# The assets carry the release-workflow attestation BPFCOMPAT_TEST_ATTESTED +# ("<source ref> <source commit>", default: the expected release). Like gh, a +# --source-ref / --source-digest that is absent or empty accepts any value. cat >"$tmp/bin/gh" <<'EOF' #!/usr/bin/env bash set -euo pipefail @@ -34,7 +46,24 @@ fi if [[ "${1:-}" == "attestation" && "${2:-}" == "verify" ]]; then [[ "$*" == *"--repo Kernel-Guard/bpfcompat"* ]] [[ "$*" == *"--signer-workflow Kernel-Guard/bpfcompat/.github/workflows/release-artifacts.yml"* ]] - [[ "${BPFCOMPAT_TEST_ATTESTATION:-ok}" == "ok" ]] + [[ "${BPFCOMPAT_TEST_ATTESTATION:-ok}" == "ok" ]] || { echo "stub gh: attestation broken" >&2; exit 1; } + read -r attested_ref attested_commit <<<"${BPFCOMPAT_TEST_ATTESTED:-refs/tags/v0.4.0-rc.5 $(printf 'c%.0s' {1..40})}" + shift 2 + while [[ $# -gt 0 ]]; do + case "$1" in + --repo|--signer-workflow) shift 2 ;; + --source-ref) + [[ -z "$2" || "$2" == "$attested_ref" ]] || + { echo "stub gh: expected SourceRepositoryRef $2, got $attested_ref" >&2; exit 1; } + shift 2 ;; + --source-digest) + [[ -z "$2" || "$2" == "$attested_commit" ]] || + { echo "stub gh: expected SourceRepositoryDigest $2, got $attested_commit" >&2; exit 1; } + shift 2 ;; + --*) echo "stub gh: unknown flag $1" >&2; exit 1 ;; + *) shift ;; + esac + done exit 0 fi exit 1 @@ -141,6 +170,11 @@ if BPFCOMPAT_TEST_ATTESTATION=broken \ echo "[release-assets-test] expected FAIL but verification passed: failed attestation" >&2 exit 1 fi +grep -Fq "stub gh: attestation broken" "$tmp/out.log" || { + echo "[release-assets-test] failed-attestation case failed for the wrong reason:" >&2 + cat "$tmp/out.log" >&2 + exit 1 +} # 11. A runner with no usable gh must not fall through to "verified". The PATH # has to be genuinely empty -- keeping /usr/bin on it leaves the real gh @@ -160,4 +194,51 @@ grep -Fq "GitHub CLI is required for attestation verification" "$tmp/out.log" || exit 1 } +# 12. Assets of another release, with their own SHA256SUMS and their genuine +# release-workflow attestations, presented as this release: refused. One +# case per binding, so dropping either --source-ref or --source-digest +# fails here. +for attested in \ + "refs/tags/v0.4.0-rc.4 ${OTHER_COMMIT}" \ + "refs/tags/v0.4.0-rc.4 ${RELEASE_COMMIT}" \ + "refs/tags/v0.4.0-rc.5 ${OTHER_COMMIT}"; do + seed "$d" + if BPFCOMPAT_TEST_ATTESTED="$attested" verify "$d" "$CLI" "$VALIDATOR"; then + echo "[release-assets-test] expected FAIL but verification passed: attested by ${attested}" >&2 + exit 1 + fi + grep -Eq "stub gh: expected SourceRepository(Ref|Digest) " "$tmp/out.log" || { + echo "[release-assets-test] attestation of ${attested} failed for the wrong reason:" >&2 + cat "$tmp/out.log" >&2 + exit 1 + } +done + +# 13. Without the exact release identity the verifier refuses to run at all +# (exit 2) instead of accepting any release's attestation. +expect_usage_error() { # $1 = label, $2 = expected diagnostic, rest = env assignments + local label="$1" message="$2" + shift 2 + seed "$d" + local rc=0 + env "$@" PATH="$tmp/bin:$PATH" bash "$verifier" "$d" Kernel-Guard/bpfcompat \ + "$CLI" "$VALIDATOR" >"$tmp/out.log" 2>&1 || rc=$? + if [[ "$rc" -ne 2 ]] || ! grep -Fq "$message" "$tmp/out.log"; then + echo "[release-assets-test] ${label}: expected exit 2 with '${message}', got exit ${rc}:" >&2 + cat "$tmp/out.log" >&2 + exit 1 + fi +} +commit_required="exact source commit is required" +ref_required="exact release tag ref is required" +expect_usage_error "no source commit" "$commit_required" -u BPFCOMPAT_EXPECTED_SOURCE_SHA +expect_usage_error "empty source commit" "$commit_required" BPFCOMPAT_EXPECTED_SOURCE_SHA= +expect_usage_error "abbreviated source commit" "$commit_required" BPFCOMPAT_EXPECTED_SOURCE_SHA=cccccccc +expect_usage_error "upper-case source commit" "$commit_required" \ + BPFCOMPAT_EXPECTED_SOURCE_SHA="$(printf 'C%.0s' {1..40})" +expect_usage_error "no release ref" "$ref_required" -u BPFCOMPAT_EXPECTED_SOURCE_REF +expect_usage_error "branch ref" "$ref_required" BPFCOMPAT_EXPECTED_SOURCE_REF=refs/heads/main +expect_usage_error "bare tag name" "$ref_required" BPFCOMPAT_EXPECTED_SOURCE_REF=v0.4.0-rc.5 +expect_usage_error "case-variant tag ref" "$ref_required" BPFCOMPAT_EXPECTED_SOURCE_REF=refs/tags/V0.4.0-rc.5 + echo "[release-assets-test] PASS" From 6705bb1bad0847bef401cf8c24d92761397cd657 Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 09:39:55 +0300 Subject: [PATCH 31/38] install.sh: retry the API probe on a 5xx or no response only The RC canary installs through the gh path, where one transient 5xx or dropped connection at the read-only API probe ended the install, although gh's own verify retries a 5xx. The probe now makes up to three attempts, 3 s apart (6 s of waiting at most), for a 5xx or no HTTP response only, and the last attempt decides. A 429 or rate-limited 403 is not retried: GitHub asks the client to wait longer than an installer should, and asking again sooner spends more of the limit. 401, 404 and 200 are answers. Every outcome still refuses or verifies as before. Text quoted from gh is reduced to printable ASCII by an allow-list (octal, because busybox tr has no [:print:] class), so C1 controls such as a raw or UTF-8 encoded CSI no longer pass. Tests: retry sequences (5xx then 200; 502, 503, 200; no response then 200; 5xx then 401, 429 or 404; no response then a rate-limited 403 or a refusal) with exact attempt counts and waits; C1 bytes in both quoting places; and the defensive branches no case pinned: a probe exit other than 1 or 4 with no output, exit 1 with output that is no status line, HTTP/1.1 status lines, and a verify exit other than 0, 1 or 4. Signed-off-by: ErenAri <erenari27@gmail.com> --- scripts/install.sh | 60 ++++++++----- scripts/install_test.sh | 181 +++++++++++++++++++++++++++++++++------- 2 files changed, 191 insertions(+), 50 deletions(-) diff --git a/scripts/install.sh b/scripts/install.sh index 041f27a..4474d6d 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -35,9 +35,13 @@ ALT_LIBEXEC_DIR="/usr/libexec/bpfcompat" err() { printf 'bpfcompat-install: %s\n' "$*" >&2; } die() { err "$*"; exit 1; } have() { command -v "$1" >/dev/null 2>&1; } -# first_line FILE: its first non-blank line, without control characters, to -# quote gh's own words in a refusal (never more, never terminal escapes). -first_line() { sed -n '/[^[:space:]]/{p;q;}' "$1" | tr -d '[:cntrl:]'; } +# printable: keep printable ASCII only (space to '~'), so text quoted from +# gh carries no C0 or C1 control (terminal escapes) and no other bytes. +# Octal, not [:print:]: busybox tr has no [:print:] class. +printable() { LC_ALL=C tr -cd '\040-\176'; } +# first_line FILE: its first non-blank line, printable, to quote gh's own +# words in a refusal (never more, never terminal escapes). +first_line() { sed -n '/[^[:space:]]/{p;q;}' "$1" | printable; } # gh_no_auth REASON FILE: gh cannot authenticate to GitHub; quote FILE's # first line (gh's own words) and refuse. gh_no_auth() { @@ -176,7 +180,7 @@ elif have gh; then # A gh that cannot run at all (say, an unreadable config) fails every # command; say so with gh's own error rather than blame its version. if ! gh --help >/dev/null 2>&1; then - gh_err="$( { gh --help >/dev/null; } 2>&1 | sed -n '1p' | tr -d '[:cntrl:]')" || : + gh_err="$( { gh --help >/dev/null; } 2>&1 | sed -n '1p' | printable)" || : die "installed gh could not run 'gh --help'${gh_err:+ ($gh_err)}; fix gh or install cosign - refusing checksum-only installation" fi gh_lacks="the 'gh attestation' command" @@ -197,7 +201,7 @@ elif have gh; then # token rejected, or the GitHub API out of reach. Tell those apart from a # refused verification without trusting gh's wording, which changes # between releases and is all gh prints when stdout is not a terminal: - # before the first verify call, make one read-only request to the + # before the first verify call, make a read-only request to the # attestation API that call uses (same gh, host, token and proxy), and # refuse here only on positive evidence - gh's documented auth exit code # (4), an HTTP 401, no HTTP response at all, a 5xx, a 429 or a @@ -206,24 +210,36 @@ elif have gh; then # always reported as a failed verification. Probing first, not after a # failure, means a refusal can never be relabelled as an auth or network # problem, however gh words it or whatever the API does next. + # A 5xx or no HTTP response can be transient (gh's own verify retries a + # 5xx), so those alone are tried again: three attempts at most, 3 s apart, + # and the last one decides. A 429 or rate-limited 403 is not retried: it + # asks the client to wait (Retry-After, X-Ratelimit-Reset) far longer than + # an installer should, and asking again sooner only spends more of the + # limit. 200, 404 and 401 are answers. gh_digest="$($SHA256 <"$CLI_ASSET" | sed -n '1s/^\([0-9a-fA-F]\{64\}\).*/\1/p')" || : - gh_rc=0 - gh api --method GET --include "repos/$REPO/attestations/sha256:$gh_digest?per_page=1" \ - >.gh-api.out 2>.gh-api.err || gh_rc=$? - gh_status="$(tr -d '\r' <.gh-api.out | sed -n '1s/^HTTP\/[0-9.]* \([0-9][0-9][0-9]\).*/\1/p')" || : - gh_down="" - case "$gh_rc:$gh_status" in - 4:) gh_no_auth "gh is not logged in" .gh-api.err ;; - *:401) gh_no_auth "GitHub rejected gh's credentials (HTTP 401)" .gh-api.err ;; - *:5[0-9][0-9]) gh_down="HTTP $gh_status" ;; - *:429) gh_down="HTTP 429, rate limited" ;; - *:403) - # A 403 is an outage only with GitHub's own rate-limit headers. - gh_rl="$(tr -d '\r\033' <.gh-api.out | sed -n '2,/^$/{s/\[[0-9;]*m//g;p;}' | - grep -ciE '^(x-ratelimit-remaining: *0|retry-after: *[0-9]+)$')" || : - [ "${gh_rl:-0}" = 0 ] || gh_down="HTTP 403, rate limited" ;; - 1:) [ -s .gh-api.out ] || gh_down="no HTTP response" ;; - esac + gh_try=1 + while :; do + gh_rc=0 + gh api --method GET --include "repos/$REPO/attestations/sha256:$gh_digest?per_page=1" \ + >.gh-api.out 2>.gh-api.err || gh_rc=$? + gh_status="$(tr -d '\r' <.gh-api.out | sed -n '1s/^HTTP\/[0-9.]* \([0-9][0-9][0-9]\).*/\1/p')" || : + gh_down="" gh_again="" + case "$gh_rc:$gh_status" in + 4:) gh_no_auth "gh is not logged in" .gh-api.err ;; + *:401) gh_no_auth "GitHub rejected gh's credentials (HTTP 401)" .gh-api.err ;; + *:5[0-9][0-9]) gh_down="HTTP $gh_status" gh_again=1 ;; + *:429) gh_down="HTTP 429, rate limited" ;; + *:403) + # A 403 is an outage only with GitHub's own rate-limit headers. + gh_rl="$(tr -d '\r\033' <.gh-api.out | sed -n '2,/^$/{s/\[[0-9;]*m//g;p;}' | + grep -ciE '^(x-ratelimit-remaining: *0|retry-after: *[0-9]+)$')" || : + [ "${gh_rl:-0}" = 0 ] || gh_down="HTTP 403, rate limited" ;; + 1:) [ -s .gh-api.out ] || gh_down="no HTTP response" gh_again=1 ;; + esac + if [ -z "$gh_again" ] || [ "$gh_try" -ge 3 ]; then break; fi + gh_try=$((gh_try + 1)) + sleep 3 + done if [ -n "$gh_down" ]; then gh_err="$(first_line .gh-api.err)" || : die "GitHub API unavailable for attestation verification: $gh_down${gh_err:+ ($gh_err)}; check the network or proxy and https://www.githubstatus.com, then retry, or install cosign - refusing checksum-only installation" diff --git a/scripts/install_test.sh b/scripts/install_test.sh index c7b5f14..2be0311 100644 --- a/scripts/install_test.sh +++ b/scripts/install_test.sh @@ -20,7 +20,9 @@ # also answers the installer's read-only API probe (`gh api --include`) # from the attestation DB and records it, and can play a capable gh that # is not logged in, holds a rejected token, cannot reach the API or gets -# a 5xx / 429 / rate-limited 403, modelled on real gh 2.68.0 / 2.102.0; +# a 5xx / 429 / rate-limited 403, modelled on real gh 2.68.0 / 2.102.0, +# one probe attempt at a time; +# * sleep only records how long the installer would have waited; # * sudo pretends to be root by unlocking the test's read-only tree. # # Cases: latest -> stable, arch mapping (incl. extra SHA256SUMS lines, the @@ -37,7 +39,8 @@ # unavailable, each refused under its own name, never as a failed # verification, while every refusal - including one whose stderr claims a # network or auth problem - still reads "verification FAILED"), with gh's -# words stripped of control bytes and imported pipefail tolerated. +# words reduced to printable ASCII and imported pipefail tolerated; the +# API probe retried only on a 5xx or no response, at most three times. set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -63,6 +66,7 @@ CURL_LOG="$tmp/curl.log" GH_LOG="$tmp/gh.log" GH_VERIFY_LOG="$tmp/gh-verify.log" GH_API_LOG="$tmp/gh-api.log" +SLEEP_LOG="$tmp/sleep.log" ITMP="$tmp/itmp" mkdir -p "$WWW" "$SHIM" "$TOOLS" "$tmp/home" : >"$ATTEST_DB" @@ -172,14 +176,24 @@ cat >"$SHIM/gh" <<EOF # FAKE_GH_NET=down no HTTP response (proxy/DNS/connect failure), exit 1 # FAKE_GH_NET=503|429|403|403rl|403ra that HTTP status; 403rl carries # X-Ratelimit-Remaining: 0, 403ra a Retry-After header +# FAKE_GH_PROBE_SEQ what each API probe attempt gets, in turn (the last +# entry repeats): ok (the DB's answer), down, 401, 503, +# 429, 403rl, crash (exit 2, no stdout) or garbage +# (exit 1, stdout that is no status line); the verify +# calls are not affected +# FAKE_GH_PROTO=1.1 the API answers over HTTP/1.1 (default HTTP/2.0) # FAKE_GH_COLOR=1 header names coloured, as with CLICOLOR_FORCE -# FAKE_GH_ERR_CTRL=1 terminal control bytes in gh's error lines -# FAKE_GH_VERIFY_RC=4 verify exits 4 even though the API probe answered +# FAKE_GH_ERR_CTRL=1 C0 terminal control bytes in gh's error lines; =c1: +# C1 CSI bytes, raw (0x9b) and UTF-8 encoded (c2 9b) +# FAKE_GH_VERIFY_RC=4 verify exits 4 even though the API probe answered; +# =2 verify exits 2 (gh's "cancelled" code): no refusal, no auth # FAKE_GH_VERIFY_ERR text a failing verify prints instead of its error # FAKE_GH_ATTEST_SIGNER workflow the stored attestations were signed by +# More than 10 probes in one run is an unbounded retry: exit 4 to stop it. echo "\$*" >>"$GH_LOG" ctl="" cr="" -if [[ -n "\${FAKE_GH_ERR_CTRL:-}" ]]; then ctl=\$'\e]0;PWNED\a\e[31m' cr=\$'\r'; fi +if [[ "\${FAKE_GH_ERR_CTRL:-}" == c1 ]]; then ctl=\$'\x9b''31m'\$'\xc2\x9b''32m' +elif [[ -n "\${FAKE_GH_ERR_CTRL:-}" ]]; then ctl=\$'\e]0;PWNED\a\e[31m' cr=\$'\r'; fi if [[ -n "\${FAKE_GH_BROKEN:-}" ]]; then echo "\${ctl}failed to create root command: failed to read configuration: invalid config file /home/u/.config/gh/config.yml: invalid yaml\$cr" >&2 exit 1 @@ -198,11 +212,11 @@ login_hint() { printf 'To get started with GitHub CLI, please run: gh auth login\nAlternatively, populate the GH_TOKEN environment variable with a GitHub API authentication token.\n' >&2 exit 4 } -# hdrs <status line> [header...]: what gh api --include prints before the body. +# hdrs <status> <reason> [header...]: what gh api --include prints before the body. hdrs() { local h c="" r="" if [[ -n "\${FAKE_GH_COLOR:-}" ]]; then c=\$'\e[1;34m' r=\$'\e[m'; fi - printf '%s\n' "\$1"; shift + printf 'HTTP/%s %s %s\n' "\${FAKE_GH_PROTO:-2.0}" "\$1" "\$2"; shift 2 for h in "Content-Type: application/json; charset=utf-8" "\$@"; do printf '%s%s%s:%s\r\n' "\$c" "\${h%%:*}" "\$r" "\${h#*:}" done @@ -210,16 +224,32 @@ hdrs() { } if [[ "\${1:-}" == api ]]; then [[ \$# -eq 5 && "\$2 \$3 \$4" == "--method GET --include" ]] || { echo "gh shim: unexpected api call: \$*" >&2; exit 2; } - path="\$5" st="" + path="\$5" st="" n="\$(grep -c . "$GH_API_LOG" || true)" + if [[ \$n -ge 10 ]]; then echo "\$path stop" >>"$GH_API_LOG"; echo "gh shim: API probe made \$((n + 1)) times" >&2; exit 4; fi if [[ "\${FAKE_GH_AUTH:-}" == none ]]; then echo "\$path noauth" >>"$GH_API_LOG"; login_hint; fi - if [[ "\${FAKE_GH_AUTH:-}" == bad ]]; then st=401 + net="\${FAKE_GH_NET:-}" + if [[ -n "\${FAKE_GH_PROBE_SEQ:-}" ]]; then + read -r -a seq <<<"\$FAKE_GH_PROBE_SEQ" + [[ \$n -lt \${#seq[@]} ]] || n=\$((\${#seq[@]} - 1)) + net="\${seq[\$n]}"; [[ "\$net" != ok ]] || net="" + fi + if [[ "\${FAKE_GH_AUTH:-}" == bad || "\$net" == 401 ]]; then st=401 else - case "\${FAKE_GH_NET:-}" in + case "\$net" in down) echo "\$path -" >>"$GH_API_LOG" echo "\${ctl}Get \"https://api.github.com/\$path\": proxyconnect tcp: dial tcp 127.0.0.1:9: connect: connection refused\$cr" >&2 exit 1;; - 503|429|403|403rl|403ra) st="\$FAKE_GH_NET";; + crash) + echo "\$path crash" >>"$GH_API_LOG" + echo "panic: runtime error: invalid memory address or nil pointer dereference" >&2 + exit 2;; + garbage) + echo "\$path garbage" >>"$GH_API_LOG" + echo "<html>proxy login page</html>" + echo "gh: unexpected response" >&2 + exit 1;; + 5[0-9][0-9]|429|403|403rl|403ra) st="\$net";; "") d="\${path#repos/$REPO/attestations/sha256:}"; d="\${d%%\?*}" if [[ "\$path" == "repos/$REPO/attestations/sha256:\$d?per_page=1" && "\$d" =~ ^[0-9a-f]{64}\$ ]] && @@ -229,22 +259,23 @@ if [[ "\${1:-}" == api ]]; then fi echo "\$path \$st" >>"$GH_API_LOG" case "\$st" in - 200) hdrs "HTTP/2.0 200 OK" "X-Ratelimit-Remaining: 4999"; echo '{"attestations":[{"bundle":{}}]}'; exit 0;; - 404) hdrs "HTTP/2.0 404 Not Found" "X-Ratelimit-Remaining: 4998" + 200) hdrs 200 OK "X-Ratelimit-Remaining: 4999"; echo '{"attestations":[{"bundle":{}}]}'; exit 0;; + 404) hdrs 404 "Not Found" "X-Ratelimit-Remaining: 4998" echo '{"message":"Not Found","status":"404"}'; echo "gh: Not Found (HTTP 404)" >&2;; - 401) hdrs "HTTP/2.0 401 Unauthorized" + 401) hdrs 401 Unauthorized echo '{"message":"Bad credentials","status":"401"}'; echo "gh: Bad credentials (HTTP 401)" >&2;; - 503) hdrs "HTTP/2.0 503 Service Unavailable"; echo 'unicorn'; echo "\${ctl}gh: HTTP 503\$cr" >&2;; - 429) hdrs "HTTP/2.0 429 Too Many Requests" "Retry-After: 60" + 503) hdrs 503 "Service Unavailable"; echo 'unicorn'; echo "\${ctl}gh: HTTP 503\$cr" >&2;; + 5??) hdrs "\$st" "Bad Gateway"; echo '<html>bad gateway</html>'; echo "gh: HTTP \$st" >&2;; + 429) hdrs 429 "Too Many Requests" "Retry-After: 60" echo '{"message":"You have exceeded a secondary rate limit."}' echo "gh: You have exceeded a secondary rate limit. (HTTP 429)" >&2;; - 403rl) hdrs "HTTP/2.0 403 Forbidden" "X-Ratelimit-Remaining: 0" "X-Ratelimit-Reset: 1760000000" + 403rl) hdrs 403 Forbidden "X-Ratelimit-Remaining: 0" "X-Ratelimit-Reset: 1760000000" echo '{"message":"API rate limit exceeded for user ID 1."}' echo "gh: API rate limit exceeded for user ID 1. (HTTP 403)" >&2;; - 403ra) hdrs "HTTP/2.0 403 Forbidden" "Retry-After: 60" "X-Ratelimit-Remaining: 4000" + 403ra) hdrs 403 Forbidden "Retry-After: 60" "X-Ratelimit-Remaining: 4000" echo '{"message":"You have exceeded a secondary rate limit."}' echo "gh: You have exceeded a secondary rate limit. (HTTP 403)" >&2;; - 403) hdrs "HTTP/2.0 403 Forbidden" "X-Ratelimit-Remaining: 4000" + 403) hdrs 403 Forbidden "X-Ratelimit-Remaining: 4000" echo '{"message":"Resource not accessible by personal access token"}' echo "gh: Resource not accessible by personal access token (HTTP 403)" >&2;; esac @@ -284,6 +315,7 @@ while [[ \$# -gt 0 ]]; do esac done printf 'verify %s repo=%s signer=%s ref=%s\n' "\$file" "\$repo" "\$signer" "\$ref" >>"$GH_VERIFY_LOG" +if [[ "\${FAKE_GH_VERIFY_RC:-}" == 2 ]]; then echo "gh shim: verify ended with exit code 2" >&2; exit 2; fi # refuse <error>: how gh reports a failed verify when stdout is not a TTY. refuse() { if [[ -n "\${FAKE_GH_VERIFY_ERR:-}" ]]; then printf '%s\n' "\$FAKE_GH_VERIFY_ERR" >&2 @@ -306,6 +338,11 @@ sum="\$("$REAL_SHA" <"\$file" | cut -d' ' -f1)" if [[ -n "\$ref" ]]; then grep -qx "\$sum \$ref" "$ATTEST_DB" || refuse "expected SourceRepositoryRef to be \$ref" else grep -q "^\$sum " "$ATTEST_DB" || refuse "no attestations found"; fi EOF +cat >"$SHIM/sleep" <<EOF +#!/bin/sh +# Record the wait instead of waiting. +echo "\$*" >>"$SLEEP_LOG" +EOF cat >"$SHIM/uname" <<'EOF' #!/bin/sh case "$1" in -s) echo "${FAKE_OS:-Linux}";; -m) echo "${FAKE_ARCH:-x86_64}";; *) exit 1;; esac @@ -363,7 +400,8 @@ OUT="$tmp/out" RUN_SH=(sh) run() { local scen="$1" path="$2"; shift 2 - : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; : >"$GH_API_LOG"; rm -f "$tmp/sudo.log" + : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; : >"$GH_API_LOG"; : >"$SLEEP_LOG" + rm -f "$tmp/sudo.log" rm -rf "$ITMP"; mkdir -p "$ITMP" set +e env -i HOME="$tmp/home" TMPDIR="$ITMP" PATH="$path" FAKE_PORT="$FAKE_PORT" SCENARIO="$scen" "$@" \ @@ -374,6 +412,13 @@ run() { WITH_COSIGN="$SHIM:$TOOLS" GH_ONLY="$tmp/ghonly:$TOOLS" show() { sed 's/^/ | /' "$OUT" >&2; } +# A shell that runs its own sleep (busybox's applet) bypasses the sleep +# shim: then the retry cases can check the attempts but not the waits. +: >"$SLEEP_LOG" +env -i PATH="$GH_ONLY" sh -c 'sleep 0' || true +SLEEP_SHIMMED=0 +if [[ -s "$SLEEP_LOG" ]]; then SLEEP_SHIMMED=1 +else echo "[install-test] note: this sh does not run sleep from PATH; waits between probe attempts are not checked" >&2; fi # verified <PATH> <tag> <asset...>: on the gh-only PATH, gh verified exactly # these assets, once each, bound to the release workflow and refs/tags/<tag>; # with cosign on PATH, gh verified nothing. So a dropped, unbound or extra @@ -670,15 +715,26 @@ C4='GitHub attestation verification FAILED for' cp "$(rel_dir ghrefuse v0.4.0-rc.4)/$CLI_AMD64" "$(rel_dir ghrefuse v0.3.7)/$CLI_AMD64" printf '#!/bin/sh\necho "unattested build"\n' >"$(rel_dir ghnoatt v0.3.7)/$CLI_AMD64" for s in ghrefuse ghnoatt; do (cd "$(rel_dir "$s" v0.3.7)" && "$REAL_SHA" bpfcompat-* >SHA256SUMS); done -# probe_path <scenario>: the probe the installer must make for v0.3.7's CLI. +# probe_path <scenario> [tag] [cli asset]: the probe the installer must make +# for that release's CLI (default: v0.3.7's amd64 CLI). probe_path() { - echo "repos/$REPO/attestations/sha256:$("$REAL_SHA" <"$(rel_dir "$1" v0.3.7)/$CLI_AMD64" | cut -d' ' -f1)?per_page=1" + echo "repos/$REPO/attestations/sha256:$("$REAL_SHA" <"$(rel_dir "$1" "${2:-v0.3.7}")/${3:-$CLI_AMD64}" | cut -d' ' -f1)?per_page=1" +} +# probes <scenario> <status...>: the probe log of one attempt per status. +probes() { local s="$1" st; shift; for st in "$@"; do echo "$(probe_path "$s") $st"; done; } +# waited <status...>: the installer waited 3 s before every attempt but the +# first, and at no other time (unchecked when sh bypasses the sleep shim). +waited() { + local want="" i + for ((i = 1; i < $#; i++)); do want+="3"$'\n'; done + [[ $SLEEP_SHIMMED == 0 || "$(cat "$SLEEP_LOG")" == "${want%$'\n'}" ]] } itmp_clean() { [[ -d "$ITMP" && -z "$(ls -A "$ITMP")" ]]; } no_ctl() { [[ -z "$(LC_ALL=C tr -d '\n[:print:]' <"$OUT")" ]]; } -# gh_class <title> <scenario> <class 2|3|4> <text> <verify calls> <probe status> [VAR=value...] +# gh_class <title> <scenario> <class 2|3|4> <text> <verify calls> <probe statuses> [VAR=value...] # verify calls: how many `gh attestation verify` calls may have been made; -# probe status: what the one probe got ("-" = no HTTP response). +# probe statuses: what each probe attempt got, in order ("-" = no HTTP +# response); the waits between attempts are checked too. gh_class() { local title="$1" scen="$2" cls="$3" want="$4" nver="$5" pst="$6" d="$tmp/i-ghclass" bad="" n shift 6 @@ -696,7 +752,10 @@ gh_class() { ! grep -qF -- "$C2" "$OUT" && ! grep -qF -- "$C3" "$OUT" || bad+=" other-class-named";; esac no_ctl || bad+=" control-bytes" - [[ "$(cat "$GH_API_LOG")" == "$(probe_path "$scen") $pst" ]] || bad+=" probe($(tr '\n' '|' <"$GH_API_LOG"))" + # shellcheck disable=SC2086 # one probe attempt per word + [[ "$(cat "$GH_API_LOG")" == "$(probes "$scen" $pst)" ]] || bad+=" probe($(tr '\n' '|' <"$GH_API_LOG"))" + # shellcheck disable=SC2086 + waited $pst || bad+=" waits($(tr '\n' '|' <"$SLEEP_LOG"))" n="$(grep -c . "$GH_VERIFY_LOG" || true)" [[ "$n" == "$nver" ]] || bad+=" verify-calls=$n" if [[ -z "$bad" ]]; then ok "gh $title: class $cls, refused" @@ -710,8 +769,8 @@ gh_class "token rejected (HTTP 401)" ghapi 2 \ 0 401 FAKE_GH_AUTH=bad gh_class "API unreachable" ghapi 3 \ "$C3: no HTTP response (Get \"https://api.github.com/$(probe_path ghapi)\": proxyconnect tcp: dial tcp 127.0.0.1:9: connect: connection refused); check the network" \ - 0 - FAKE_GH_NET=down -gh_class "API HTTP 503" ghapi 3 "$C3: HTTP 503 (gh: HTTP 503)" 0 503 FAKE_GH_NET=503 + 0 "- - -" FAKE_GH_NET=down +gh_class "API HTTP 503" ghapi 3 "$C3: HTTP 503 (gh: HTTP 503)" 0 "503 503 503" FAKE_GH_NET=503 gh_class "API HTTP 429" ghapi 3 "$C3: HTTP 429, rate limited (gh: You have exceeded a secondary rate limit. (HTTP 429))" 0 429 FAKE_GH_NET=429 gh_class "API rate limit (403, X-Ratelimit-Remaining: 0, coloured headers)" ghapi 3 \ "$C3: HTTP 403, rate limited (gh: API rate limit exceeded for user ID 1. (HTTP 403))" 0 403rl FAKE_GH_NET=403rl FAKE_GH_COLOR=1 @@ -728,7 +787,7 @@ gh_class "refusal whose stderr claims missing auth" ghrefuse 4 "$C4 $CLI_AMD64" gh_class "verify exits 4 after the API answered" ghapi 2 \ "$C2: gh is not logged in (To get started with GitHub CLI, please run: gh auth login)" 1 200 FAKE_GH_VERIFY_RC=4 gh_class "API unreachable, control bytes in gh's error" ghapi 3 "$C3: no HTTP response (]0;PWNED[31mGet \"https://api.github.com/" \ - 0 - FAKE_GH_NET=down FAKE_GH_ERR_CTRL=1 + 0 "- - -" FAKE_GH_NET=down FAKE_GH_ERR_CTRL=1 # R2-N2: the "could not run 'gh --help'" line is stripped of control bytes too. d="$tmp/i-ghbroken-ctl" bad="" run ghcap "$GH_ONLY" FAKE_GH_BROKEN=1 FAKE_GH_ERR_CTRL=1 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" @@ -738,6 +797,18 @@ grep -qF "installed gh could not run 'gh --help' (]0;PWNED[31mfailed to create r no_ctl || bad+=" control-bytes" if [[ -z "$bad" ]]; then ok "gh that cannot run, control bytes in its error: stripped, refused" else fail "gh that cannot run with control bytes (rc=$rc):$bad"; show; fi +# N2: gh's words keep printable ASCII only, so C1 controls go too - a raw +# CSI byte (0x9b) and its UTF-8 form (c2 9b) - in both quoting places. +gh_class "API unreachable, C1 control bytes in gh's error" ghapi 3 "$C3: no HTTP response (31m32mGet \"https://api.github.com/" \ + 0 "- - -" FAKE_GH_NET=down FAKE_GH_ERR_CTRL=c1 +d="$tmp/i-ghbroken-c1" bad="" +run ghcap "$GH_ONLY" FAKE_GH_BROKEN=1 FAKE_GH_ERR_CTRL=c1 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" +[[ $rc -ne 0 && ! -e "$d" ]] && itmp_clean || bad+=" installed-or-rc0-or-tmpdir" +grep -qF "installed gh could not run 'gh --help' (31m32mfailed to create root command: failed to read configuration: invalid config file /home/u/.config/gh/config.yml: invalid yaml); fix gh" "$OUT" || + bad+=" reason-not-named" +no_ctl || bad+=" control-bytes" +if [[ -z "$bad" ]]; then ok "gh that cannot run, C1 control bytes in its error: stripped, refused" +else fail "gh that cannot run with C1 control bytes (rc=$rc):$bad"; show; fi # 10b. The probe on a good release: exactly one read-only GET of the CLI's # attestations, before the first verify call; with cosign present, gh @@ -749,6 +820,7 @@ verified "$GH_ONLY" v0.3.7 "$CLI_AMD64" "$VAL_AMD64" || bad+=" verify-calls" [[ "$(cat "$GH_API_LOG")" == "$(probe_path ghapi) 200" ]] || bad+=" probe" [[ "$(grep -nE '^(api |attestation verify [^-])' "$GH_LOG" | head -1)" == *":api --method GET --include $(probe_path ghapi)" ]] || bad+=" probe-not-first" +waited 200 || bad+=" waited" if [[ -z "$bad" ]]; then ok "gh path: one read-only API probe of the CLI digest, before the verify calls" else fail "gh path probe (rc=$rc):$bad"; show; sed 's/^/ gh | /' "$GH_LOG" >&2; fi for ghenv in FAKE_GH_AUTH=none FAKE_GH_AUTH=bad FAKE_GH_NET=down FAKE_GH_NET=503 FAKE_GH_NET=429 FAKE_GH_NET=403rl \ @@ -785,6 +857,59 @@ if [[ $rc -eq 0 && -x "$d/bin/bpfcompat" ]] && verified "$GH_ONLY" v0.3.7 "$CLI_ else fail "pipefail imported, good release (rc=$rc)"; show; fi RUN_SH=(sh) +# 10d. N7: a 5xx or no HTTP response at the probe may be transient (gh's +# own verify retries a 5xx), so the probe is made up to three times, +# 3 s apart, and the last attempt decides. Nothing else is retried: the +# single-attempt expectations in 10 pin 401, 404, 200, a plain 403, a +# 429, rate-limited 403s and gh's auth exit code 4. +# gh_installs <title> <scenario> <probe statuses> [VAR=value...]: the good +# v0.3.7 release still installs, verified, after exactly these attempts. +gh_installs() { + local title="$1" scen="$2" pst="$3" d="$tmp/i-ghok" bad="" + shift 3 + run "$scen" "$GH_ONLY" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" "$@" + [[ $rc -eq 0 && -x "$d/bin/bpfcompat" && -x "$d/libexec/bpfcompat-validator" ]] && itmp_clean || bad+=" not-installed-or-tmpdir" + grep -q 'GitHub build attestations verified' "$OUT" || bad+=" not-verified" + verified "$GH_ONLY" v0.3.7 "$CLI_AMD64" "$VAL_AMD64" || bad+=" verify-calls" + # shellcheck disable=SC2086 # one probe attempt per word + [[ "$(cat "$GH_API_LOG")" == "$(probes "$scen" $pst)" ]] || bad+=" probe($(tr '\n' '|' <"$GH_API_LOG"))" + # shellcheck disable=SC2086 + waited $pst || bad+=" waits($(tr '\n' '|' <"$SLEEP_LOG"))" + ! grep -qE "$C2|$C3|FAILED" "$OUT" || bad+=" refusal-wording" + if [[ -z "$bad" ]]; then ok "gh $title: installed" + else fail "gh $title (rc=$rc):$bad"; show; fi + rm -rf "$d" +} +gh_installs "probe HTTP 503, then 200" ghapi "503 200" FAKE_GH_PROBE_SEQ="503 ok" +gh_installs "probe HTTP 502, 503, then 200 on the third and last attempt" ghapi "502 503 200" FAKE_GH_PROBE_SEQ="502 503 ok" +gh_installs "probe gets no HTTP response, then 200" ghapi "- 200" FAKE_GH_PROBE_SEQ="down ok" +gh_class "probe HTTP 503, then 401" ghapi 2 \ + "$C2: GitHub rejected gh's credentials (HTTP 401) (gh: Bad credentials (HTTP 401)); run 'gh auth login'" 0 "503 401" \ + FAKE_GH_PROBE_SEQ="503 401" +gh_class "probe HTTP 503, then 429 (not retried)" ghapi 3 "$C3: HTTP 429, rate limited (gh: You have exceeded" 0 "503 429" \ + FAKE_GH_PROBE_SEQ="503 429" +gh_class "probe: no HTTP response, then 403 rate-limited (not retried)" ghapi 3 "$C3: HTTP 403, rate limited" 0 "- 403rl" \ + FAKE_GH_PROBE_SEQ="down 403rl" +gh_class "probe HTTP 503, then 404: the verify calls decide" ghnoatt 4 "$C4 $CLI_AMD64" 1 "503 404" FAKE_GH_PROBE_SEQ="503 ok" +gh_class "probe: no HTTP response, then 200, CLI attested for another tag" ghrefuse 4 "$C4 $CLI_AMD64" 1 "- 200" \ + FAKE_GH_PROBE_SEQ="down ok" + +# 10e. N1: ambiguous probe results go to the verify calls; a verify exit +# code that is neither a refusal (1) nor gh's auth code (4) is a +# failed verification; the status line is read for HTTP/1.1 too (as +# behind a TLS-intercepting proxy). +gh_installs "probe exits 2 with no output (no status line): the verify calls decide" ghapi crash FAKE_GH_PROBE_SEQ=crash +gh_class "probe exits 2 with no output, CLI attested for another tag" ghrefuse 4 "$C4 $CLI_AMD64" 1 crash FAKE_GH_PROBE_SEQ=crash +gh_installs "probe exits 1 with output that is no status line: the verify calls decide" ghapi garbage FAKE_GH_PROBE_SEQ=garbage +gh_class "verify exits 2 after the API answered" ghapi 4 "$C4 $CLI_AMD64" 1 200 FAKE_GH_VERIFY_RC=2 +gh_installs "API over HTTP/1.1" ghapi 200 FAKE_GH_PROTO=1.1 +gh_class "token rejected over HTTP/1.1" ghapi 2 \ + "$C2: GitHub rejected gh's credentials (HTTP 401) (gh: Bad credentials (HTTP 401)); run 'gh auth login'" 0 401 \ + FAKE_GH_AUTH=bad FAKE_GH_PROTO=1.1 +gh_class "HTTP/1.1 503, three times" ghapi 3 "$C3: HTTP 503 (gh: HTTP 503)" 0 "503 503 503" FAKE_GH_NET=503 FAKE_GH_PROTO=1.1 +gh_class "HTTP/1.1 403 rate-limited" ghapi 3 "$C3: HTTP 403, rate limited (gh: API rate limit exceeded" 0 403rl \ + FAKE_GH_NET=403rl FAKE_GH_PROTO=1.1 + if [[ $failures -ne 0 ]]; then echo "[install-test] $failures failure(s)" >&2 exit 1 From 4a720974eb4e9c4942f94df7906c71989581f2af Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 10:14:04 +0300 Subject: [PATCH 32/38] test(release): pin the exact promotion signer identity and ref anchors The promotion cases only used signatures from v0.4.0-rc.4 and v9.9.9-rogue, which no regexp derived from the v0.4.0-rc.5 identity matches. A regexp of the identity, even start-anchored, therefore passed every case, although it accepts a signature from a newly created tag such as v0.4.0-rc.50-rogue. - release-workflow-gates_test.sh: sign SHA256SUMS, the SBOM and the image, one site at a time, by four identities that are close to but not exactly this tag's release run: a tag extending the release tag, a tag differing only at a dot, the release tag in another repository, and another workflow on the release tag. For the blobs the run artifact and the draft agree, so the identity check is the only barrier. Each must be refused for the identity reason, and every identity must be used at all three sites. - verify-release-assets_test.sh: a tag ref under another ref and a tag ref with a trailing path are usage errors, which pins both anchors of the ref check. No workflow or script changes; the exact --certificate-identity already refuses every new case. Signed-off-by: ErenAri <erenari27@gmail.com> --- scripts/release-workflow-gates_test.sh | 60 +++++++++++++++++++++++++- scripts/verify-release-assets_test.sh | 6 +++ 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/scripts/release-workflow-gates_test.sh b/scripts/release-workflow-gates_test.sh index 18ebc0c..e0ae1e9 100755 --- a/scripts/release-workflow-gates_test.sh +++ b/scripts/release-workflow-gates_test.sh @@ -13,8 +13,10 @@ # re-verification and image steps run against stub cosign/gh/docker. The # draft's signature files and SBOM must be the candidate run's own # copies; every cosign identity is the exact -# release-artifacts.yml@refs/tags/<tag> (a signature made by another v* -# tag's run is refused); attestation checks are bound to the tag ref and +# release-artifacts.yml@refs/tags/<tag> of this repository (a signature +# made by another v* tag's run, by a tag whose name extends or resembles +# the release tag, by the tag in another repository or by another +# workflow is refused); attestation checks are bound to the tag ref and # the expected commit. # 3. Every workflow call of scripts/verify-release-assets.sh supplies the # source commit and tag ref that script requires. @@ -169,6 +171,21 @@ other_commit="$(printf 'e%.0s' {1..40})" digest="sha256:$(printf 'd%.0s' {1..64})" image=ghcr.io/kernel-guard/bpfcompat san="https://github.com/${repo}/.github/workflows/release-artifacts.yml@refs/tags/" +# Signer identities that are close to, but not exactly, this tag's release +# run. cosign matches an identity regexp unanchored (the stub's grep -E does +# too), so only the exact identity refuses all of them: a regexp of the +# identity accepts the first even when start-anchored, and the second even +# when anchored at both ends, because its dots are unescaped. +declare -A ROGUE_SIGNER=( + # A newly created tag whose name starts with the release tag (v0.4.0-rc.50-rogue). + [a-tag-extending-the-release-tag]="${san}${tag}0-rogue" + # A tag that differs from the release tag only where it has a dot (v0.4.0-rc-5). + [a-tag-differing-at-a-dot]="${san}${tag%.*}-${tag##*.}" + # The release tag's name in another repository's release workflow. + [the-tag-in-another-repository]="https://github.com/rogue/bpfcompat/.github/workflows/release-artifacts.yml@refs/tags/${tag}" + # Another workflow of this repository, run on the release tag. + [another-workflow-on-the-tag]="https://github.com/${repo}/.github/workflows/ci.yml@refs/tags/${tag}" +) work="$tmp/promote" mkdir -p "$work/stubs" @@ -387,6 +404,26 @@ tamper() { # $1 = case, $2 = case directory mv "$2/attest-db.new" "$2/attest-db" ;; tampered-sbom-bytes-control) sbom "$draft/bpfcompat.sbom.cdx.json" tampered ;; + run-checksums-signed-by-* | run-sbom-signed-by-* | image-signed-only-by-*) + # A ROGUE_SIGNER identity. For the blobs the run artifact and the draft + # agree, so the exact identity is the only barrier. + local who="${1#*-by-}" id + id="${ROGUE_SIGNER[$who]-}" + [[ -n "$id" ]] || { echo "unknown signer in case $1" >&2; exit 1; } + case "$1" in + run-checksums-*) + for f in "$run" "$draft"; do + mkcert "$f/SHA256SUMS.crt" "$id" rogue-sums + mksig "$f/SHA256SUMS.sig" "$f/SHA256SUMS" rogue-sums + done ;; + run-sbom-*) + for f in "$run" "$draft"; do + mkcert "$f/bpfcompat.sbom.cdx.crt" "$id" rogue-sbom + mksig "$f/bpfcompat.sbom.cdx.sig" "$f/bpfcompat.sbom.cdx.json" rogue-sbom + done ;; + image-*) + printf '%s@%s %s rogue-image\n' "$image" "$digest" "$id" >"$2/image-sigs" ;; + esac ;; *) echo "unknown case $1" >&2; exit 1 ;; esac } @@ -444,8 +481,27 @@ run-sbom-signed-by-another-tag refuse ID evidence-attested-by-another-tag-at-the-commit refuse ATT evidence-attested-for-the-tag-at-another-commit refuse ATT tampered-sbom-bytes-control refuse CMP_OR_SIG +run-checksums-signed-by-a-tag-extending-the-release-tag refuse ID +run-sbom-signed-by-a-tag-extending-the-release-tag refuse ID +image-signed-only-by-a-tag-extending-the-release-tag refuse ID +run-checksums-signed-by-a-tag-differing-at-a-dot refuse ID +run-sbom-signed-by-a-tag-differing-at-a-dot refuse ID +image-signed-only-by-a-tag-differing-at-a-dot refuse ID +run-checksums-signed-by-the-tag-in-another-repository refuse ID +run-sbom-signed-by-the-tag-in-another-repository refuse ID +image-signed-only-by-the-tag-in-another-repository refuse ID +run-checksums-signed-by-another-workflow-on-the-tag refuse ID +run-sbom-signed-by-another-workflow-on-the-tag refuse ID +image-signed-only-by-another-workflow-on-the-tag refuse ID CASES +# Every ROGUE_SIGNER identity is used at all three cosign sites above. +for who in "${!ROGUE_SIGNER[@]}"; do + for site in run-checksums-signed-by run-sbom-signed-by image-signed-only-by; do + [[ -d "$work/${site}-${who}" ]] || bad "no promotion case ${site}-${who}" + done +done + want="SHA=${commit} REF=refs/tags/${tag}" if [[ "$(cat "$work/untouched-draft/vra.log" 2>/dev/null)" == "$want" ]]; then ok "promotion verifies the draft assets for ${tag} at the expected commit" diff --git a/scripts/verify-release-assets_test.sh b/scripts/verify-release-assets_test.sh index 302a1e7..1c78f40 100755 --- a/scripts/verify-release-assets_test.sh +++ b/scripts/verify-release-assets_test.sh @@ -240,5 +240,11 @@ expect_usage_error "no release ref" "$ref_required" -u BPFCOMPAT_EXPECTED_SOURCE expect_usage_error "branch ref" "$ref_required" BPFCOMPAT_EXPECTED_SOURCE_REF=refs/heads/main expect_usage_error "bare tag name" "$ref_required" BPFCOMPAT_EXPECTED_SOURCE_REF=v0.4.0-rc.5 expect_usage_error "case-variant tag ref" "$ref_required" BPFCOMPAT_EXPECTED_SOURCE_REF=refs/tags/V0.4.0-rc.5 +# The whole ref must be one tag: not a tag-like path under another ref, and +# nothing after the tag name. +expect_usage_error "tag ref under another ref" "$ref_required" \ + BPFCOMPAT_EXPECTED_SOURCE_REF=refs/heads/refs/tags/v0.4.0-rc.5 +expect_usage_error "tag ref with a trailing path" "$ref_required" \ + BPFCOMPAT_EXPECTED_SOURCE_REF=refs/tags/v0.4.0-rc.5/x echo "[release-assets-test] PASS" From 5794ee48babe3dc1e41b6850605ec8c253b3485b Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 10:14:12 +0300 Subject: [PATCH 33/38] fix(action): bound the release tag resolution in time Every curl in the prebuilt step has --connect-timeout and --max-time, but the git ls-remote that resolves a tag pin to its commit had no bound: a stalled HTTPS connection would hold the consumer's job until the job timeout instead of taking the warned source build. Run it under timeout 60, the bound of the tags API call in the same step. An expired limit is an ls-remote failure, which already takes the warned source build; verification and the prebuilt install are unchanged. git's low-speed limit (GIT_HTTP_LOW_SPEED_*) was not used: it aborts a stalled transfer but not a stalled TLS handshake, which libcurl only ends at its 300 s connect timeout. The Action test records, for every ls-remote, the prompt setting and the command line of the process that ran git, and requires timeout(1) with a limit of at most 300 s, the step's longest curl bound. Signed-off-by: ErenAri <erenari27@gmail.com> --- action.yml | 7 ++++--- scripts/action-validator-pin_test.sh | 19 +++++++++++++++++++ 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/action.yml b/action.yml index 8528203..6fac46f 100644 --- a/action.yml +++ b/action.yml @@ -232,12 +232,13 @@ runs: fi # The commit the release was built from: a SHA pin names it; a tag # pin is resolved to the commit the tag points at (peeled when - # annotated). If that fails, build from source rather than install - # binaries that cannot be bound to this release. + # annotated). If that fails or does not finish within 60 s (the + # bound of the tags API call above), build from source rather than + # install binaries that cannot be bound to this release. source_sha="${ACTION_REF:-}" if [[ ! "$source_sha" =~ ^[0-9a-f]{40}$ ]]; then source_sha="" - if tag_refs="$(GIT_TERMINAL_PROMPT=0 git ls-remote https://github.com/Kernel-Guard/bpfcompat.git \ + if tag_refs="$(GIT_TERMINAL_PROMPT=0 timeout 60 git ls-remote https://github.com/Kernel-Guard/bpfcompat.git \ "refs/tags/${ref}" "refs/tags/${ref}^{}" 2>/dev/null)"; then source_sha="$(awk -v tag="refs/tags/${ref}" ' $2 == tag && !peeled { sha = $1 } diff --git a/scripts/action-validator-pin_test.sh b/scripts/action-validator-pin_test.sh index 2f671b7..6a3ff9d 100755 --- a/scripts/action-validator-pin_test.sh +++ b/scripts/action-validator-pin_test.sh @@ -180,12 +180,16 @@ done exit 0 """) git_log = os.path.join(case, "git_calls") + git_env_log = os.path.join(case, "git_ls_remote_env") ls_file = os.path.join(case, "ls_remote") with open(ls_file, "w") as fh: fh.write(ls_remote or "") write_exec(os.path.join(stubs, "git"), f"""#!/usr/bin/env bash printf '%s\\n' "$*" >>'{git_log}' if [[ "${{1:-}}" == ls-remote ]]; then + # The prompt setting, and the command line of the process that ran git. + printf 'PROMPT=%s PARENT=%s\\n' "${{GIT_TERMINAL_PROMPT-<unset>}}" \\ + "$(tr '\\0' ',' </proc/$PPID/cmdline 2>/dev/null)" >>'{git_env_log}' {"echo 'fatal: unable to access https://github.com/Kernel-Guard/bpfcompat.git/: Could not resolve host: github.com' >&2; exit 128" if ls_remote is None else ""} cat '{ls_file}' exit 0 @@ -217,6 +221,8 @@ exec '{REAL_GIT}' "$@" with open(verifier_log) as fh: LAST["verifier"] = dict(l.rstrip("\n").split("=", 1) for l in fh if "=" in l) LAST["git"] = open(git_log).read().splitlines() if os.path.exists(git_log) else [] + LAST["git_env"] = ([dict(kv.split("=", 1) for kv in line.split()) for line in open(git_env_log)] + if os.path.exists(git_env_log) else []) LAST["installed"] = os.path.exists(os.path.join(root, "bin", "bpfcompat")) return res.returncode, res.stdout + res.stderr, outputs, root @@ -365,6 +371,19 @@ check(got.get("REF") == "refs/tags/v9.9.9", "tag pin: verifier bound to refs/tag check(got.get("SHA") == TAG_COMMIT, "tag pin: verifier bound to the tag's peeled commit") check(any(c.startswith("ls-remote ") and "refs/tags/v9.9.9" in c for c in LAST["git"]), "tag pin: the tag is resolved with git ls-remote") +# The resolution cannot hang the job, as every curl in the step is +# time-bounded too: git never prompts for credentials, and it runs under +# timeout(1) with a limit of at most the step's longest curl bound (300 s). +# An expired limit is an ls-remote failure: the warned source build (9). +def bounded(env): + parent = env.get("PARENT", "").split(",") + if env.get("PROMPT") != "0" or len(parent) < 3 or os.path.basename(parent[0]) != "timeout": + return False + m = re.fullmatch(r"(\d+(?:\.\d+)?)([smhd]?)", parent[1]) + seconds = m and float(m.group(1)) * {"": 1, "s": 1, "m": 60, "h": 3600, "d": 86400}[m.group(2)] + return bool(seconds) and 0 < seconds <= 300 and os.path.basename(parent[2]) == "git" +check(len(LAST["git_env"]) >= 1 and all(bounded(e) for e in LAST["git_env"]), + f"tag pin: git ls-remote cannot prompt and runs under a time limit ({LAST['git_env']})") # 8b. A lightweight tag has no ^{} line: its own line is the commit. case = new_case() From 69cebeae6791d3c8f016cdb0a10ed3b68bce0778 Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 09:40:05 +0300 Subject: [PATCH 34/38] install.sh: bind gh-path installs to the release's attested names On the gh path SHA256SUMS is unsigned, and gh attestation verify matches an artifact by digest only. So a party able to change release assets could delete the validator or license-bundle line (a CLI-only install with a false "not published" note), or serve one genuine asset of the tag under another asset's name, and the install still passed. The CLI's verify call now also prints the subjects of its verified provenance (--format json --jq, as "<sha256> <name>" lines), which name every asset the tag's release run attested. The installer refuses, as a failed verification naming the mismatch, when SHA256SUMS and those subjects disagree on whether the validator or the license bundle exists, or when a file to be installed is not attested under its own name. That call's stderr is still kept for the auth classification. The cosign path is unchanged. Tests: the gh stub prints the subjects of every statement that holds the verified digest. Refused: the validator line deleted (v0.3.7 and rc.5 shapes), the bundle line deleted, the arm64 CLI served as the amd64 CLI, the CLI served as the validator, an arm64 validator line added on an aarch64 host, and bytes from another statement of the run served as the validator or as the CLI; each also refused by cosign. Installed: a CLI-only install, an aarch64 host and a re-run release with two statements. Signed-off-by: ErenAri <erenari27@gmail.com> --- scripts/install.sh | 29 ++++++- scripts/install_test.sh | 176 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 196 insertions(+), 9 deletions(-) diff --git a/scripts/install.sh b/scripts/install.sh index 4474d6d..817d6c1 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -124,7 +124,8 @@ install_validator=1 # The static C validator is published for amd64 only today. If this release has # no validator for the current arch, install the CLI without it rather than # failing - VM artifact validation then needs `make validator-static` locally or -# command mode. (SHA256SUMS is integrity-checked below regardless.) +# command mode. (SHA256SUMS is authenticated below regardless: by its cosign +# signature, or on the gh path against the release run's attested assets.) if [ "$install_validator" = "1" ]; then if grep -qE "^[0-9a-fA-F]{64} ${VALIDATOR_ASSET}\$" SHA256SUMS; then curl -fsSLO --retry 2 --connect-timeout 10 --max-time 300 \ @@ -244,9 +245,13 @@ elif have gh; then gh_err="$(first_line .gh-api.err)" || : die "GitHub API unavailable for attestation verification: $gh_down${gh_err:+ ($gh_err)}; check the network or proxy and https://www.githubstatus.com, then retry, or install cosign - refusing checksum-only installation" fi + # The CLI's call also prints its verified provenance's subjects: every + # asset the tag's release run attested, as "<sha256> <name>" lines. GH_REPO="$REPO" gh attestation verify "$CLI_ASSET" --repo "$REPO" \ --signer-workflow "$REPO/.github/workflows/release-artifacts.yml" \ - --source-ref "refs/tags/$VERSION" >.gh-verify.out 2>&1 || + --source-ref "refs/tags/$VERSION" --format json \ + --jq '.[].verificationResult.statement.subject[] | "\(.digest.sha256) \(.name)"' \ + >.gh-attested 2>.gh-verify.out || gh_refused "$CLI_ASSET" "$?" if [ "$install_validator" = "1" ]; then GH_REPO="$REPO" gh attestation verify "$VALIDATOR_ASSET" --repo "$REPO" \ @@ -260,6 +265,26 @@ elif have gh; then --source-ref "refs/tags/$VERSION" >.gh-verify.out 2>&1 || gh_refused "$LICENSE_ASSET" "$?" fi + # SHA256SUMS is unsigned on this path, and an attestation vouches for + # bytes, not for a file name. So SHA256SUMS must agree with the CLI's + # verified provenance on whether the release has a validator and a license + # bundle (it decides what is downloaded), and that provenance must list + # every file under the name it is installed as - not only as some other + # asset of the run. + for f in "$VALIDATOR_ASSET" "$LICENSE_ASSET"; do + gh_a=0; grep -qE "^[0-9a-f]{64} ${f}\$" .gh-attested && gh_a=1 + gh_s=0; grep -qE "^[0-9a-fA-F]{64} ${f}\$" SHA256SUMS && gh_s=1 + case "$gh_a$gh_s" in + 10) die "GitHub attestation verification FAILED for $f: SHA256SUMS does not list it, but the $VERSION release provenance of $CLI_ASSET does" ;; + 01) die "GitHub attestation verification FAILED for $f: SHA256SUMS lists it, but the $VERSION release provenance of $CLI_ASSET does not" ;; + esac + done + for f in $DL_FILES; do + gh_d="$($SHA256 <"$f" | sed -n '1s/^\([0-9a-fA-F]\{64\}\).*/\1/p')" || : + grep -qxF "$gh_d $f" .gh-attested && continue + gh_as="$(sed -n "s/^$gh_d //p" .gh-attested | sed -n '1p' | printable)" || : + die "GitHub attestation verification FAILED for $f: the $VERSION release provenance of $CLI_ASSET does not list these bytes under that name${gh_as:+ (it lists them as $gh_as)}" + done err "GitHub build attestations verified" else die "cryptographic verification requires cosign or GitHub CLI (gh); refusing checksum-only installation" diff --git a/scripts/install_test.sh b/scripts/install_test.sh index 2be0311..2eb0801 100644 --- a/scripts/install_test.sh +++ b/scripts/install_test.sh @@ -21,7 +21,9 @@ # from the attestation DB and records it, and can play a capable gh that # is not logged in, holds a rejected token, cannot reach the API or gets # a 5xx / 429 / rate-limited 403, modelled on real gh 2.68.0 / 2.102.0, -# one probe attempt at a time; +# one probe attempt at a time. For `--format json --jq <the installer's +# expression>` it prints, like real gh, the "<sha256> <name>" subjects +# of every attested statement that holds the verified digest; # * sleep only records how long the installer would have waited; # * sudo pretends to be root by unlocking the test's read-only tree. # @@ -40,7 +42,10 @@ # verification, while every refusal - including one whose stderr claims a # network or auth problem - still reads "verification FAILED"), with gh's # words reduced to printable ASCII and imported pipefail tolerated; the -# API probe retried only on a 5xx or no response, at most three times. +# API probe retried only on a 5xx or no response, at most three times; and +# on the gh path, SHA256SUMS held to the CLI's attested provenance (it must +# agree on whether a validator and a license bundle exist, and each file is +# installed only under the name its bytes were attested as). set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" @@ -62,14 +67,19 @@ WWW="$tmp/www" SHIM="$tmp/shim" TOOLS="$tmp/tools" ATTEST_DB="$tmp/attestations" +# One line per attested subject: "<statement> refs/tags/<tag> <sha256> <name>". +SUBJ_DB="$tmp/subjects" CURL_LOG="$tmp/curl.log" GH_LOG="$tmp/gh.log" GH_VERIFY_LOG="$tmp/gh-verify.log" GH_API_LOG="$tmp/gh-api.log" +GH_FMT_LOG="$tmp/gh-format.log" SLEEP_LOG="$tmp/sleep.log" ITMP="$tmp/itmp" +# The installer's request for the CLI's attested subjects (gh --jq). +JQ_SUBJECTS='.[].verificationResult.statement.subject[] | "\(.digest.sha256) \(.name)"' mkdir -p "$WWW" "$SHIM" "$TOOLS" "$tmp/home" -: >"$ATTEST_DB" +: >"$ATTEST_DB"; : >"$SUBJ_DB" failures=0 fail() { echo "[install-test] FAIL: $*" >&2; failures=$((failures + 1)); } @@ -78,7 +88,8 @@ ok() { echo "[install-test] ok: $*"; } # ---------------------------------------------------------------- fake release # release <scenario> <tag> [validator-arches] [bundle]: builds a signed, # attested release under the scenario's fake github.com tree, the way -# release-artifacts.yml does. bundle: "yes" adds the third-party license +# release-artifacts.yml does: one provenance statement ("<scenario>-<tag>") +# names every attested asset. bundle: "yes" adds the third-party license # bundle; "unattested" adds it to SHA256SUMS without an attestation. release() { local scen="$1" tag="$2" varches="${3:-amd64}" bundle="${4:-no}" d a b @@ -102,6 +113,7 @@ release() { for a in "$d"/bpfcompat-*; do [[ "$bundle" == unattested && "$a" == *third-party-licenses* ]] && continue printf '%s refs/tags/%s\n' "$("$REAL_SHA" <"$a" | cut -d' ' -f1)" "$tag" >>"$ATTEST_DB" + printf '%s-%s refs/tags/%s %s %s\n' "$scen" "$tag" "$tag" "$("$REAL_SHA" <"$a" | cut -d' ' -f1)" "${a##*/}" >>"$SUBJ_DB" done printf 'sig:%s\n' "$("$REAL_SHA" <"$d/SHA256SUMS" | cut -d' ' -f1)" >"$d/SHA256SUMS.sig" printf '%s@refs/tags/%s\n' "$WF" "$tag" >"$d/SHA256SUMS.crt" @@ -303,18 +315,21 @@ if [[ "\${1:-} \${2:-} \${3:-}" == "attestation verify --help" ]]; then fi [[ "\${1:-} \${2:-}" == "attestation verify" ]] || exit 1 shift 2 -file="" repo="" signer="" ref="" +file="" repo="" signer="" ref="" fmt="" jq="" fmtset="" while [[ \$# -gt 0 ]]; do if [[ "\$1" == --* && "\$lacks" == *" \$1 "* ]]; then echo "unknown flag: \$1" >&2; exit 1; fi case "\$1" in --repo) repo="\$2"; shift 2;; --signer-workflow) signer="\$2"; shift 2;; --source-ref) ref="\$2"; shift 2;; + --format) fmt="\$2" fmtset=1; shift 2;; + --jq) jq="\$2" fmtset=1; shift 2;; -*) echo "gh shim: unknown flag \$1" >&2; exit 2;; *) file="\$1"; shift;; esac done printf 'verify %s repo=%s signer=%s ref=%s\n' "\$file" "\$repo" "\$signer" "\$ref" >>"$GH_VERIFY_LOG" +[[ -z "\$fmtset" ]] || printf '%s fmt=%s jq=%s\n' "\$file" "\$fmt" "\$jq" >>"$GH_FMT_LOG" if [[ "\${FAKE_GH_VERIFY_RC:-}" == 2 ]]; then echo "gh shim: verify ended with exit code 2" >&2; exit 2; fi # refuse <error>: how gh reports a failed verify when stdout is not a TTY. refuse() { @@ -337,6 +352,17 @@ esac sum="\$("$REAL_SHA" <"\$file" | cut -d' ' -f1)" if [[ -n "\$ref" ]]; then grep -qx "\$sum \$ref" "$ATTEST_DB" || refuse "expected SourceRepositoryRef to be \$ref" else grep -q "^\$sum " "$ATTEST_DB" || refuse "no attestations found"; fi +[[ -n "\$fmtset" ]] || exit 0 +[[ "\$fmt" == json && "\$jq" == '$JQ_SUBJECTS' ]] || + { echo "gh shim: unexpected --format/--jq: \$fmt \$jq" >&2; exit 2; } +# Like real gh: the subjects of every verified statement (one whose ref +# passed) that holds this digest, as "<sha256> <name>" lines. +seen=" " +while read -r st sref sdig _; do + [[ "\$sdig" == "\$sum" && ( -z "\$ref" || "\$sref" == "\$ref" ) && "\$seen" != *" \$st "* ]] || continue + seen+="\$st " + while read -r s2 _ d2 n2; do [[ "\$s2" != "\$st" ]] || printf '%s %s\n' "\$d2" "\$n2"; done <"$SUBJ_DB" +done <"$SUBJ_DB" EOF cat >"$SHIM/sleep" <<EOF #!/bin/sh @@ -400,7 +426,7 @@ OUT="$tmp/out" RUN_SH=(sh) run() { local scen="$1" path="$2"; shift 2 - : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; : >"$GH_API_LOG"; : >"$SLEEP_LOG" + : >"$CURL_LOG"; : >"$GH_LOG"; : >"$GH_VERIFY_LOG"; : >"$GH_API_LOG"; : >"$GH_FMT_LOG"; : >"$SLEEP_LOG" rm -f "$tmp/sudo.log" rm -rf "$ITMP"; mkdir -p "$ITMP" set +e @@ -445,6 +471,11 @@ for s in stable arm64 nosha noverify mismatch bindcosign bindgh bindghcli bindgh release "$s" v0.4.0-rc.4 amd64 latest "$s" v0.3.7 done +for s in rbval037 rbcli rbval rbarm rbsep rbsepcli rbrerun; do + release "$s" v0.3.7 amd64 + latest "$s" v0.3.7 +done +for s in rbval5 rblic5; do release "$s" v0.4.0-rc.5 amd64 yes; done for s in lic licbad licmissing licunattested; do release "$s" v0.3.7 amd64 latest "$s" v0.3.7 @@ -637,7 +668,10 @@ if [[ $rc -ne 0 && ! -e "$d" ]] && grep -q 'download failed: bpfcompat-third-par else fail "missing license bundle (rc=$rc)"; show; fi d="$tmp/i-licunattested" run licunattested "$GH_ONLY" BPFCOMPAT_VERSION=v0.4.0-rc.5 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" -if [[ $rc -ne 0 && ! -e "$d" ]] && grep -q 'attestation verification FAILED for bpfcompat-third-party-licenses.tar.gz' "$OUT"; then +# The bundle's own verify call refuses it (the whole line), before the +# comparison with the CLI's attested subjects in 11 would. +if [[ $rc -ne 0 && ! -e "$d" ]] && grep -q 'attestation verification FAILED for bpfcompat-third-party-licenses.tar.gz' "$OUT" && + grep -qx 'bpfcompat-install: GitHub attestation verification FAILED for bpfcompat-third-party-licenses.tar.gz' "$OUT"; then ok "unattested license bundle refused on the gh path" else fail "unattested license bundle (rc=$rc)"; show; fi if [[ "$(id -u)" -ne 0 ]]; then @@ -820,6 +854,8 @@ verified "$GH_ONLY" v0.3.7 "$CLI_AMD64" "$VAL_AMD64" || bad+=" verify-calls" [[ "$(cat "$GH_API_LOG")" == "$(probe_path ghapi) 200" ]] || bad+=" probe" [[ "$(grep -nE '^(api |attestation verify [^-])' "$GH_LOG" | head -1)" == *":api --method GET --include $(probe_path ghapi)" ]] || bad+=" probe-not-first" +# Only the CLI's verify call asks for its attested subjects (T4 R-B). +[[ "$(cat "$GH_FMT_LOG")" == "$CLI_AMD64 fmt=json jq=$JQ_SUBJECTS" ]] || bad+=" subjects-request($(tr '\n' '|' <"$GH_FMT_LOG"))" waited 200 || bad+=" waited" if [[ -z "$bad" ]]; then ok "gh path: one read-only API probe of the CLI digest, before the verify calls" else fail "gh path probe (rc=$rc):$bad"; show; sed 's/^/ gh | /' "$GH_LOG" >&2; fi @@ -910,6 +946,132 @@ gh_class "HTTP/1.1 503, three times" ghapi 3 "$C3: HTTP 503 (gh: HTTP 503)" 0 "5 gh_class "HTTP/1.1 403 rate-limited" ghapi 3 "$C3: HTTP 403, rate limited (gh: API rate limit exceeded" 0 403rl \ FAKE_GH_NET=403rl FAKE_GH_PROTO=1.1 +# 11. T4 (remedy R-B): on the gh path SHA256SUMS is unsigned, and an +# attestation binds bytes, not a file name. The CLI's verified provenance +# names every asset its release run attested, so SHA256SUMS must agree +# with it on whether the validator and the license bundle exist, and each +# installed file must be attested under the name it is installed as. +# Every tamper below passes each gh verify call (genuine bytes of the +# requested tag) and the checksum step: only that comparison can catch +# it. It reads as a failed verification (class 4) naming the mismatch; +# with cosign, the edited SHA256SUMS fails its signature. +C4_NOT='authenticat|logged in|credential|GH_TOKEN|network|proxy|unavailable|unreachable|rate.limit|refused|retry|HTTP [0-9]|lacks' +resum() { (cd "$1" && "$REAL_SHA" bpfcompat-* >SHA256SUMS); } +# rb_refused <title> <scenario> <tag> <text after "FAILED for "> <verified assets> [VAR=value...] +rb_refused() { + local title="$1" scen="$2" tag="$3" want="$4" assets="$5" d="$tmp/i-rb" bad="" cli ghrc + shift 5 + cli="${assets%% *}" + run "$scen" "$GH_ONLY" BPFCOMPAT_VERSION="$tag" BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" "$@" + ghrc=$rc + [[ $rc -ne 0 && ! -e "$d" ]] || bad+=" installed-or-rc0" + itmp_clean || bad+=" tmpdir-left" + grep -qxF -- "bpfcompat-install: $C4 $want" "$OUT" || bad+=" reason-not-named" + ! grep -qiE "$C4_NOT" "$OUT" || bad+=" auth-or-network-wording" + ! grep -qE "$C2|$C3|attestations verified|checksums verified" "$OUT" || bad+=" other-outcome-named" + no_ctl || bad+=" control-bytes" + # shellcheck disable=SC2086 # one asset per word + verified "$GH_ONLY" "$tag" $assets || bad+=" verify-calls" + [[ "$(cat "$GH_API_LOG")" == "$(probe_path "$scen" "$tag" "$cli") 200" ]] || bad+=" probe" + [[ "$(cat "$GH_FMT_LOG")" == "$cli fmt=json jq=$JQ_SUBJECTS" ]] || bad+=" subjects-request" + [[ -z "$bad" ]] || show + rm -rf "$d" + run "$scen" "$WITH_COSIGN" BPFCOMPAT_VERSION="$tag" BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" "$@" + if ! { [[ $rc -ne 0 && ! -e "$d" && ! -s "$GH_LOG" ]] && itmp_clean && + grep -q 'cosign signature verification FAILED for SHA256SUMS' "$OUT"; }; then bad+=" cosign-path"; show; fi + rm -rf "$d" + if [[ -z "$bad" ]]; then ok "gh: $title: refused as a failed verification (cosign: bad signature)" + else fail "gh: $title (gh rc=$ghrc, cosign rc=$rc):$bad"; fi +} +# rb_installs <title> <scenario> <tag> <verified assets> [VAR=value...]: an +# untampered release passes the comparison; the installed bytes are the +# verified ones. +rb_installs() { + local title="$1" scen="$2" tag="$3" assets="$4" d="$tmp/i-rbok" bad="" a + shift 4 + run "$scen" "$GH_ONLY" BPFCOMPAT_VERSION="$tag" BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" "$@" + [[ $rc -eq 0 ]] && itmp_clean && grep -q 'GitHub build attestations verified' "$OUT" || bad+=" not-installed" + # shellcheck disable=SC2086 # one asset per word + verified "$GH_ONLY" "$tag" $assets || bad+=" verify-calls" + [[ "$(cat "$GH_FMT_LOG")" == "${assets%% *} fmt=json jq=$JQ_SUBJECTS" ]] || bad+=" subjects-request" + for a in $assets; do + case "$a" in + bpfcompat-linux-*) cmp -s "$(rel_dir "$scen" "$tag")/$a" "$d/bin/bpfcompat" || bad+=" cli-bytes" ;; + bpfcompat-validator-*) cmp -s "$(rel_dir "$scen" "$tag")/$a" "$d/libexec/bpfcompat-validator" || bad+=" validator-bytes" ;; + esac + done + [[ "$assets" == *validator* || ! -e "$d/libexec" ]] || bad+=" unexpected-validator" + if [[ -z "$bad" ]]; then ok "gh: $title: installed" + else fail "gh: $title (rc=$rc):$bad"; show; fi + rm -rf "$d" +} +# S1: the validator line deleted from SHA256SUMS (v0.3.7 and rc.5 shapes). +sed -i "/ $VAL_AMD64\$/d" "$(rel_dir rbval037 v0.3.7)/SHA256SUMS" "$(rel_dir rbval5 v0.4.0-rc.5)/SHA256SUMS" +rb_refused "validator line deleted from SHA256SUMS (v0.3.7)" rbval037 v0.3.7 \ + "$VAL_AMD64: SHA256SUMS does not list it, but the v0.3.7 release provenance of $CLI_AMD64 does" "$CLI_AMD64" +rb_refused "validator line deleted from SHA256SUMS (v0.4.0-rc.5, with the license bundle)" rbval5 v0.4.0-rc.5 \ + "$VAL_AMD64: SHA256SUMS does not list it, but the v0.4.0-rc.5 release provenance of $CLI_AMD64 does" \ + "$CLI_AMD64 bpfcompat-third-party-licenses.tar.gz" +# S2: the license bundle line deleted (rc.5). +sed -i '/ bpfcompat-third-party-licenses\.tar\.gz$/d' "$(rel_dir rblic5 v0.4.0-rc.5)/SHA256SUMS" +rb_refused "license bundle line deleted from SHA256SUMS" rblic5 v0.4.0-rc.5 \ + "bpfcompat-third-party-licenses.tar.gz: SHA256SUMS does not list it, but the v0.4.0-rc.5 release provenance of $CLI_AMD64 does" \ + "$CLI_AMD64 $VAL_AMD64" +# S3: the same tag's arm64 CLI served as the amd64 CLI, its line rewritten. +cp "$(rel_dir rbcli v0.3.7)/bpfcompat-linux-arm64" "$(rel_dir rbcli v0.3.7)/$CLI_AMD64"; resum "$(rel_dir rbcli v0.3.7)" +rb_refused "the tag's arm64 CLI served as the amd64 CLI" rbcli v0.3.7 \ + "$CLI_AMD64: the v0.3.7 release provenance of $CLI_AMD64 does not list these bytes under that name (it lists them as bpfcompat-linux-arm64)" \ + "$CLI_AMD64 $VAL_AMD64" +# S4: the CLI's bytes served as the validator. +cp "$(rel_dir rbval v0.3.7)/$CLI_AMD64" "$(rel_dir rbval v0.3.7)/$VAL_AMD64"; resum "$(rel_dir rbval v0.3.7)" +rb_refused "the CLI's bytes served as the validator" rbval v0.3.7 \ + "$VAL_AMD64: the v0.3.7 release provenance of $CLI_AMD64 does not list these bytes under that name (it lists them as $CLI_AMD64)" \ + "$CLI_AMD64 $VAL_AMD64" +# T4-VERIFY's arm64-host variant: no arm64 validator is published, so +# SHA256SUMS gains a line for one, served with the amd64 validator's bytes. +cp "$(rel_dir rbarm v0.3.7)/$VAL_AMD64" "$(rel_dir rbarm v0.3.7)/bpfcompat-validator-static-linux-arm64" +resum "$(rel_dir rbarm v0.3.7)" +rb_refused "aarch64 host, an arm64 validator added to SHA256SUMS (amd64 bytes)" rbarm v0.3.7 \ + "bpfcompat-validator-static-linux-arm64: SHA256SUMS lists it, but the v0.3.7 release provenance of bpfcompat-linux-arm64 does not" \ + "bpfcompat-linux-arm64 bpfcompat-validator-static-linux-arm64" FAKE_ARCH=aarch64 +# Bytes the same run attested in another statement (release-candidate +# evidence) served as the validator: not in the CLI's provenance at all. +printf '{"tag": "v0.3.7", "checksums_sha256": "x"}\n' >"$tmp/rbsep-evidence.json" +ev="$("$REAL_SHA" <"$tmp/rbsep-evidence.json" | cut -d' ' -f1)" +printf '%s refs/tags/v0.3.7\n' "$ev" >>"$ATTEST_DB" +printf 'rbsep-v0.3.7-evidence refs/tags/v0.3.7 %s release-candidate-evidence.json\n' "$ev" >>"$SUBJ_DB" +cp "$tmp/rbsep-evidence.json" "$(rel_dir rbsep v0.3.7)/$VAL_AMD64"; resum "$(rel_dir rbsep v0.3.7)" +rb_refused "bytes from another statement of the same run served as the validator" rbsep v0.3.7 \ + "$VAL_AMD64: the v0.3.7 release provenance of $CLI_AMD64 does not list these bytes under that name" "$CLI_AMD64 $VAL_AMD64" +# The same bytes served as the CLI: the provenance verified for the CLI is +# then that other statement, which names no validator. +cp "$tmp/rbsep-evidence.json" "$(rel_dir rbsepcli v0.3.7)/$CLI_AMD64"; resum "$(rel_dir rbsepcli v0.3.7)" +rb_refused "bytes from another statement of the same run served as the CLI" rbsepcli v0.3.7 \ + "$VAL_AMD64: SHA256SUMS lists it, but the v0.3.7 release provenance of $CLI_AMD64 does not" "$CLI_AMD64 $VAL_AMD64" +# Controls: untampered releases still install on the gh path (the cases in +# 1, 1b, 2 and 8 are controls too). +rb_installs "untampered v0.3.7 with BPFCOMPAT_NO_VALIDATOR=1" ghapi v0.3.7 "$CLI_AMD64" BPFCOMPAT_NO_VALIDATOR=1 +rb_installs "aarch64 host, untampered: arm64 CLI only" arm64 v0.3.7 bpfcompat-linux-arm64 FAKE_ARCH=aarch64 +# A re-run release job: a second statement for the same tag holds the same +# CLI and a rebuilt validator, which is the one published. +v2="$(rel_dir rbrerun v0.3.7)/$VAL_AMD64" +printf 'validator v0.3.7 amd64, rebuilt by attempt 2\n' >"$v2" +for a in "$CLI_AMD64" bpfcompat-linux-arm64 "$VAL_AMD64"; do + printf 'rbrerun-v0.3.7-attempt2 refs/tags/v0.3.7 %s %s\n' "$("$REAL_SHA" <"$(rel_dir rbrerun v0.3.7)/$a" | cut -d' ' -f1)" "$a" >>"$SUBJ_DB" +done +printf '%s refs/tags/v0.3.7\n' "$("$REAL_SHA" <"$v2" | cut -d' ' -f1)" >>"$ATTEST_DB" +resum "$(rel_dir rbrerun v0.3.7)" +rb_installs "re-run release, validator from the second attested statement" rbrerun v0.3.7 "$CLI_AMD64 $VAL_AMD64" +# With pipefail imported, the name comparison still refuses with its reason. +RUN_SH=(bash --posix) +d="$tmp/i-rb-pipefail" +run rbcli "$GH_ONLY" "$PF" BPFCOMPAT_VERSION=v0.3.7 BPFCOMPAT_BIN_DIR="$d/bin" BPFCOMPAT_LIBEXEC_DIR="$d/libexec" +if [[ $rc -ne 0 && ! -e "$d" ]] && itmp_clean && + grep -qxF "bpfcompat-install: $C4 $CLI_AMD64: the v0.3.7 release provenance of $CLI_AMD64 does not list these bytes under that name (it lists them as bpfcompat-linux-arm64)" "$OUT"; then + ok "pipefail imported, arm64 CLI served as amd64: refused with its reason" +else fail "pipefail imported, arm64 CLI served as amd64 (rc=$rc)"; show; fi +RUN_SH=(sh) + if [[ $failures -ne 0 ]]; then echo "[install-test] $failures failure(s)" >&2 exit 1 From fdae550d4d24c920f19e7f6f3a26290bcdc184d1 Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 21:20:47 +0300 Subject: [PATCH 35/38] readiness: look up the canary runs, their attempts and the release Written policy says a canary is a run of the temporary release-candidate-canary workflow, the timed ones from its dated schedule, and that its plan job admits T+24h only 20-30 and T+72h only 68-78 whole hours after publication. Readiness checked only fields the canary evidence reports about itself and made no run lookup. A ci.yml, branch, fork, dispatched, failed or nonexistent run was accepted, and a partial re-run of a failed in-window run, which keeps the plan job's earlier decision, could write evidence long after the window closed (F6). With attestation verification on, readiness now reads publication (T0) from the candidate release, which must be published, not a draft, and not before generated_at. For each canary it reads the run, its attempts and the chosen attempt's jobs: - the run is a successful release-candidate-canary.yml run of this repository from workflow_dispatch (manual) or schedule (timed, on main); - the evidence's completed_at falls inside a successful attempt, in which the "Assemble RC canary evidence" job itself ran and succeeded; - that attempt started at or after T0, and a timed attempt started before its window closed (T0+31h, T0+79h); a first attempt, which ran its own plan job, also completed after the window opened (T0+20h, T0+68h). The attempt, its start, the run's branch and T0 are recorded in the JSON and Markdown. No written rule names the manual dispatch's branch, so it is recorded, not enforced. A re-run started inside the window is accepted, as the windows are written; whether re-runs should count stays an open operator decision. The bypass mode is unchanged. The test's gh stub answers the new lookups in this same commit, because the test runs in ci.yml and in the release quality gates. The runbook no longer says the plan job alone bounds re-runs and describes the new checks. Signed-off-by: ErenAri <erenari27@gmail.com> --- docs/production-release-process.md | 33 ++- scripts/production-readiness-report.sh | 132 +++++++++- scripts/production-readiness-report_test.sh | 274 +++++++++++++++++++- 3 files changed, 425 insertions(+), 14 deletions(-) diff --git a/docs/production-release-process.md b/docs/production-release-process.md index 2c924dc..febc55f 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -67,8 +67,12 @@ digest, and the external Inspektor Gadget/KubeArmor/Falco workflow dispatched at the immutable RC tag. Run it once immediately after publication. Its two dated schedules run just after T+24 hours and T+72 hours and emit `[bpfcompat-rc-canary:v1]` evidence; any out-of-window invocation exits after -the planning job. Remove or retarget these temporary schedules after -graduation. +the planning job. A partial re-run ("Re-run failed jobs", or re-running one +job) is not a new invocation: it does not run the planning job again, keeps +the first attempt's window decision, and can write evidence after the window +has closed. The planning job alone therefore does not bound re-runs; the +readiness report checks each canary run's attempts (see **Canaries** below). +Remove or retarget these temporary schedules after graduation. Every job in `release-candidate-canary`, `external-consumer-canary`, and `consumer-canary` runs on the pinned `ubuntu-24.04` image, not the moving @@ -181,9 +185,25 @@ aggregator binds every piece of evidence to the attested candidate every canary must bind the same RC tag, commit, and image digest as the attested candidate. Completion times must be in milestone order and at least 0, 20, and 68 hours after `generated_at`. These are floors derived from the - plan job's 20-30h and 68-78h windows after publication: publication time is - not in the attested evidence, but it never precedes `generated_at`. The plan - job itself enforces the upper bounds. Each canary's components must be + plan job's 20-30h and 68-78h windows after publication (T0): publication time + is not in the attested evidence, but it never precedes `generated_at`. With + attestation verification on, T0 is the release's `published_at`; the release + must be published, not a draft, and not before `generated_at`. Each canary's + `workflow_run_id` must be a successful `release-candidate-canary.yml` run of + this repository from `workflow_dispatch` (manual) or `schedule` (timed, on + `main`). Its `completed_at` must fall inside a successful attempt of that + run in which the `Assemble RC canary evidence` job itself ran and succeeded, + and that attempt must have started at or after T0. A timed canary's attempt + must have started before its window closed (T0+31h or T0+79h), and a first + attempt, which ran its own plan job, must have completed after the window + opened (T0+20h or T0+68h). The output records each canary's + `workflow_run_attempt`, `attempt_started_at` and `head_branch`, and the + release's `published_at`; no written rule names the manual dispatch's + branch, so it is recorded, not enforced. A re-run started inside the window + is accepted, as the windows are written. Whether a re-run should count is an + open operator decision: counting first attempts only would tighten this, and + counting a re-run started after the window closed would relax the windows + and must be written here first. Each canary's components must be artifacts of that canary's own run (`*canary-component-<check>-<run ID>/`), and the `<check>.json` record of each of the five checks must be present, match its recorded SHA-256, pass, and name the candidate tag and commit; the @@ -228,7 +248,8 @@ The command writes both Markdown and JSON. Add the reviewed outputs to request. The stable release workflow validates, checksums, attests, and publishes those files; a stable tag fails if they are absent or not `ready`. The online candidate check requires an authenticated GitHub CLI version that -provides `gh attestation`, and the drill and incident runs are read through the +provides `gh attestation`, and the candidate release, the canary runs with +their attempts and jobs, and the drill and incident runs are read through the GitHub API; an older or unauthenticated CLI, or no network, fails closed. Do not use `BPFCOMPAT_SKIP_READINESS_ATTESTATION=1` outside the regression diff --git a/scripts/production-readiness-report.sh b/scripts/production-readiness-report.sh index 6b47f08..f832007 100755 --- a/scripts/production-readiness-report.sh +++ b/scripts/production-readiness-report.sh @@ -377,6 +377,21 @@ fi : >"$tmp/canaries.ndjson" : >"$tmp/canary_run_ids" previous_canary_epoch=0 +# The canary plan job measures its windows from publication (T0), which the +# canary designation records as CANDIDATE_PUBLISHED_AT. With attestation +# verification on, T0 is read from the release itself: it must be published, +# not a draft, and cannot precede the candidate evidence it publishes. +candidate_published_at="" +if [[ "${BPFCOMPAT_SKIP_READINESS_ATTESTATION:-0}" != "1" ]]; then + candidate_release="$(gh api "repos/Kernel-Guard/bpfcompat/releases/tags/${candidate_tag}" 2>/dev/null)" || + fail "release ${candidate_tag} cannot be read from GitHub" + jq -e '.draft == false' <<<"$candidate_release" >/dev/null || + fail "release ${candidate_tag} is a draft, not a published release" + candidate_published_at="$(jq -r '.published_at // ""' <<<"$candidate_release")" + candidate_published_epoch="$(iso_epoch "$candidate_published_at" "release ${candidate_tag} published_at")" + [[ "$candidate_published_epoch" -ge "$candidate_generated_epoch" ]] || + fail "release ${candidate_tag} was published at ${candidate_published_at}, before its candidate evidence was generated" +fi while IFS= read -r canary; do milestone="$(jq -r '.milestone' <<<"$canary")" evidence_rel="$(jq -r '.evidence' <<<"$canary")" @@ -426,7 +441,8 @@ while IFS= read -r canary; do # hours, and T+72h only 68-78 whole hours, after CANDIDATE_PUBLISHED_AT. # Publication time is not part of the attested candidate evidence, but it # cannot precede generated_at, so generated_at + 0/20/68h is a floor every - # authentic observation meets. Upper bounds are left to the plan job. + # authentic observation meets. The plan job does not bound a partial re-run, + # which keeps its earlier decision; the run lookup below does. case "$milestone" in manual) floor_hours=0 ;; t-plus-24h) floor_hours=20 ;; @@ -439,6 +455,92 @@ while IFS= read -r canary; do fail "RC canaries must complete in manual, T+24h, T+72h order" previous_canary_epoch="$canary_completed_epoch" + # Written policy: a canary is a run of the temporary release-candidate-canary + # workflow, the timed ones from its dated schedule, and its plan job admits + # T+24h only 20-30 and T+72h only 68-78 whole hours after T0. A partial + # re-run ("Re-run failed jobs") does not run the plan job again: it keeps + # the earlier decision and writes new evidence later. With attestation + # verification on, the run is looked up. It must be a successful run of that + # workflow on this repository from the expected event, and the evidence must + # come from the successful attempt whose span contains completed_at, with + # the evidence job itself run in that attempt. That attempt started after + # T0 and, for a timed milestone, before its window closed (T0+31h, T0+79h); + # a first attempt ran its own plan job, so its evidence also completed after + # the window opened (T0+20h, T0+68h). Whether a re-run started inside the + # window should count is an open operator decision; this applies the windows + # as written. The attempt is recorded. + canary_attempt=null + canary_attempt_started_at="" + canary_head_branch="" + if [[ "${BPFCOMPAT_SKIP_READINESS_ATTESTATION:-0}" != "1" ]]; then + canary_run="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${canary_run_id}" 2>/dev/null)" || + fail "RC canary ${milestone} run ${canary_run_id} cannot be read from GitHub" + jq -e --arg event "$expected_event" ' + .path == ".github/workflows/release-candidate-canary.yml" and + .event == $event and + .head_repository.full_name == "Kernel-Guard/bpfcompat" and + .status == "completed" and .conclusion == "success" + ' <<<"$canary_run" >/dev/null || + fail "RC canary ${milestone} run ${canary_run_id} is not a successful ${expected_event} run of release-candidate-canary.yml on this repository" + canary_head_branch="$(jq -r '.head_branch // ""' <<<"$canary_run")" + # Scheduled runs execute on the default branch. Written policy names no + # branch for the manual dispatch, so its branch is recorded, not enforced. + if [[ "$milestone" != manual ]]; then + [[ "$canary_head_branch" == main ]] || + fail "RC canary ${milestone} run ${canary_run_id} is not on main" + fi + canary_last_attempt="$(jq -r '.run_attempt' <<<"$canary_run")" + [[ "$canary_last_attempt" =~ ^[1-9][0-9]*$ ]] || + fail "RC canary ${milestone} run ${canary_run_id} has no valid run_attempt" + for ((attempt = 1; attempt <= canary_last_attempt; attempt++)); do + attempt_run="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${canary_run_id}/attempts/${attempt}" 2>/dev/null)" || + fail "RC canary ${milestone} run ${canary_run_id} attempt ${attempt} cannot be read from GitHub" + jq -e '.conclusion == "success"' <<<"$attempt_run" >/dev/null || continue + attempt_started_at="$(jq -r '.run_started_at // ""' <<<"$attempt_run")" + attempt_started_epoch="$(iso_epoch "$attempt_started_at" "RC canary run ${canary_run_id} attempt ${attempt} start")" + attempt_updated_epoch="$(iso_epoch "$(jq -r '.updated_at // ""' <<<"$attempt_run")" "RC canary run ${canary_run_id} attempt ${attempt} end")" + if [[ "$canary_completed_epoch" -ge "$attempt_started_epoch" && "$canary_completed_epoch" -le "$attempt_updated_epoch" ]]; then + canary_attempt="$attempt" + canary_attempt_started_at="$attempt_started_at" + canary_attempt_started_epoch="$attempt_started_epoch" + break + fi + done + [[ "$canary_attempt" != null ]] || + fail "RC canary ${milestone} completed_at ${canary_completed_at} is not inside a successful attempt of run ${canary_run_id}" + # An out-of-window or late "Re-run all jobs" attempt also concludes success, + # with every check and the evidence job skipped, and a re-run attempt lists + # carried-over jobs with their earlier start. The jobs API names a job by + # its display name, which release-candidate-canary.yml gives its evidence + # job; a renamed job fails closed. + canary_jobs="$(gh api "repos/Kernel-Guard/bpfcompat/actions/runs/${canary_run_id}/attempts/${canary_attempt}/jobs?per_page=100" 2>/dev/null)" || + fail "RC canary ${milestone} run ${canary_run_id} attempt ${canary_attempt} jobs cannot be read from GitHub" + evidence_job_started_at="$(jq -r ' + [(.jobs // [])[] | select(.name == "Assemble RC canary evidence")] | + if length == 1 and .[0].conclusion == "success" then .[0].started_at // "" else "" end + ' <<<"$canary_jobs")" + [[ -n "$evidence_job_started_at" ]] || + fail "RC canary ${milestone} run ${canary_run_id} attempt ${canary_attempt} has no successful \"Assemble RC canary evidence\" job" + evidence_job_started_epoch="$(iso_epoch "$evidence_job_started_at" "RC canary run ${canary_run_id} evidence job start")" + [[ "$evidence_job_started_epoch" -ge "$canary_attempt_started_epoch" ]] || + fail "RC canary ${milestone} run ${canary_run_id}: its evidence job was carried over into attempt ${canary_attempt}, not run in it" + [[ "$canary_attempt_started_epoch" -ge "$candidate_published_epoch" ]] || + fail "RC canary ${milestone} attempt ${canary_attempt} of run ${canary_run_id} started before release ${candidate_tag} was published" + case "$milestone" in + t-plus-24h) window_open=20 window_close=31 ;; + t-plus-72h) window_open=68 window_close=79 ;; + *) window_open="" window_close="" ;; + esac + if [[ -n "$window_close" ]]; then + [[ "$canary_attempt_started_epoch" -lt $((candidate_published_epoch + window_close * 3600)) ]] || + fail "RC canary ${milestone} attempt ${canary_attempt} of run ${canary_run_id} started ${window_close}h or more after release ${candidate_tag} was published" + if [[ "$canary_attempt" -eq 1 ]]; then + [[ "$canary_completed_epoch" -ge $((candidate_published_epoch + window_open * 3600)) ]] || + fail "RC canary ${milestone} completed less than ${window_open}h after release ${candidate_tag} was published" + fi + fi + fi + # Components: each is an artifact of this exact canary run (the workflow # downloads rc*-canary-component-<check>-<run id>), every check is # covered, and each check's own record -- shipped in the same artifact -- @@ -475,17 +577,29 @@ while IFS= read -r canary; do done printf '%s\n' "$canary_run_id" >>"$tmp/canary_run_ids" - printf "| %s | \`%s\` | %s |\n" \ - "$milestone" "$canary_run_id" "$canary_completed_at" >>"$tmp/canary_rows" + if [[ "$canary_attempt" == null ]]; then + printf "| %s | \`%s\` | not looked up | not looked up | not looked up | %s |\n" \ + "$milestone" "$canary_run_id" "$canary_completed_at" >>"$tmp/canary_rows" + else + printf "| %s | \`%s\` | %s | %s | %s | %s |\n" \ + "$milestone" "$canary_run_id" "$canary_attempt" "$canary_attempt_started_at" \ + "$canary_head_branch" "$canary_completed_at" >>"$tmp/canary_rows" + fi jq -n \ --arg milestone "$milestone" \ --argjson workflow_run_id "$canary_run_id" \ + --argjson attempt "$canary_attempt" \ + --arg attempt_started_at "$canary_attempt_started_at" \ + --arg head_branch "$canary_head_branch" \ --arg completed_at "$canary_completed_at" \ --arg evidence_sha256 "$canary_sha" \ --argjson minutes "$(((canary_completed_epoch - candidate_generated_epoch) / 60))" \ '{ milestone: $milestone, workflow_run_id: $workflow_run_id, + workflow_run_attempt: $attempt, + attempt_started_at: (if $attempt_started_at == "" then null else $attempt_started_at end), + head_branch: (if $head_branch == "" then null else $head_branch end), completed_at: $completed_at, minutes_after_candidate_generated: $minutes, components_verified: true, @@ -627,8 +741,12 @@ mkdir -p "$(dirname "$output_json")" echo echo "## Release-Candidate Canaries" echo - echo "| Milestone | Workflow run | Completed (UTC) |" - echo "|---|---|---|" + if [[ -n "$candidate_published_at" ]]; then + echo "Windows are measured from the release's publication at ${candidate_published_at}; each run, attempt and evidence job was read from GitHub." + echo + fi + echo "| Milestone | Workflow run | Attempt | Attempt started (UTC) | Branch | Completed (UTC) |" + echo "|---|---|---|---|---|---|" cat "$tmp/canary_rows" echo echo "## Required External Evidence" @@ -682,6 +800,7 @@ jq -n \ --arg candidate_commit "$candidate_commit" \ --arg candidate_image "$candidate_image" \ --arg candidate_sha "$candidate_sha" \ + --arg candidate_published_at "$candidate_published_at" \ --arg rollback_sha "$rollback_sha" \ --arg incident_sha "$incident_sha" \ --arg operator "$operator" \ @@ -735,7 +854,8 @@ jq -n \ tag: $candidate_tag, commit_sha: $candidate_commit, image: $candidate_image, - evidence_sha256: $candidate_sha + evidence_sha256: $candidate_sha, + published_at: (if $candidate_published_at == "" then null else $candidate_published_at end) }, canaries: $canaries, rollback: { diff --git a/scripts/production-readiness-report_test.sh b/scripts/production-readiness-report_test.sh index 013e698..1adab37 100755 --- a/scripts/production-readiness-report_test.sh +++ b/scripts/production-readiness-report_test.sh @@ -626,8 +626,18 @@ mkdir -p "$tmp/bin" cat >"$tmp/bin/gh" <<'STUB' #!/usr/bin/env bash [[ "$1" == attestation && "$2" == --help ]] && exit 0 +# serve <API path>: answer a read from $STUB_API_DIR/<path without its query +# string>.json (the canary runs, their attempts and jobs, and the candidate +# release); a missing file is a 404. +serve() { + local file="${STUB_API_DIR:?}/${1#repos/Kernel-Guard/bpfcompat/}" + file="${file%%\?*}.json" + [[ -f "$file" ]] || { echo "gh: Not Found (HTTP 404)" >&2; exit 1; } + cat "$file" +} # Workflow run lookups: run 5000 is the rollback drill and 6000 the rejected -# promotion; STUB_* variables override their fields, any other run is a 404. +# promotion; STUB_* variables override their fields. Any other run is served +# from $STUB_API_DIR, else a 404. if [[ "$1" == api && "$2" =~ ^repos/Kernel-Guard/bpfcompat/actions/runs/([0-9]+)$ ]]; then printf '%s\n' "$*" >>"$STUB_LOG" case "${BASH_REMATCH[1]}" in @@ -642,10 +652,15 @@ if [[ "$1" == api && "$2" =~ ^repos/Kernel-Guard/bpfcompat/actions/runs/([0-9]+) --arg r "${STUB_INCIDENT_REPO:-Kernel-Guard/bpfcompat}" \ '{path: $p, head_branch: "v0.4.0-rc.1", event: $e, head_repository: {full_name: $r}, status: "completed", conclusion: $c}' ;; - *) echo "gh: Not Found (HTTP 404)" >&2; exit 1 ;; + *) serve "$2" ;; esac exit 0 fi +if [[ "$1" == api && "$2" == repos/Kernel-Guard/bpfcompat/* ]]; then + printf '%s\n' "$*" >>"$STUB_LOG" + serve "$2" + exit 0 +fi [[ "$1 $2" == "attestation verify" ]] || { echo "unexpected gh $*" >&2; exit 99; } printf '%s\n' "$*" >>"$STUB_LOG" [[ -n "${STUB_FAIL:-}" && "$3" == *"$STUB_FAIL"* ]] && exit 1 @@ -660,6 +675,51 @@ if [[ "$*" == *latest-kernel-compatibility.yml* ]]; then fi STUB chmod +x "$tmp/bin/gh" +# GitHub API answers for the canary lookups, one file per endpoint. The +# candidate release was published (T0) at 14:50Z, 50 minutes after +# generated_at. Canary run 300<i> has one successful attempt, from the +# workflow_dispatch (manual) or schedule event, whose evidence job ran in it +# just before the canary's completed_at. +api="$tmp/api" +# api_put <API dir> <path> <jq program> [jq args...]: write one answer. +api_put() { + local dir="$1" path="$2" program="$3" + shift 3 + mkdir -p "$(dirname "$dir/$path")" + jq -n "$@" "$program" >"$dir/$path.json" +} +# canary_attempt <API dir> <run> <attempt> <event> <conclusion> <started> +# <updated> <evidence job conclusion> <evidence job started>: one attempt of +# a canary run and its jobs. The run itself shows its latest attempt, so the +# last attempt written also becomes the run. +# shellcheck disable=SC2016 # the single-quoted arguments are jq programs +canary_attempt() { + api_put "$1" "actions/runs/$2/attempts/$3" \ + '{id: $run, path: ".github/workflows/release-candidate-canary.yml", head_branch: "main", + event: $e, head_repository: {full_name: "Kernel-Guard/bpfcompat"}, run_attempt: $n, + status: "completed", conclusion: $c, run_started_at: $s, updated_at: $u}' \ + --argjson run "$2" --argjson n "$3" --arg e "$4" --arg c "$5" --arg s "$6" --arg u "$7" + api_put "$1" "actions/runs/$2/attempts/$3/jobs" \ + '{total_count: 2, jobs: [ + {name: "Select RC observation window", run_attempt: $n, status: "completed", + conclusion: "success", started_at: $s, completed_at: $s}, + {name: "Assemble RC canary evidence", run_attempt: $n, status: "completed", + conclusion: $ec, started_at: $es, completed_at: $u}]}' \ + --argjson n "$3" --arg s "$6" --arg u "$7" --arg ec "$8" --arg es "$9" + cp "$1/actions/runs/$2/attempts/$3.json" "$1/actions/runs/$2.json" +} +api_put "$api" releases/tags/v0.4.0-rc.1 \ + '{tag_name: "v0.4.0-rc.1", draft: false, prerelease: true, published_at: "2026-07-30T14:50:00Z"}' +for index in 0 1 2; do + day=$((index * (index + 1) / 2)) + event=schedule + [[ "$index" == 0 ]] && event=workflow_dispatch + canary_attempt "$api" "$((3000 + index))" 1 "$event" success \ + "$(date -u -d "2026-07-30 14:59:00 UTC +${day} days" +%Y-%m-%dT%H:%M:%SZ)" \ + "$(date -u -d "2026-07-30 15:06:00 UTC +${day} days" +%Y-%m-%dT%H:%M:%SZ)" success \ + "$(date -u -d "2026-07-30 15:05:20 UTC +${day} days" +%Y-%m-%dT%H:%M:%SZ)" +done +export STUB_API_DIR="$api" : >"$tmp/gh.log" PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" "$script" "$tmp/input.json" "$tmp/attested.md" >/dev/null grep -Fq -- "--source-digest $(printf 'c%.0s' {1..40}) --source-ref refs/tags/v0.4.0-rc.1" "$tmp/gh.log" || { @@ -711,6 +771,216 @@ fi echo "[production-readiness-test] refused: an incident linking a run that does not exist" mv "$incident_note.keep" "$incident_note" +# --- Canary runs: identity, attempt and observation windows -------------------- +# Written policy: a canary is a run of the temporary release-candidate-canary +# workflow, the timed ones from its dated schedule, and its plan job admits +# T+24h only 20-30 and T+72h only 68-78 whole hours after publication (T0). A +# partial re-run keeps the plan job's earlier decision and writes new evidence +# later, so with attestation verification on every canary run is looked up. +for path in releases/tags/v0.4.0-rc.1 \ + actions/runs/3000 actions/runs/3000/attempts/1 "actions/runs/3000/attempts/1/jobs?per_page=100" \ + actions/runs/3001 actions/runs/3001/attempts/1 "actions/runs/3001/attempts/1/jobs?per_page=100" \ + actions/runs/3002 actions/runs/3002/attempts/1 "actions/runs/3002/attempts/1/jobs?per_page=100"; do + grep -Fxq "api repos/Kernel-Guard/bpfcompat/$path" "$tmp/gh.log" || { + echo "[production-readiness-test] canary lookup $path was not made" >&2; exit 1; } +done +jq -e ' + [.canaries[].workflow_run_attempt] == [1, 1, 1] and + [.canaries[].attempt_started_at] == ["2026-07-30T14:59:00Z", "2026-07-31T14:59:00Z", "2026-08-02T14:59:00Z"] and + [.canaries[].head_branch] == ["main", "main", "main"] and + .candidate.published_at == "2026-07-30T14:50:00Z" +' "$tmp/attested.json" >/dev/null || { + echo "[production-readiness-test] attested readiness output does not record the canary attempts" >&2; exit 1; } +grep -Fq "| t-plus-24h | \`3001\` | 1 | 2026-07-31T14:59:00Z | main | 2026-07-31T15:05:34Z |" "$tmp/attested.md" || { + echo "[production-readiness-test] attested readiness Markdown does not show the canary attempts" >&2; exit 1; } +jq -e '([.canaries[] | .workflow_run_attempt == null and .attempt_started_at == null and .head_branch == null] | all) and + .candidate.published_at == null' "$tmp/readiness.json" >/dev/null || { + echo "[production-readiness-test] bypass readiness output claims canary lookups it did not make" >&2; exit 1; } +echo "[production-readiness-test] attested: the release and every canary run, attempt and job list were looked up and recorded" +# The jobs API names a job only by its display name. While the temporary +# canary workflow exists, its evidence job must carry the name readiness +# looks for. +rcc="${ROOT_DIR}/.github/workflows/release-candidate-canary.yml" +grep -Fq 'select(.name == "Assemble RC canary evidence")' "$script" || { + echo "[production-readiness-test] readiness no longer looks up the evidence job by name" >&2; exit 1; } +if [[ -f "$rcc" ]]; then + awk '/^ evidence:$/ {getline; found = ($0 == " name: Assemble RC canary evidence")} END {exit !found}' "$rcc" || { + echo "[production-readiness-test] release-candidate-canary.yml does not name its evidence job \"Assemble RC canary evidence\"" >&2; exit 1; } + echo "[production-readiness-test] release-candidate-canary.yml names its evidence job as readiness expects" +fi + +c="$tmp/api-case" +# api_case: a fresh copy of the default API answers for one case. +api_case() { rm -rf "$c"; cp -R "$api" "$c"; } +# api_edit <path> <jq filter>: change one answer of the current case. +api_edit() { jq "$2" "$c/$1.json" >"$c/$1.json.new" && mv "$c/$1.json.new" "$c/$1.json"; } +# canary_completed <index> <completed_at> <out manifest>: a manifest pinning +# canary <index> evidence that records another completed_at; canary_restore +# puts the original evidence back. +canary_completed() { + local file="$tmp/canary-$1/release-candidate-canary.json" + cp "$file" "$file.orig" + jq --arg t "$2" '.completed_at = $t' "$file.orig" >"$file" + rehash "$file" ".canaries[$1].evidence_sha256" "$3" +} +canary_restore() { mv "$tmp/canary-$1/release-candidate-canary.json.orig" "$tmp/canary-$1/release-candidate-canary.json"; } +# canary_refused <label> <manifest> <message fragment>: refused, for that +# reason, with the current case's API answers. +canary_refused() { + local out + if out="$(PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" STUB_API_DIR="$c" "$script" "$2" "$tmp/bad.md" 2>&1)"; then + echo "[production-readiness-test] $1 was accepted" >&2 + exit 1 + fi + grep -Fq -- "$3" <<<"$out" || { + echo "[production-readiness-test] $1 refused for the wrong reason: $out" >&2 + exit 1 + } + echo "[production-readiness-test] refused: $1" +} +# canary_accepted <label> <manifest> <jq assertion on the readiness JSON> +canary_accepted() { + local out + out="$(PATH="$tmp/bin:$PATH" STUB_LOG="$tmp/gh.log" STUB_API_DIR="$c" "$script" "$2" "$tmp/case.md" 2>&1)" || { + echo "[production-readiness-test] $1 was refused: $out" >&2 + exit 1 + } + jq -e "$3" "$tmp/case.json" >/dev/null || { + echo "[production-readiness-test] $1 is not recorded as expected: $(jq -c '.canaries' "$tmp/case.json")" >&2 + exit 1 + } + echo "[production-readiness-test] accepted and recorded: $1" +} + +# Run identity. +api_case; api_edit actions/runs/3001 '.path = ".github/workflows/ci.yml"' +canary_refused "a T+24h canary run of another workflow" "$tmp/input.json" \ + "RC canary t-plus-24h run 3001 is not a successful schedule run of release-candidate-canary.yml on this repository" +api_case; api_edit actions/runs/3002 '.head_branch = "feature"' +canary_refused "a T+72h canary run not on main" "$tmp/input.json" "RC canary t-plus-72h run 3002 is not on main" +api_case; api_edit actions/runs/3001 '.head_repository.full_name = "someone/bpfcompat"' +canary_refused "a T+24h canary run from a fork" "$tmp/input.json" "RC canary t-plus-24h run 3001 is not a successful schedule run" +api_case; api_edit actions/runs/3001 '.event = "workflow_dispatch"' +canary_refused "a T+24h canary from a dispatched run" "$tmp/input.json" "RC canary t-plus-24h run 3001 is not a successful schedule run" +api_case; api_edit actions/runs/3000 '.event = "schedule"' +canary_refused "a manual canary from a scheduled run" "$tmp/input.json" "RC canary manual run 3000 is not a successful workflow_dispatch run" +api_case; api_edit actions/runs/3001 '.conclusion = "failure"' +canary_refused "a T+24h canary run that failed" "$tmp/input.json" "RC canary t-plus-24h run 3001 is not a successful schedule run" +api_case; rm "$c/actions/runs/3002.json" +canary_refused "a T+72h canary naming a run that does not exist" "$tmp/input.json" "RC canary t-plus-72h run 3002 cannot be read from GitHub" +# No written rule names the manual dispatch's branch: recorded, not enforced. +api_case; api_edit actions/runs/3000 '.head_branch = "feature"' +canary_accepted "a manual canary dispatched on a branch (branch recorded)" "$tmp/input.json" '.canaries[0].head_branch == "feature"' + +# The candidate release: T0 is its publication. +api_case; api_edit releases/tags/v0.4.0-rc.1 '.draft = true' +canary_refused "an unpublished (draft) candidate release" "$tmp/input.json" "release v0.4.0-rc.1 is a draft, not a published release" +api_case; rm "$c/releases/tags/v0.4.0-rc.1.json" +canary_refused "a candidate release that cannot be read" "$tmp/input.json" "release v0.4.0-rc.1 cannot be read from GitHub" +api_case; api_edit releases/tags/v0.4.0-rc.1 '.published_at = "2026-07-30T13:59:59Z"' +canary_refused "a candidate release published before its evidence was generated" "$tmp/input.json" \ + "release v0.4.0-rc.1 was published at 2026-07-30T13:59:59Z, before its candidate evidence was generated" +api_case; api_edit releases/tags/v0.4.0-rc.1 '.published_at = "2026-07-30T15:00:00Z"' +canary_refused "a manual canary attempt that started before publication" "$tmp/input.json" \ + "RC canary manual attempt 1 of run 3000 started before release v0.4.0-rc.1 was published" + +# The attempt: the successful one whose span contains completed_at. +api_case +api_edit actions/runs/3001 '.updated_at = "2026-07-31T15:05:00Z"' +api_edit actions/runs/3001/attempts/1 '.updated_at = "2026-07-31T15:05:00Z"' +canary_refused "a T+24h canary that completed after its run ended" "$tmp/input.json" \ + "RC canary t-plus-24h completed_at 2026-07-31T15:05:34Z is not inside a successful attempt of run 3001" +# A later "re-run all jobs" outside the window concludes success with every job +# skipped; the evidence is attempt 1's. +api_case +canary_attempt "$c" 3001 2 schedule success 2026-08-01T05:00:00Z 2026-08-01T05:01:00Z skipped 2026-08-01T05:00:40Z +canary_accepted "T+24h evidence from an earlier successful attempt of a re-run run" "$tmp/input.json" \ + '.canaries[1].workflow_run_attempt == 1 and .canaries[1].attempt_started_at == "2026-07-31T14:59:00Z"' +# A failed attempt never counts, even when its span contains completed_at. +api_case +canary_attempt "$c" 3001 1 schedule failure 2026-07-31T14:59:00Z 2026-07-31T15:06:00Z success 2026-07-31T15:05:20Z +canary_attempt "$c" 3001 2 schedule success 2026-07-31T16:20:00Z 2026-07-31T16:31:00Z success 2026-07-31T16:29:30Z +canary_refused "T+24h evidence from a failed attempt" "$tmp/input.json" "is not inside a successful attempt of run 3001" +# A late re-run's completed_at edited back into the window: no successful +# attempt spans it. +api_case +canary_attempt "$c" 3001 1 schedule failure 2026-07-31T14:59:00Z 2026-07-31T15:03:00Z skipped 2026-07-31T15:03:00Z +canary_attempt "$c" 3001 2 schedule success 2026-08-01T01:50:00Z 2026-08-01T02:01:00Z success 2026-08-01T01:59:30Z +canary_refused "a late re-run whose completed_at was edited into the window" "$tmp/input.json" \ + "is not inside a successful attempt of run 3001" + +# The evidence job itself ran, and succeeded, in that attempt. +api_case +api_edit actions/runs/3001/attempts/1/jobs '(.jobs[] | select(.name == "Assemble RC canary evidence") | .conclusion) = "skipped"' +canary_refused "a T+24h attempt whose evidence job was skipped" "$tmp/input.json" \ + "RC canary t-plus-24h run 3001 attempt 1 has no successful \"Assemble RC canary evidence\" job" +api_case +api_edit actions/runs/3001/attempts/1/jobs '.jobs |= map(select(.name != "Assemble RC canary evidence"))' +canary_refused "a T+24h attempt without an evidence job" "$tmp/input.json" "has no successful \"Assemble RC canary evidence\" job" +api_case +canary_attempt "$c" 3001 1 schedule failure 2026-07-31T14:59:00Z 2026-07-31T15:03:00Z skipped 2026-07-31T15:03:00Z +canary_attempt "$c" 3001 2 schedule success 2026-07-31T16:20:00Z 2026-07-31T16:31:00Z success 2026-07-31T15:02:00Z +canary_completed 1 2026-07-31T16:30:00Z "$tmp/canary-case.json" +canary_refused "a T+24h evidence job carried over into a re-run attempt" "$tmp/canary-case.json" \ + "RC canary t-plus-24h run 3001: its evidence job was carried over into attempt 2, not run in it" +canary_restore 1 + +# Observation windows from T0 (14:50Z): T+24h [T0+20h, T0+31h) = [07-31 10:50Z, +# 07-31 21:50Z), T+72h [T0+68h, T0+79h) = [08-02 10:50Z, 08-02 21:50Z). +api_case +canary_attempt "$c" 3001 1 schedule success 2026-07-31T22:00:00Z 2026-07-31T22:15:00Z success 2026-07-31T22:09:30Z +canary_completed 1 2026-07-31T22:10:00Z "$tmp/canary-case.json" +canary_refused "a first-attempt T+24h canary started after its window closed" "$tmp/canary-case.json" \ + "RC canary t-plus-24h attempt 1 of run 3001 started 31h or more after release v0.4.0-rc.1 was published" +canary_restore 1 +# F6: "re-run failed jobs" on a failed in-window run, after the window closed. +api_case +canary_attempt "$c" 3001 1 schedule failure 2026-07-31T14:59:00Z 2026-07-31T15:03:00Z skipped 2026-07-31T15:03:00Z +canary_attempt "$c" 3001 2 schedule success 2026-08-01T01:50:00Z 2026-08-01T02:01:00Z success 2026-08-01T01:59:30Z +canary_completed 1 2026-08-01T02:00:00Z "$tmp/canary-case.json" +canary_refused "a late partial re-run of a failed in-window T+24h run" "$tmp/canary-case.json" \ + "RC canary t-plus-24h attempt 2 of run 3001 started 31h or more after release v0.4.0-rc.1 was published" +canary_restore 1 +api_case +canary_attempt "$c" 3002 1 schedule failure 2026-08-02T14:59:00Z 2026-08-02T15:03:00Z skipped 2026-08-02T15:03:00Z +canary_attempt "$c" 3002 2 schedule success 2026-08-02T21:50:00Z 2026-08-02T21:58:00Z success 2026-08-02T21:56:30Z +canary_completed 2 2026-08-02T21:57:00Z "$tmp/canary-case.json" +canary_refused "a T+72h re-run started exactly when its window closed" "$tmp/canary-case.json" \ + "RC canary t-plus-72h attempt 2 of run 3002 started 79h or more after release v0.4.0-rc.1 was published" +canary_restore 2 +# A re-run started inside the window counts as the text is written: the open +# operator decision may tighten or relax this, the test records today's rule. +api_case +canary_attempt "$c" 3001 1 schedule failure 2026-07-31T14:59:00Z 2026-07-31T15:03:00Z skipped 2026-07-31T15:03:00Z +canary_attempt "$c" 3001 2 schedule success 2026-07-31T16:20:00Z 2026-07-31T16:31:00Z success 2026-07-31T16:29:30Z +canary_completed 1 2026-07-31T16:30:00Z "$tmp/canary-case.json" +canary_accepted "an in-window re-run of a failed T+24h run (attempt 2)" "$tmp/canary-case.json" \ + '.canaries[1].workflow_run_attempt == 2 and .canaries[1].attempt_started_at == "2026-07-31T16:20:00Z"' +grep -Fq "| t-plus-24h | \`3001\` | 2 | 2026-07-31T16:20:00Z | main | 2026-07-31T16:30:00Z |" "$tmp/case.md" || { + echo "[production-readiness-test] the re-run attempt is not shown in the readiness Markdown" >&2; exit 1; } +canary_restore 1 +api_case +canary_attempt "$c" 3001 1 schedule failure 2026-07-31T14:59:00Z 2026-07-31T15:03:00Z skipped 2026-07-31T15:03:00Z +canary_attempt "$c" 3001 2 schedule success 2026-07-31T21:49:59Z 2026-07-31T21:56:00Z success 2026-07-31T21:54:30Z +canary_completed 1 2026-07-31T21:55:00Z "$tmp/canary-case.json" +canary_accepted "a T+24h re-run started one second before its window closed" "$tmp/canary-case.json" \ + '.canaries[1].workflow_run_attempt == 2 and .canaries[1].attempt_started_at == "2026-07-31T21:49:59Z"' +canary_restore 1 +# A first attempt ran its own plan job, so its evidence cannot complete before +# the window opened, even 20h (68h) after generated_at. +api_case +canary_attempt "$c" 3001 1 schedule success 2026-07-31T10:25:00Z 2026-07-31T10:32:00Z success 2026-07-31T10:29:40Z +canary_completed 1 2026-07-31T10:30:00Z "$tmp/canary-case.json" +canary_refused "a first-attempt T+24h canary completed before T0+20h" "$tmp/canary-case.json" \ + "RC canary t-plus-24h completed less than 20h after release v0.4.0-rc.1 was published" +canary_restore 1 +api_case +canary_attempt "$c" 3002 1 schedule success 2026-08-02T10:25:00Z 2026-08-02T10:32:00Z success 2026-08-02T10:29:40Z +canary_completed 2 2026-08-02T10:30:00Z "$tmp/canary-case.json" +canary_refused "a first-attempt T+72h canary completed before T0+68h" "$tmp/canary-case.json" \ + "RC canary t-plus-72h completed less than 68h after release v0.4.0-rc.1 was published" +canary_restore 2 + # --- Readiness output v0.2 and the real stable gate ---------------------------- jq -e --arg c "$(printf 'c%.0s' {1..40})" ' .schema_version == "v0.2" and .campaign_binding == "candidate-bound-v2" and From 4a1466dd514f5cdc297094a361ed3e3744fa9430 Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 22:29:27 +0300 Subject: [PATCH 36/38] security(release): refuse stable readiness records without the canary run lookups Integration-only commit on the v0.4.0-rc.5 landing branch; not part of any source PR. Reconciles #268 (fdae550) with claude/release-provenance-binding (5794ee4). #268's production-readiness-report.sh looks up the candidate release and every canary run, attempt and evidence job in attested mode, and records candidate.published_at and each canary's workflow_run_attempt, attempt_started_at and head_branch. The provenance branch's stable gate in release-artifacts.yml ("Stage production readiness evidence") read only `.canaries | length == 3`, so a readiness record generated by a checkout from before those lookups (main 402d7c9, #268 7f7a4ef) still passed the v0.4.0 tag build. The gate now also requires every canary's workflow_run_attempt to be a positive integer and its attempt_started_at a UTC timestamp, and candidate.published_at a UTC timestamp. The attested report writes these only after the lookups succeed (production-readiness-report.sh:390-391, :492-504, :600-602, :858 at fdae550); the attestation bypass writes null and older reports omit the keys. head_branch is not required: the manual canary's branch is recorded, not enforced. No other gate term changes. scripts/release-workflow-gates_test.sh: the fixture is regenerated as the unedited attested output of this branch's production-readiness-report.sh for production-readiness-report_test.sh's input. 14 new refusals: no lookups at all, the bypass's nulls with attestations_verified true, and each field absent, null or malformed on one or every canary. 2 new acceptances: an in-window re-run recorded as attempt 2, and a manual canary dispatched on a branch. Against the unchanged gate exactly the 14 refusals fail; with the new terms the test passes, and production-readiness-report_test.sh still extracts the gate and accepts its freshly generated attested output. Signed-off-by: ErenAri <erenari27@gmail.com> --- .github/workflows/release-artifacts.yml | 5 ++ scripts/release-workflow-gates_test.sh | 63 ++++++++++++++++++++----- 2 files changed, 55 insertions(+), 13 deletions(-) diff --git a/.github/workflows/release-artifacts.yml b/.github/workflows/release-artifacts.yml index ab9aec0..1878c26 100644 --- a/.github/workflows/release-artifacts.yml +++ b/.github/workflows/release-artifacts.yml @@ -201,6 +201,11 @@ jobs: .slo.campaign_count == 4 and .slo.infrastructure_errors == 0 and (.canaries | length == 3) and + ([.canaries[] | + (.workflow_run_attempt | type == "number" and . >= 1 and . == floor) and + (.attempt_started_at | type == "string" and test("^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$")) + ] | all) and + ($c.published_at | type == "string" and test("^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$")) and .rollback.completed == true and .rollback.alias == (($c.image | sub("@[^@]*$"; "")) + ":rollback-drill-" + $c.tag) and .rollback.candidate_digest == ($c.image | sub("^.*@"; "")) and diff --git a/scripts/release-workflow-gates_test.sh b/scripts/release-workflow-gates_test.sh index e0ae1e9..a5ac952 100755 --- a/scripts/release-workflow-gates_test.sh +++ b/scripts/release-workflow-gates_test.sh @@ -7,8 +7,12 @@ # 1. Stable gate (release-artifacts.yml, "Stage production readiness # evidence"): a ready readiness record must carry a completed rollback # drill of this release's own candidate (drill alias and image digest), -# like the incident section. The gate's jq program is extracted from the -# workflow and run against real production-readiness-report.sh output. +# like the incident section, and the canary run lookups the attested +# report records (each canary's run attempt and its start, and the +# release's publication time), so a record written by a report that did +# not make those lookups is refused. The gate's jq program is extracted +# from the workflow and run against real production-readiness-report.sh +# output. # 2. Promotion (promote-release.yml): the exact run: bodies of the # re-verification and image steps run against stub cosign/gh/docker. The # draft's signature files and SBOM must be the candidate run's own @@ -56,7 +60,11 @@ run_gate() { # Unedited production-readiness-report.sh output (attested mode) for the # fixture input of production-readiness-report_test.sh, which also runs the -# extracted gate against freshly generated output. +# extracted gate against freshly generated output. In attested mode the report +# looks up the candidate release and every canary run and records +# candidate.published_at and each canary's workflow_run_attempt, +# attempt_started_at and head_branch; with the attestation bypass they are +# null, and a report from before those lookups has none of them. cat >"$tmp/attested.json" <<'JSON' { "schema_version": "v0.2", @@ -65,12 +73,12 @@ cat >"$tmp/attested.json" <<'JSON' "gate_status": "ready", "release_version": "0.4.0", "supported_boundary": "CLI + GitHub Action + disposable QEMU/KVM validation", - "generated_at": "2026-10-19T06:18:49Z", + "generated_at": "2026-10-19T19:24:43Z", "campaigns": [ - {"workflow_run_id": 1000, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000001", "started_at": "2026-08-03T03:21:00Z", "report_sha256": "bbbfcb6775f7cc9aeb67c38de6344bf05fe55ba048b64565c6a808460063748a", "target_count": 1}, - {"workflow_run_id": 1001, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000002", "started_at": "2026-08-10T03:21:00Z", "report_sha256": "b453d76863cda6c5b8f908e61ec16a11343caebbb6b06632efcdb229f0e72e3b", "target_count": 1}, - {"workflow_run_id": 1002, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000003", "started_at": "2026-08-17T03:21:00Z", "report_sha256": "62aef96eef32fc2e74fb0c45d6f40b7d048a9a66c8e26f80ba0a9207fd99dda9", "target_count": 1}, - {"workflow_run_id": 1003, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000004", "started_at": "2026-08-24T03:21:00Z", "report_sha256": "34c6d19b431fc19ca3af155fb90a5afcc91da5611e3f98a50045f66e85804c41", "target_count": 1} + {"workflow_run_id": 1000, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000001", "started_at": "2026-08-03T03:21:00Z", "report_sha256": "0291d23025530eccf75e3e2b5922edbb6c7f39aee1e48688213a668866cc795c", "target_count": 1}, + {"workflow_run_id": 1001, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000002", "started_at": "2026-08-10T03:21:00Z", "report_sha256": "2247827b5fe5a0a2a5abd4aa27729809e9fe0ab4dfc4ae3a1eb3a5ffe337806d", "target_count": 1}, + {"workflow_run_id": 1002, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000003", "started_at": "2026-08-17T03:21:00Z", "report_sha256": "3acb06c0c5086a1bf756c6af91dc911ea324b862596b7283aa877d9f554adca7", "target_count": 1}, + {"workflow_run_id": 1003, "candidate_tag": "v0.4.0-rc.1", "candidate_commit": "cccccccccccccccccccccccccccccccccccccccc", "executed": {"cli_sha256": "8888888888888888888888888888888888888888888888888888888888888888", "validator_sha256": "7777777777777777777777777777777777777777777777777777777777777777"}, "workflow_sha": "0000000000000000000000000000000000000004", "started_at": "2026-08-24T03:21:00Z", "report_sha256": "dbfeaf53a9af3d5951b354d2d6a057e7010f49fee52d79742dceedfd3e1a4af0", "target_count": 1} ], "slo": { "campaign_count": 4, @@ -83,7 +91,7 @@ cat >"$tmp/attested.json" <<'JSON' "workflow_run_id": 2000, "commit_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "started_at": "2026-08-03T06:00:00Z", - "report_sha256": "b066de461debfb7a52511e99a0e9f6209bbb0b2e25da441c1563659030c6e5a7", + "report_sha256": "477b3159399a5eec5be74073d087fa48982fb3501ebc320f6a61169fcce9cbd0", "report_run_id": "falco-run", "run_identity": "operator-asserted", "started_after_candidate": true, @@ -95,12 +103,13 @@ cat >"$tmp/attested.json" <<'JSON' "tag": "v0.4.0-rc.1", "commit_sha": "cccccccccccccccccccccccccccccccccccccccc", "image": "ghcr.io/kernel-guard/bpfcompat@sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", - "evidence_sha256": "01463694049c7ae9040e4957ae63b2109a3a689ee53f5e1f7355bfd7825cd94b" + "evidence_sha256": "01463694049c7ae9040e4957ae63b2109a3a689ee53f5e1f7355bfd7825cd94b", + "published_at": "2026-07-30T14:50:00Z" }, "canaries": [ - {"milestone": "manual", "workflow_run_id": 3000, "completed_at": "2026-07-30T15:05:34Z", "minutes_after_candidate_generated": 65, "components_verified": true, "evidence_sha256": "9c600cc50ac1ad95d5b2eac72cd7e9bd64209e02527df5f9e23f6366db49e813"}, - {"milestone": "t-plus-24h", "workflow_run_id": 3001, "completed_at": "2026-07-31T15:05:34Z", "minutes_after_candidate_generated": 1505, "components_verified": true, "evidence_sha256": "ba0b104f1f0d6715fb4948d3109431426a00c1725b439f370068d94ab6b99323"}, - {"milestone": "t-plus-72h", "workflow_run_id": 3002, "completed_at": "2026-08-02T15:05:34Z", "minutes_after_candidate_generated": 4385, "components_verified": true, "evidence_sha256": "3f7fdf3d7c1a45e6f7dba2a644575d7bb3776737d3e438640878ed0a8695b53e"} + {"milestone": "manual", "workflow_run_id": 3000, "workflow_run_attempt": 1, "attempt_started_at": "2026-07-30T14:59:00Z", "head_branch": "main", "completed_at": "2026-07-30T15:05:34Z", "minutes_after_candidate_generated": 65, "components_verified": true, "evidence_sha256": "9c600cc50ac1ad95d5b2eac72cd7e9bd64209e02527df5f9e23f6366db49e813"}, + {"milestone": "t-plus-24h", "workflow_run_id": 3001, "workflow_run_attempt": 1, "attempt_started_at": "2026-07-31T14:59:00Z", "head_branch": "main", "completed_at": "2026-07-31T15:05:34Z", "minutes_after_candidate_generated": 1505, "components_verified": true, "evidence_sha256": "ba0b104f1f0d6715fb4948d3109431426a00c1725b439f370068d94ab6b99323"}, + {"milestone": "t-plus-72h", "workflow_run_id": 3002, "workflow_run_attempt": 1, "attempt_started_at": "2026-08-02T14:59:00Z", "head_branch": "main", "completed_at": "2026-08-02T15:05:34Z", "minutes_after_candidate_generated": 4385, "components_verified": true, "evidence_sha256": "3f7fdf3d7c1a45e6f7dba2a644575d7bb3776737d3e438640878ed0a8695b53e"} ], "rollback": { "completed": true, @@ -161,6 +170,34 @@ another candidate's image digest|.rollback.candidate_digest = $digest this tag's drill alias in another image repository|.rollback.alias = "ghcr.io/example/bpfcompat:rollback-drill-v0.4.0-rc.1" no incident section (control)|del(.incident) incident.completed false (control)|.incident.completed = false +none of the canary run lookups (a report from before they were made)|del(.canaries[].workflow_run_attempt, .canaries[].attempt_started_at, .canaries[].head_branch, .candidate.published_at) +the attestation bypass's null lookups, with attestations_verified true|.canaries[] |= (.workflow_run_attempt = null | .attempt_started_at = null | .head_branch = null) | .candidate.published_at = null +no canary run attempts|del(.canaries[].workflow_run_attempt) +null canary run attempts|.canaries[].workflow_run_attempt = null +one canary without its run attempt|.canaries[2].workflow_run_attempt = null +a canary run attempt as the string "1"|.canaries[1].workflow_run_attempt = "1" +canary run attempt 0|.canaries[0].workflow_run_attempt = 0 +a fractional canary run attempt|.canaries[0].workflow_run_attempt = 1.5 +no canary attempt start times|del(.canaries[].attempt_started_at) +one canary without its attempt start|.canaries[1].attempt_started_at = null +a canary attempt start that is not a UTC timestamp|.canaries[1].attempt_started_at = "2026-07-31 14:59:00" +no release publication time|del(.candidate.published_at) +a null release publication time|.candidate.published_at = null +a release publication time that is not a UTC timestamp|.candidate.published_at = "2026-07-30T14:50:00+00:00" +CASES + +# Canary records the attested report writes for runs it accepts. The manual +# canary's branch is recorded, not enforced, so the gate does not read it. +while IFS='|' read -r label edit; do + jq "$edit" "$tmp/attested.json" >"$tmp/gate-case.json" + if run_gate "$tmp/gate-case.json"; then + ok "stable gate accepts readiness with ${label}" + else + bad "stable gate refused readiness with ${label}" + fi +done <<'CASES' +a T+24h re-run recorded as attempt 2|.canaries[1].workflow_run_attempt = 2 | .canaries[1].attempt_started_at = "2026-07-31T21:49:59Z" +a manual canary dispatched on a branch|.canaries[0].head_branch = "rc5-shakeout" CASES # ------------------------------------------------------------------ 2. promotion From 94f03333b040965b59404af2a8d2f8eef2011bd3 Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 22:30:56 +0300 Subject: [PATCH 37/38] docs: state the Action's tag-bound prebuilt verification and canary re-run voiding Integration-only commit on the v0.4.0-rc.5 landing branch; not part of any source PR. Reconciles claude/release-provenance-binding (5794ee4) with #267 (2269647) and #268 (fdae550): wording each owner deferred to the union. - action.yml `prebuilt` description (#267 owns the input descriptions, the provenance branch the prebuilt step): the attestations are bound to the pinned release's tag and commit, and gh older than 2.68.0, a tag pin that cannot be resolved to its commit, or a resolution taking more than 60 s gives the warned source build (action.yml:223-230, :241, :251-259 at 5794ee4). Provenance review N3. - docs/verifying-releases.md (#267's Action section): from v0.4.0-rc.5 the Action passes --source-ref refs/tags/<tag> and --source-digest <commit>, with the same fallbacks; the v0.4.0-rc.2 to rc.4 Actions do not bind the tag. T3 remedy item 5. docs/compatibility-contract.md is unchanged: with the provenance branch its rc.5 sentence ("other bytes fail closed") is accurate as written. - docs/production-release-process.md (#268's Canaries bullet): GitHub reports a run's conclusion from its latest attempt, so a failed re-run of a successful canary voids it under the readiness run check (production-readiness-report.sh:482 at fdae550). Readiness review N1. make check-docs-drift, scripts/check-doc-pins.sh and scripts/check-doc-pins_test.sh pass. Signed-off-by: ErenAri <erenari27@gmail.com> --- action.yml | 2 +- docs/production-release-process.md | 5 ++++- docs/verifying-releases.md | 10 ++++++++++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/action.yml b/action.yml index 1544b69..bfafca1 100644 --- a/action.yml +++ b/action.yml @@ -76,7 +76,7 @@ inputs: required: false default: "true" prebuilt: - description: Use prebuilt release binaries when the action is pinned to a release tag or to a commit SHA that matches a release tag ("auto"), or always build from source ("never"). Downloads are checksum-checked and verified against GitHub build attestations, and the run then uses that verified validator by path and pinned digest (a workflow that sets BPFCOMPAT_VALIDATOR_BIN or BPFCOMPAT_VALIDATOR_SHA256 keeps its own settings, with a warning). Runners without gh build from source; while pinned to a release, every fallback to a source build is reported as a warning because the run then does not use the attested release validator. + description: Use prebuilt release binaries when the action is pinned to a release tag or to a commit SHA that matches a release tag ("auto"), or always build from source ("never"). Downloads are checksum-checked and verified against GitHub build attestations bound to the pinned release's tag and commit, and the run then uses that verified validator by path and pinned digest (a workflow that sets BPFCOMPAT_VALIDATOR_BIN or BPFCOMPAT_VALIDATOR_SHA256 keeps its own settings, with a warning). Runners without gh 2.68.0 or later build from source, and so does a tag pin that cannot be resolved to its commit or whose resolution takes more than 60 s; while pinned to a release, every fallback to a source build is reported as a warning because the run then does not use the attested release validator. required: false default: "auto" binary: diff --git a/docs/production-release-process.md b/docs/production-release-process.md index aa951cf..2340094 100644 --- a/docs/production-release-process.md +++ b/docs/production-release-process.md @@ -191,7 +191,10 @@ aggregator binds every piece of evidence to the attested candidate must be published, not a draft, and not before `generated_at`. Each canary's `workflow_run_id` must be a successful `release-candidate-canary.yml` run of this repository from `workflow_dispatch` (manual) or `schedule` (timed, on - `main`). Its `completed_at` must fall inside a successful attempt of that + `main`). GitHub reports a run's conclusion from its latest attempt, so + re-running a successful canary can void it: if the re-run fails, the run is + no longer successful. The canary's `completed_at` must fall inside a + successful attempt of that run in which the `Assemble RC canary evidence` job itself ran and succeeded, and that attempt must have started at or after T0. A timed canary's attempt must have started before its window closed (T0+31h or T0+79h), and a first diff --git a/docs/verifying-releases.md b/docs/verifying-releases.md index 8f6e11f..09661ae 100644 --- a/docs/verifying-releases.md +++ b/docs/verifying-releases.md @@ -140,6 +140,16 @@ attestation support), the v0.4.0 release candidates' Action builds from its pinned source instead of trusting unverifiable release bytes; the v0.3.8 release candidates' Action fails the job instead. +From v0.4.0-rc.5 the Action also binds that verification to the pinned +release: `--source-ref refs/tags/<tag>` and `--source-digest <commit>`, where +the commit is the pinned SHA or, for a tag pin, the commit the tag points at +(resolved with `git ls-remote`). Assets attested by another release's run are +a hard failure. This needs gh 2.68.0 or later: with an older gh, or when the +tag cannot be resolved to its commit within 60 seconds, the Action builds from +its pinned source with a warning instead of using the prebuilt binaries. The +v0.4.0-rc.2 to rc.4 Actions do not bind the tag, so an attestation from +another tag's release run also passes there. + From v0.4.0-rc.5, after verification the Action runs that validator by path and pinned digest (`BPFCOMPAT_VALIDATOR_BIN` / `BPFCOMPAT_VALIDATOR_SHA256`), so a validator left in `/usr/local/libexec` or `/usr/libexec` on a self-hosted From 0826cdeb8b4fab93cc4919c460732ace297133d9 Mon Sep 17 00:00:00 2001 From: ErenAri <erenari27@gmail.com> Date: Mon, 19 Oct 2026 22:33:57 +0300 Subject: [PATCH 38/38] docs(changelog): reconcile the v0.4.0-rc.5 section with the landing branch Integration-only commit on the v0.4.0-rc.5 landing branch; not part of any source PR. Reconciles #265's rc.5 section (e1cbd97; #265 owns CHANGELOG.md) with what the other sources bring into this branch: #266 (4cb4516), #267 (2269647), #268 (fdae550), #269 (69cebea), claude/release-provenance-binding (5794ee4) and this branch's stable-gate commit 4a1466d. The heading and its TBD date are unchanged. - Corrects CHANGELOG.md:110-112 at e1cbd97 ("asserted by the operator, not checked"): with attestation verification on, #268 reads the incident run and requires a completed, dispatched promote-release.yml run of this repository that did not succeed (production-readiness-report.sh:685-688 at fdae550). Decision-packet critique M3. - Security: the installer's gh path checks SHA256SUMS against the CLI's attested subjects (#269); the Action and verify-release-assets.sh bind attestations to the tag ref and commit, promotion checks the signature files and the exact signer identity, and the stable gate requires the rollback drill and the canary run lookups (provenance branch, 4a1466d). - Fixed: the installer's gh failure classes, probe retry and whole-value tag check (#269); the tag-bound manual verification commands (#267). - Changed: #268's Falco start and drill, incident and canary run lookups; #266's stable-window consistency checks and runbook order. - Migration notes: the Action's prebuilt path needs gh 2.68.0 or later (and git and timeout for a tag pin) or takes the warned source build; verify-release-assets.sh callers must set its two variables; a readiness record from an earlier report must be regenerated. The wording starts from the writers' proposed lines (installer round 2, provenance round 1, readiness R1) and was checked against the code on this branch. Signed-off-by: ErenAri <erenari27@gmail.com> --- CHANGELOG.md | 109 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 106 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8dcd0e1..f991ce6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,14 @@ qualification evidence does not carry over to this candidate. target directory before anything is installed. (The installer is served from `main`, so this already applies to `curl … /main/scripts/install.sh | sh` users since #236 merged, including v0.3.7 installs.) +- `install.sh` without cosign (the gh path) also checks the unsigned + `SHA256SUMS` against the subjects of the CLI's verified provenance. It + refuses, as a failed verification, when the two disagree on whether the + release has a validator or a license bundle, or when a file would be + installed under a name that provenance does not give its bytes. Before, a + deleted `SHA256SUMS` line, or a genuine asset of the same release served + under another asset's name, installed without error. The cosign path is + unchanged. - The GitHub Action no longer pastes inputs into its `run:` scripts. Every input reaches the scripts through `env:` and is read as quoted data, so a value containing quotes, `$(...)`, backticks or newlines can no longer run as @@ -31,6 +39,35 @@ qualification evidence does not carry over to this candidate. - Values containing `$VAR`, backticks or `\` used to be expanded by bash as a side effect; they are now passed literally. `command` is unchanged: it was already passed through `env:` and is still run by the guest shell. +- The GitHub Action verifies prebuilt binaries against the attestations of + exactly the pinned release: its tag ref and the commit it was built from + (`--source-ref`, `--source-digest`). Assets attested by another release's + run are a hard error. A tag pin is resolved to its commit with + `git ls-remote`, bounded to 60 seconds; if that fails or times out, or gh is + older than 2.68.0, the Action builds from its pinned source with a warning + instead of using the prebuilt binaries. The v0.4.0-rc.2 to rc.4 Actions + accept assets attested by another tag's release run. +- `scripts/verify-release-assets.sh` requires `BPFCOMPAT_EXPECTED_SOURCE_SHA` + (a full commit SHA) and `BPFCOMPAT_EXPECTED_SOURCE_REF` (`refs/tags/v…`), + refuses to run without them (exit 2), and binds every + `gh attestation verify` to them. The release, promotion and maintenance + workflows pass the release's own commit and tag. +- Promotion requires the draft release's `SHA256SUMS.sig`, `SHA256SUMS.crt` + and SBOM files (`bpfcompat.sbom.cdx.json` and its `.sig` and `.crt`) to be + the candidate run's own copies, and binds the attestations of the draft + assets and the candidate evidence to the tag and commit. Every cosign + signature it checks (`SHA256SUMS`, the SBOM and the image) must carry the + exact `release-artifacts.yml@refs/tags/<tag>` identity, as on the v0.3.8 + line, so a signature made by another tag's run is refused. +- The stable release gate (the `vX.Y.Z` tag build) requires the committed + readiness record to show a completed rollback drill of the release's own + candidate (drill alias and image digest), as it already required a + completed incident drill. It also requires the canary run lookups an + attested readiness report records (each canary's run attempt and attempt + start, and the release's publication time), so a record written by an + earlier `production-readiness-report.sh` is refused. + `scripts/release-workflow-gates_test.sh` runs the stable gate and the + promotion steps above against stub tools in CI. ### Fixed - A release-pinned GitHub Action now runs the validator it downloaded and @@ -65,6 +102,26 @@ qualification evidence does not carry over to this candidate. the Action treats an `artifact` input that looks remote (including dot-directories such as `.output/`) as remote even when a workspace file of that name exists; prefix workspace files with `./`. +- `install.sh` without cosign names why gh cannot check a release, instead of + saying gh is missing or that verification FAILED, and still refuses: gh + lacks the attestation command or flags (gh 2.68.0 or later is needed) or + cannot run at all; gh is not logged in or its token is rejected; or the + GitHub API cannot be reached (no response, a 5xx, a 429 or a rate-limited + 403). One read-only API request made before any verification tells these + apart from a refusal; a 5xx or no response there is tried up to three + times, 3 s apart. A refused verification still reads "GitHub attestation + verification FAILED". +- `install.sh` checks the whole release tag (`BPFCOMPAT_VERSION` or the + resolved latest tag): a value that holds a valid tag on one of several lines + is refused before any download, not only by curl. +- The manual verification commands in `README.md` and + `docs/verifying-releases.md` are bound to the exact release tag, as + `install.sh` is: `cosign verify-blob` uses the exact + `release-artifacts.yml@refs/tags/$VER` identity, and `gh attestation verify` + passes `--source-ref "refs/tags/$VER"` (gh 2.68.0 or later). The README's + image check uses the exact signer identity of the pinned image. Before, they + accepted another tag's signature or attestation, and the image check any + signer whose identity contained the repository name. ### Added - A read-only RC preflight inventories observed prerequisites without counting @@ -108,12 +165,45 @@ qualification evidence does not carry over to this candidate. must contain the exact candidate-evidence hash; the Falco run must start after the candidate was built (whether it ran the candidate's validator is recorded, not required); the promotion incident note must name `promote-release` and - link exactly one workflow run (that it is the rejected `promote-release` run - is asserted by the operator, not checked), and an earlier candidate's note is - accepted with that fact recorded. + link exactly one workflow run, and an earlier candidate's note is accepted + with that fact recorded. + - The Falco run's start is the earliest of the report's `run.started_at`, + which the runner writes after the targets ran, and every target's + `started_at`; every required or passing target must record one. + - With attestation verification on, the drill and incident runs are read + from GitHub. The drill must be a successful, dispatched + `rollback-drill.yml` run of this repository on `main` that started after + the candidate was built, with the note's completion time inside it. The + incident must link a completed, dispatched `promote-release.yml` run of + this repository that did not succeed; which rejection qualifies is the + operator's call. + - With attestation verification on, every release-candidate canary run is + looked up. The run must be a successful `release-candidate-canary.yml` run + of this repository from the expected event (the timed ones on `main`). The + evidence must come from a successful attempt in which the evidence job + itself ran. That attempt must start after the release was published (read + from the release, which must be published and not predate the candidate + evidence) and, for T+24h and T+72h, before the plan job's window closes: + a partial re-run of a failed canary started after its window closed is + refused, and one started inside it is accepted. GitHub reports a run's + conclusion from its latest attempt, so a failed re-run of a successful + canary voids it. The attempt, its start and the run's branch are recorded, + with the release's publication time. - CI: top-level workflow tokens are read-only; the hosted compatibility gate is required for `pkg/` and `scripts/` changes; untrusted-input parsers have fuzz coverage. +- The stable release can pass its own consistency checks. While + `release.yaml` is on the `stable` channel and `vX.Y.Z` positively does not + exist (the stable pull request) or resolves to the commit under check (its + tag build), the consumer canary's `# bpfcompat-pin: stable` step may still + pin the most recent published stable release before `X.Y.Z`, and the + SHA-pinned integration templates may stay on that release; tag-form pins in + the docs may name `vX.Y.Z` while its tag does not exist. Every later commit + is strict again, so both are repinned once the tag exists, and a failed + lookup fails the check. `check-release-consistency_test.sh` reads the + channel and stable version from `release.yaml`. The runbook lists the + stable-only tag gates and the order: operator record, readiness report, + stable pull request, tag, promotion. - **Breaking** (CLI, API and `pkg/bpfcompat`; the Action already required an explicit host): implicit Docker Hub short refs in tag form, such as `org/name:tag` or `name:tag` (e.g. `alpine:3`), are no longer pulled. Spell @@ -148,6 +238,19 @@ qualification evidence does not carry over to this candidate. release publishes one (default `<BIN_DIR>/../share/doc/bpfcompat`; set `BPFCOMPAT_DOC_DIR` to change it). If that directory is not writable, the installer uses sudo or stops before installing anything. +- GitHub Action pinned to a release with `prebuilt: auto` (the default): the + prebuilt binaries now need gh 2.68.0 or later on the runner, and a tag pin + also needs `git` and `timeout` to resolve the tag within 60 seconds. + Otherwise the run builds from the pinned source with a warning, which needs + the source-build dependencies. With the v0.4.0-rc.4 Action, self-hosted + runners on gh 2.51 to 2.67 used the prebuilt binaries, and on gh 2.47 to + 2.50 the job failed. A SHA pin needs neither `git` nor `timeout`. +- Callers of `scripts/verify-release-assets.sh` must set + `BPFCOMPAT_EXPECTED_SOURCE_SHA` and `BPFCOMPAT_EXPECTED_SOURCE_REF`; without + them it exits 2. +- A production readiness record generated with an earlier + `production-readiness-report.sh` must be regenerated: the stable tag build + refuses it. ### Known limitations - When this section was drafted, every stable-blocking decision in