[ErenAri]

Draft. Merge only on an exact-head operator decision. This changes the live installer: scripts/install.sh on main is what every curl … /main/scripts/install.sh | sh user runs. Every refusal stays fail-closed, and cosign is still preferred whenever it is installed.

Hosted CI status (2026-10-19T18:47Z): the check runs on this head did not start. GitHub reports "The job was not started because your account is locked due to a billing issue." They are not code results; this head has no hosted CI result yet. All results below are LOCAL unless stated otherwise.

Final head: 69cebeae6791d3c8f016cdb0a10ed3b68bce0778 (5 commits on main 402d7c9). All five commits are pushed (723b5e2, 6705bb1 and 69cebea on 2026-10-19 after the earlier f8809a9); see the hosted-CI note above.

Why

A LOCAL smoke of main's installer at 402d7c9 against the real v0.3.7 and v0.4.0-rc.4 releases found:

• cosign path: works end to end. Both installs pass, and a cross-tag substitution and a tampered SHA256SUMS are refused.
• F1: with gh older than 2.47 (no gh attestation; it is hidden in 2.47–2.48 and public from 2.49) and no cosign, every install is refused with "requires cosign or GitHub CLI (gh)", although gh is installed. Ubuntu noble's packaged gh is 2.45.0.
• F2: the installer checked only that gh attestation exists, but its verify call also needs --signer-workflow (gh ≥ 2.51.0) and --source-ref (gh ≥ 2.68.0). These minimums were checked in gh's verify.go at those tags. On gh 2.47–2.67 the user saw "GitHub attestation verification FAILED", which reads like tampering. The 2.51–2.67 part is new since install.sh: bind signatures to the requested tag, validate the version, install atomically #236; rc.4's installer did not pass --source-ref.
• F3: the tag check passed if any line of BPFCOMPAT_VERSION matched, so a value with an embedded newline got through to curl.

Later reviews and triage added:

• S2 (review of f8809a9): a gh that is new enough but cannot complete the check still ended in "GitHub attestation verification FAILED", with gh's output sent to /dev/null: a gh that is not logged in, holds a rejected token, or cannot reach the GitHub API. Reproduced LOCAL with real gh 2.102.0 and f8809a9's installer.

• N7 (review of 723b5e2): the RC canary's clean-install job runs the candidate commit's install.sh on the gh path with github.token. After 723b5e2, one transient 5xx or dropped connection at the new API probe failed that time-windowed job, although gh's own verify retries a 5xx.

• T4 / F4 (triage): on the gh path SHA256SUMS is unsigned, and gh attestation verify matches an artifact by digest only. A party able to change release assets (a TLS interceptor against a separately obtained installer, an asset-write credential against pinned-version users, or an honest bad re-upload) could, with every verify call passing:

  • T4-S1: delete the validator line, giving a CLI-only install and a false "no prebuilt validator" note;
  • T4-S2: delete the license-bundle line (rc.5 and later), giving no license texts and a false "publishes no third-party license bundle" note;
  • T4-S3: serve the same tag's arm64 CLI as bpfcompat-linux-amd64;
  • T4-S4: serve the CLI's bytes as the validator.

  Reproduced LOCAL with real gh 2.68.0 and 2.102.0 and 723b5e2's installer: T4-S1, T4-S3 and T4-S4 all exit 0. The cosign path refuses all of them.

Changes

cc36c70: name the missing gh capability (pushed)

• gh path, runs only when cosign is absent: before verifying, the installer asks gh's own help (gh attestation --help, gh attestation verify --help) whether the command and both flags exist. No version parsing.
  • If something is missing, it refuses and names it. Example: installed gh 2.67.0 lacks --source-ref in 'gh attestation verify', needed for tag-bound attestation verification; upgrade to gh >= 2.68.0 or install cosign - refusing checksum-only installation.
  • "requires cosign or GitHub CLI (gh)" now appears only when neither tool is installed.
• Tag check: an explicit allowlist case now runs before the existing regex, so any newline or other character a tag cannot contain is refused before any download.

f8809a9: review follow-ups (S1, N1, N2) (pushed)

• A gh that cannot run (for example an unreadable ~/.config/gh/config.yml) is reported as installed gh could not run 'gh --help' (<gh's first error line>); fix gh or install cosign - refusing checksum-only installation, not as lacking a command.
• Comment: gh attestation exists, hidden, from gh 2.47.0 (public from 2.49.0).
• Tests: the gh stub records every verify call; the gh-path install cases assert that exactly the installed assets were verified, each bound to the release workflow and --source-ref refs/tags/<tag>; new cases swap only the CLI, or only the validator, for another tag's attested bytes.

723b5e2: separate auth and API failures from a refused verification (S2)

On the gh path the installer reports four distinct outcomes. Every one refuses: exit 1, nothing installed, temp dir removed, no checksum-only fallback.

Class	When	Message
1	gh lacks the command or flags, or cannot run	installed gh <ver> lacks …, or installed gh could not run 'gh --help' (…)
2	no usable credentials	GitHub authentication needed for attestation verification: gh is not logged in (<gh's first line>), or … GitHub rejected gh's credentials (HTTP 401) (…); then run 'gh auth login' or set a valid GH_TOKEN, or install cosign - refusing checksum-only installation
3	the GitHub API is out of reach	GitHub API unavailable for attestation verification: no HTTP response | HTTP 5xx | HTTP 429, rate limited | HTTP 403, rate limited (<gh's first line>); check the network or proxy and https://www.githubstatus.com, then retry, or install cosign - refusing checksum-only installation
4	anything else: no attestation for the digest (404), another signer workflow or source ref, a digest mismatch, a verify exit other than 0 or 4, anything unclassifiable, and the provenance mismatches of 69cebea	GitHub attestation verification FAILED for <asset>, followed for 69cebea's checks by the mismatch

• How classes 2 and 3 are decided: only on positive evidence from a read-only request made after the capability check and before the first verify call: gh api --method GET --include repos/<repo>/attestations/sha256:<CLI digest>?per_page=1. That is the endpoint the first verify queries, through the same gh, host, token and proxy.
  • exit 4 is gh's documented auth exit code. Its only source is the root auth check, which runs before attestation verify reads the asset or touches the network (only --bundle, never passed, lifts it).
  • With --include, the first stdout line is the status line gh builds from the response (HTTP/x NNN; HTTP/1.1 and HTTP/2.0 both parse).
  • A transport failure exits 1 with empty stdout.
  • A 403 counts as rate-limiting only with GitHub's X-Ratelimit-Remaining: 0 or a Retry-After header.
  • Any other answer (200, 404, a plain 403, other codes, unparsable output, an exit other than 1 or 4 with no output) leaves the decision to the verify calls.
• gh's verify text is never parsed: with stdout not a terminal gh prints only its final error, whose wording differs between 2.68 and 2.102 for the same 401 or 404, and on 2.102 a dead proxy fails before the API fetch with a Sigstore message.
• Probe before verifying, not after a failure: so a refusal can never be relabelled as an auth or network problem (for example when the verify itself spends a nearly exhausted rate limit, or a token expires in between).
• gh's words are quoted as one line, and every capture ends in || : so an imported pipefail cannot turn a refusal into a silent exit.

6705bb1: retry the probe on a 5xx or no response only (N7); printable ASCII only (N2); pin the untested branches (N1)

• Retry: up to three probe attempts, 3 s apart (6 s of added waiting at most), only for an HTTP 5xx or no HTTP response; the last attempt decides. Still fail-closed: after three, class 3.
• Not retried:
  • 401, 404 and 200 are answers.
  • A 429 or a rate-limited 403: GitHub asks the client to wait (Retry-After, X-Ratelimit-Reset, typically a minute or more), longer than an installer should; asking again within seconds would only spend more of the limit (and of a secondary limit's patience).
  • gh's auth exit code 4.
• N2: text quoted from gh keeps printable ASCII only, an allow-list: LC_ALL=C tr -cd '\040-\176'. C1 controls (a raw CSI byte 0x9b, or its UTF-8 form c2 9b) no longer pass. The octal range is used rather than [:print:] because busybox tr 1.36.1 does not honour [:print:] (observed: tr -cd '[:print:]' keeps only the [ of a test line, while [:alnum:] works), which would blank gh's words on busybox systems such as Alpine.
• N1: tests now pin a probe exit other than 1 or 4 with no output, exit 1 with output that is no status line, HTTP/1.1 status lines, and a verify exit other than 0, 1 or 4 (class 4).

69cebea: bind gh-path installs to the release's attested names (T4 remedy R-B)

• The CLI's verify call also prints its verified provenance's subjects: --format json --jq '.[].verificationResult.statement.subject[] | "\(.digest.sha256) \(.name)"', to a private file in the installer's 0700 temp dir. These lines name, with digests, every asset the tag's release run attested in that statement. The call's stderr is still captured separately for the exit-4 classification.
• Then, after all verify calls pass, the installer refuses as class 4, naming the mismatch:
  • when SHA256SUMS and those subjects disagree on whether the validator or the license bundle exists (it decides what is downloaded):
    • GitHub attestation verification FAILED for bpfcompat-validator-static-linux-amd64: SHA256SUMS does not list it, but the v0.3.7 release provenance of bpfcompat-linux-amd64 does
    • … FAILED for bpfcompat-validator-static-linux-arm64: SHA256SUMS lists it, but the v0.3.7 release provenance of bpfcompat-linux-arm64 does not
  • unless each file to be installed has its exact <sha256> <name> pair among the subjects:
    • GitHub attestation verification FAILED for bpfcompat-linux-amd64: the v0.3.7 release provenance of bpfcompat-linux-amd64 does not list these bytes under that name (it lists them as bpfcompat-linux-arm64)
• Every existing release still installs: each tag's release-artifacts.yml attests the CLIs and the validator (and, from rc.5, the bundle) in one attest-build-provenance step (release-artifacts.yml:254-268@402d7c9), and SHA256SUMS lists the validator and the bundle from a fixed list (:229-247). Re-run releases pass: membership is checked against the union of every verified statement holding the CLI's digest.
• Unchanged: the cosign path, every verify call's argv apart from the CLI's two output flags, the checksum step, the sudo decisions and the output paths.

Validation

All of this is LOCAL evidence on the writer's host (2026-10-19). Hosted CI for this head has not run (account billing lock; see the note at the top).

Stub suite (scripts/install_test.sh):

Head	dash as sh	bash as sh	busybox as sh
6705bb1	98/98	98/98	94/98
69cebea	110/110	110/110	104/110

• busybox: the failures are harness-only. busybox runs its own applets (uname, sha256sum, tar) instead of the PATH shims: the 4 known cases (aarch64 mapping, unsupported arch, missing sha tools, license bundle without tar) plus, at 69cebea, the 2 aarch64-host R-B cases. busybox's own sleep likewise bypasses the sleep shim, so under busybox the retry cases check the attempt counts but not the waits (the suite prints a note).
• The 80 cases of 723b5e2 are all present and pass; their assertions are unchanged or strengthened (the outage cases now expect exactly three probe attempts and the waits between them; the unattested-bundle case now requires the bundle's own verify call to refuse it).
• RED before GREEN:
  • 69cebea's suite against 723b5e2's script: 27 failures; against 6705bb1's script: 13 failures, all R-B cases (the 12 new ones and the probe case's subjects request).
  • 6705bb1's suite against 723b5e2's script: 14 failures. The 7 N1 cases that pass there pin behaviour that was already correct; their RED is the round-1 surviving mutants, now caught (below).
• New cases (30):
  • retry (8): 503 then 200; 502, 503, then 200 on the third attempt; no response then 200; 503 then 401 (class 2), 429 or 404; no response then a rate-limited 403 or a refusal. Each checks the exact attempts and the waits. Every single-attempt case (401, 404, 200, a plain 403, 429, rate-limited 403s, exit 4) now also asserts exactly one attempt and no wait.
  • N1 (8) and N2 (2, a C1 CSI raw and UTF-8, in both quoting places).
  • R-B (12): refused: validator line deleted (v0.3.7 and rc.5 shapes), bundle line deleted, arm64 CLI served as amd64, CLI served as the validator, an arm64 validator line added on an aarch64 host (served with the amd64 validator's bytes), and bytes from another statement of the same run served as the validator or as the CLI; each also refused by cosign. Installed: CLI-only, an aarch64 host, and a re-run release with two statements. Plus an imported-pipefail refusal. Every refusal asserts: nothing installed, temp dir removed, the exact class-4 line, no auth or network wording, every verify call made and passed, and only the CLI's call requesting its subjects.
• The gh stub answers --format json --jq <that exact expression> with the <sha256> <name> subjects of every attested statement holding the verified digest (it refuses any other expression), and takes one probe attempt at a time from a sequence; more than 10 probe attempts in one run stop with exit 4, so an unbounded retry fails fast.

Mutations (scratch copies of install.sh, never committed; exact-text replacements with asserted match counts; each run under dash with ok + FAIL = 110, so every catch is a named assertion): 65 of 66 caught.

• All 34 round-1 mutants (re-expressed where the text moved), the round-1 review survivors R09, R12, R13, R14 and R22 (verify stderr dropped), plus: subject check removed; presence-disagreement check removed; name ignored (digest-only); --format/--jq dropped; retry unbounded; retry applied to 404; allow-list reverted to tr -d '[:cntrl:]'; and 20 more (retry on 429 or rate-limited 403, retry dropped for 5xx or no response, 2 or 4 attempts, no wait or a 30 s wait, first answer kept, presence checked one way only or for one asset only, names checked for the CLI only or not for the CLI, the "lists them as" clause dropped, presence read from SHA256SUMS twice, high bytes kept, stderr merged into the subjects, and [:print:] in place of the octal range).
• The one survivor under dash, [:print:] instead of the octal range, is equivalent with GNU tr; under busybox it is caught (19 failures beyond the 6 harness-only ones).

Real gh (LOCAL): official linux_amd64 tarballs of gh 2.68.0 and 2.102.0, checked against their releases' checksums files; isolated HOME, XDG dirs and GH_CONFIG_DIR; no cosign on PATH; a sudo guard; the committed install.sh (69cebea, sha256 e75646b8…c5e0) fed on stdin under env -i; real release bytes served from a local file:// mirror so only gh used the network; the token taken from the host's existing login, passed only through the environment, never printed or stored.

Case	gh 2.68.0	gh 2.102.0
real v0.3.7	installed, bytes = SHA256SUMS	installed (also under busybox)
real v0.4.0-rc.4	installed	installed
aarch64 host (uname shim), real v0.3.7	installed, arm64 CLI only	same
T4-S1: validator line deleted (v0.3.7; and rc.4)	refused, class 4	refused, class 4
T4-S3: real arm64 CLI served as amd64	refused, … (it lists them as bpfcompat-linux-arm64)	same (also under busybox)
T4-S4: real CLI served as the validator	refused, … (it lists them as bpfcompat-linux-amd64)	same
aarch64 host, arm64 validator line added (amd64 bytes)	refused, SHA256SUMS lists it, but …	same
no token	class 2, gh is not logged in (…)	same
token + dead proxy	class 3, no HTTP response (…), 3 attempts	same
invalid token	class 2, GitHub rejected gh's credentials (HTTP 401) (gh: Bad credentials (HTTP 401))	same

• Before: 723b5e2's installer with the same real gh exits 0 on T4-S1, T4-S3 and T4-S4 at both versions.
• In every case the installer's temp dir was left empty, and nothing was installed except in the "installed" rows.
• The --jq output shape: real --format json output at both versions is one result per verified statement with verificationResult.statement.subject[].{name,digest.sha256}; the --jq stdout is exactly the three v0.3.7 subject lines, byte-identical with CLICOLOR_FORCE=1 and with GH_DEBUG=api. From source, at both versions: the exporter writes only the verified results (pkg/cmd/attestation/verify/verify.go:258-264@v2.68.0, :309-315@v2.102.0); sigstore-go v0.7.0 / v1.3.0 marshal the statement with protojson under statement; in-toto v1.1.1 / v1.2.0 name the fields subject, name, digest; and go-gh v2.11.2 / v2.16.1 print jq string results raw, never coloured.
• Not produced for real: a GitHub 5xx, 429 or rate-limited 403 (stub only), a tampered license bundle on a real release (no real release has one yet; stub only), and any gh between 2.68.0 and 2.102.0.

Also LOCAL:

• Real cosign path (cosign v2.4.1, real gh 2.102.0 also on PATH, 69cebea): v0.3.7 installs with "cosign signature over SHA256SUMS verified", identity bound to refs/tags/v0.3.7; gh is never run.
• Real gh 2.45.0 (scratch runs at f8809a9; the capability check is unchanged since): with a valid config, the precise "lacks the 'gh attestation' command" refusal; with a malformed config.yml, "could not run 'gh --help' (…)". The API probe never runs on 2.45, because the capability check refuses first.
• Lint: shellcheck 0.9.0 is clean on both files at -S warning and at default severity, and on install.sh with -s dash. install.sh parses under dash -n, bash --posix -n, bash -n, busybox sh -n and busybox ash -n.
• Local replay of ci.yml's test and build jobs at 69cebea: 40 steps ran, 0 failed; 1 skipped (go mod download, vendored offline replay). scripts/check-experimental-freeze.sh origin/main: PASS.

Residuals (not changed here)

• Hosted proof is still missing. Every real run above is LOCAL. The triage verifier recommends a hosted real-gh check (the runner's default gh, plus 2.68.0) against the real v0.3.7 and rc.4 assets before this merges: main's copy goes live for every gh-only user at merge, and if some gh version printed a different JSON shape, every gh-only install would be refused (fail-closed, but an outage). The RC canary's clean-install job is the first hosted run of this gh path.
• Timing (triage verifier): the RC canary runs the candidate commit's installer. If rc.5 is tagged from a commit without this change, the rc.5 canary qualifies the old gh path while users run main's copy. Landing this in the rc.5 tag commit avoids that; it is a sequencing decision for the operator.
• New coupling: the CLI, the validator and the license bundle must stay in one provenance statement (release-artifacts.yml:254-268@402d7c9). Splitting them later makes every gh-path install refuse (fail-closed). A comment or check there belongs in a later, non-frozen change.
• Other paths of the same class, outside this PR's files: the GitHub Action reads the unsigned SHA256SUMS for bundle presence and the validator pin (action.yml:207, :233@402d7c9) and verify-release-assets.sh matches by digest; README's manual prebuilt instructions also verify by digest only (README :386-408@402d7c9, :390-415@2269647). R-B covers neither.
• A non-github.com default gh host (review N3): with GH_HOST set, or a single GHES host configured, the probe and the verify both go to that host. A reachable GHES answers 404, so the verify fails with ErrUnsupportedHost and the install reads class 4 "FAILED"; or a bad GHES token gives class 2 without naming the host. This predates the change and is fail-closed, but it misdirects like S2 did. Option, not implemented because it is a behaviour choice: pin GH_HOST=github.com on the probe and the verify calls (it changes their environment, not their argv, and overrides a user's deliberate GH_HOST).
• Some 403s still read "FAILED" (review N4): a 403 without rate-limit headers stays class 4. That covers a token without attestation read access (for example a fine-grained token), SSO enforcement, and a secondary rate limit sent without Retry-After while X-Ratelimit-Remaining is non-zero.
• Outages the probe cannot see are reported as class 4, the documented "ambiguous means refusal" direction: the Sigstore or GitHub TUF repositories (which gh 2.102 contacts before the API) or the bundle storage being down while the API answers, and an outage that starts after the probe.
• Wait time: the retry adds at most 6 s of sleep, but each attempt is bounded only by gh's own HTTP limits (gh sets no overall request timeout, from source), so a black-holed connection now waits up to three times as long before class 3.
• Each gh-path install makes 1–3 more API requests than before this PR, counted against the user's rate limit.
• BPFCOMPAT_NO_VALIDATOR=1 does not skip the comparison: a SHA256SUMS that disagrees with the provenance about the validator is refused even when the validator is not wanted.
• gh ≥ 2.68.0 is the minimum that works. gh advisories fixed in 2.97 and 2.102 weaken the workflow and tag matching before those versions, and raising the floor would need version parsing. That is a design decision.
• The README and docs/verifying-releases.md state the gh ≥ 2.68.0 minimum in docs: bind manual release verification to the exact tag and mark rc-only behaviour #267 (head 2269647), not here.
• No CHANGELOG entry, to avoid conflicting with release: prepare v0.4.0-rc.5 metadata and an offline, AST-based RC preflight (replaces #264) #265. Suggested lines are in the run record.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}