diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3d64282..0183d83 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -108,6 +108,10 @@ jobs: # missing or inconsistent inputs; open distribution decisions withhold # optional assets and block stable builds. run: bash scripts/distribution-bundle_test.sh + - name: RC preflight regression + # The read-only release-candidate inventory reports VERIFIED, MISSING + # or PENDING from observed state only; lookup failures abort it. + run: bash scripts/rc-preflight_test.sh - name: Image prefetch reliability regression # The public matrix lane lost five of seven weekly runs (Aug-Oct 2026) # to a single failed image download aborting the whole prefetch. These diff --git a/CHANGELOG.md b/CHANGELOG.md index bcf9636..8dcd0e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,21 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a ## [Unreleased] +## [0.4.0-rc.5] - TBD (date set when the tag is cut) + +Release candidate for v0.4.0: the changes merged since v0.4.0-rc.4. rc.4 +qualification evidence does not carry over to this candidate. + ### Security +- `install.sh` binds verification to the requested release: the cosign + certificate identity and the GitHub attestation source ref must name exactly + that tag, so a signature or attestation made for another release (for example + a prerelease) no longer vouches for it. `BPFCOMPAT_VERSION` and the resolved + latest tag must look like `vX.Y.Z` or `vX.Y.Z-rc.N`; the installer body runs + only when the whole script was received; whether sudo is needed is decided per + target directory before anything is installed. (The installer is served from + `main`, so this already applies to `curl … /main/scripts/install.sh | sh` + users since #236 merged, including v0.3.7 installs.) - The GitHub Action no longer pastes inputs into its `run:` scripts. Every input reaches the scripts through `env:` and is read as quoted data, so a value containing quotes, `$(...)`, backticks or newlines can no longer run as @@ -15,7 +29,8 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a (`scripts/check-action-input-interpolation.sh`) and an end-to-end injection test keep it that way. - Values containing `$VAR`, backticks or `\` used to be expanded by bash as - a side effect; they are now passed literally. + a side effect; they are now passed literally. `command` is unchanged: it + was already passed through `env:` and is still run by the guest shell. ### Fixed - A release-pinned GitHub Action now runs the validator it downloaded and @@ -32,31 +47,44 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a with no record of which kernel ran, and a required target could pass. - A missing local `--artifact` path is no longer treated as a registry reference, unless it reads like one: a relative path without a `./`, `../`, - `/` or `~` prefix that does not end in `.bpf.o` and that either has a - host-like first component (it contains `.` or `:`, or is `localhost`) or - contains `@sha256:`. A path with one of those prefixes is never pulled. + `/` or `~` prefix that does not end in `.bpf.o` and that either contains a + `/` after a host-like first component (it contains `.` or `:`, or is + `localhost`) or contains `@sha256:`. A path with one of those prefixes is + never pulled. - Before, any non-existent path that happened to parse as an image reference - (`/home/u/prog.bpf.o`, `build/prog.bpf.o`, `./prog.bpf.o`) was pulled from - Docker Hub. That leaked the local path and failed with a registry error. + (`/home/u/prog.bpf.o`, `build/prog.bpf.o`, `./prog.bpf.o`) was treated as a + registry reference (Docker Hub for most, e.g. `build/prog.bpf.o`). That + leaked the local path and failed with a registry error. - It now fails locally with `artifact "": no such file`. This covers `test`, `test-command`, `suite`, the API and `pkg/bpfcompat.Validate`. - - The CLI now classifies remote refs like the Action's `artifact` input: - the first path component must be a registry host (contains `.` or `:`, or - is `localhost`), or the ref must pin an `@sha256:` digest. - - The Action treats an `artifact` input that looks remote (see - `docs/compatibility-contract.md`, including dot-directories such as - `.output/`) as remote even when a workspace file of that name exists; - prefix workspace files with `./`. + - The CLI now classifies remote refs like the Action's `artifact` input, but + stricter in the safe direction (see `docs/compatibility-contract.md`): the + first path component must be a registry host (contains `.` or `:`, or is + `localhost`), or the ref must pin an `@sha256:` digest. + - Unchanged behaviour, now documented in `docs/compatibility-contract.md`: + the Action treats an `artifact` input that looks remote (including + dot-directories such as `.output/`) as remote even when a workspace file of + that name exists; prefix workspace files with `./`. ### Added +- A read-only RC preflight inventories observed prerequisites without counting + pending qualification as a pass. Candidate Action pins and the canary's + candidate env are read from actual workflow YAML nodes; comments, shell-script + text and steps or jobs that can never run (a constant `if` other than + `true`, such as `false` or `0`, or a job that needs one of those; an empty + `if` is no condition) cannot establish them. Conditions that read a context + or call a function are left to the runner. - Releases carry the third-party license material of their binaries as signed assets listed in `SHA256SUMS`: `bpfcompat-third-party-licenses.tar.gz` (verbatim license, notice and patent texts of every linked Go module, the Go toolchain and the validator's native components, with an `INVENTORY.json` generated from the shipped binaries) and `bpfcompat-validator-relink-kit-linux-amd64.tar.gz` (the validator's `main.o`, - link commands, linker trace and map, inputs with owning packages, sources and - `relink.sh`). `validator-build-inputs.json` is now a release asset too. + link commands, linker trace and map, inputs with owning packages, the + validator's own C sources and `relink.sh`, which installs the locked library + packages from snapshot.ubuntu.com). The static archives and the native source + packages are not included while their distribution decisions are undecided. + `validator-build-inputs.json` is now a release asset too. - `install.sh` and the GitHub Action install the license bundle, verified with the binaries, when a release lists it; releases without it install as before. The container image carries the CLI's texts in @@ -66,10 +94,72 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a refuses while a stable-blocking decision is undecided. ### Changed -- **Breaking:** implicit Docker Hub short refs in tag form, such as - `org/name:tag`, are no longer pulled. Spell them with the host, e.g. - `docker.io/org/name:tag`; the "no such file" error suggests that spelling. - Digest-pinned short refs (`org/name@sha256:…`) are still pulled. +- Release build: the static validator is built inside a digest-pinned Ubuntu + builder from `validator/build-inputs.lock` (exact package versions from a + fixed snapshot.ubuntu.com state). While the validator sources are the tree the + lock records, the build fails unless the result is byte-identical to that + recorded reproduction. A stable release must ship the same validator bytes as + the release candidate it graduates. +- Release gate: each VM gate report is evaluated on its own and the gate fails + closed on a missing or malformed report. +- Production readiness binds more evidence to the exact candidate: inputs must + be single-object JSON; the rollback drill must use the candidate's alias and + image digest and complete after the candidate was built; the operator note + must contain the exact candidate-evidence hash; the Falco run must start after + the candidate was built (whether it ran the candidate's validator is recorded, + not required); the promotion incident note must name `promote-release` and + link exactly one workflow run (that it is the rejected `promote-release` run + is asserted by the operator, not checked), and an earlier candidate's note is + accepted with that fact recorded. +- CI: top-level workflow tokens are read-only; the hosted compatibility gate is + required for `pkg/` and `scripts/` changes; untrusted-input parsers have fuzz + coverage. +- **Breaking** (CLI, API and `pkg/bpfcompat`; the Action already required an + explicit host): implicit Docker Hub short refs in tag form, such as + `org/name:tag` or `name:tag` (e.g. `alpine:3`), are no longer pulled. Spell + them with the host, e.g. `docker.io/org/name:tag`; the "no such file" error + suggests that spelling. Digest-pinned short refs (`org/name@sha256:…`) are + still pulled. + +### Migration notes +- Docker Hub images (CLI, API, `pkg/bpfcompat`): write `docker.io/org/name:tag` + instead of `org/name:tag`, and `docker.io/name:tag` instead of `name:tag`. + The "no such file" error suggests that spelling. +- GitHub Action (unchanged, now documented): an `artifact` input whose first + path component contains `.` or `:` (for example `.output/` from + libbpf-bootstrap, or `out.v2/`) is passed to the CLI as a remote reference. + Prefix workspace files with `./`. In this release such a missing `.bpf.o` + path fails with "no such file" instead of a registry error. +- Action path and option inputs (`artifact`, `matrix`, `out`, `manifest`, + `markdown`, `workdir`, `suite*`, `binary`, `timeout`, `concurrency`, + `disk-resize`, `validation-mode`, `keep-vm-on-failure`) containing `$VAR`, + backticks or `\` are now passed literally. For example, + `out: $RUNNER_TEMP/r.json` must become `out: ${{ runner.temp }}/r.json`. + `command` is unchanged: it is still run by the guest shell, so + `$BPFCOMPAT_BIN`, `$BPFCOMPAT_ARTIFACT` and `$BPFCOMPAT_REMOTE_ROOT` work as + before. +- Command mode: a target whose guest kernel release cannot be read is now + `INFRA_ERROR` (and fails a required run) instead of passing or failing. +- Self-hosted runners with a validator in `/usr/local/libexec` or + `/usr/libexec`: a release-pinned Action now runs the validator it downloaded; + set `BPFCOMPAT_VALIDATOR_BIN`/`BPFCOMPAT_VALIDATOR_SHA256` in the workflow to + keep using another one (the run then warns). +- `install.sh` installs the license bundle under `//` when the + release publishes one (default `/../share/doc/bpfcompat`; set + `BPFCOMPAT_DOC_DIR` to change it). If that directory is not writable, the + installer uses sudo or stops before installing anything. + +### Known limitations +- When this section was drafted, every stable-blocking decision in + `distribution/policy.json` was undecided. The license bundle and relink kit + record facts and ship texts verbatim; they do not elect a license or state + that any license condition is met. A stable release is refused until the + stable-blocking decisions are recorded. +- The prebuilt static validator is published for x86_64 only. +- See "Known limitations of the supported path" in + `docs/compatibility-contract.md` (artifact-path classification, untrusted + names in reports, classification codes, Action binary precedence, shared + hosts and SSH). ## [0.4.0-rc.4] - 2026-10-14 diff --git a/internal/tools/rcpreflight/main.go b/internal/tools/rcpreflight/main.go new file mode 100644 index 0000000..bb9f6d1 --- /dev/null +++ b/internal/tools/rcpreflight/main.go @@ -0,0 +1,574 @@ +// Command rcpreflight checks active candidate Action pins, workflow-level env +// bindings and the timed canary schedule in workflow YAML. +// It reads local files only; the shell inventory builds it from vendored code. +package main + +import ( + "errors" + "fmt" + "io" + "os" + "regexp" + "strconv" + "strings" + "time" + + "gopkg.in/yaml.v3" +) + +const target = "kernel-guard/bpfcompat" +const marker = "bpfcompat-pin: candidate" + +func main() { os.Exit(run(os.Args[1:], os.Stdout, os.Stderr)) } + +// run prints VERIFIED or PENDING and returns 0 for a workflow it could judge. +// A usage error or an unreadable file is not a judgement about the workflow: +// it prints nothing to stdout and returns 2, so the caller aborts instead of +// reporting a status. +func run(args []string, stdout, stderr io.Writer) int { + if len(args) != 4 || (args[0] != "all" && args[0] != "candidate" && args[0] != "env" && args[0] != "schedule") { + fmt.Fprintln(stderr, "usage: rcpreflight \n rcpreflight env \n rcpreflight schedule - ") + return 2 + } + // #nosec G304 G703 -- this local CLI intentionally reads the operator-supplied workflow path; it never writes it. + f, err := os.Open(args[3]) + if err != nil { + fmt.Fprintln(stderr, "rc-preflight pins:", err) + return 2 + } + defer f.Close() + if err := check(f, args[0], args[1], args[2]); err != nil { + fmt.Fprintln(stderr, "rc-preflight pins:", err) + fmt.Fprintln(stdout, "PENDING") + return 0 + } + fmt.Fprintln(stdout, "VERIFIED") + return 0 +} + +func commentsHave(s, want string) bool { + for _, line := range strings.Split(s, "\n") { + if strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "#")) == want { + return true + } + } + return false +} + +// keywords are the workflow keys this checker reads. GitHub matches them +// case-sensitively, so a case variant would be ignored here while a YAML tool +// might honour it. They are checked only where GitHub reads its own keywords +// (the workflow, each job and each step); env names, inputs and matrix keys +// are user data and may be spelled any way. Merge keys ("<<"), which GitHub +// does not support, are rejected everywhere. +var keywords = map[string]bool{"jobs": true, "steps": true, "uses": true, "if": true, "env": true, "needs": true} + +func keywordCase(n *yaml.Node) error { + for i := 0; i < len(n.Content); i += 2 { + if k := n.Content[i].Value; keywords[strings.ToLower(k)] && !keywords[k] { + return fmt.Errorf("case-variant key %q at line %d", k, n.Content[i].Line) + } + } + return nil +} + +// validate rejects ambiguous keys and aliases rather than guessing which use +// a runner would execute. Comment markers inside run strings are not comments. +func validate(n *yaml.Node) (int, error) { + if n.Kind == yaml.AliasNode { + return 0, fmt.Errorf("YAML aliases require manual verification") + } + count := 0 + for _, s := range []string{n.HeadComment, n.LineComment, n.FootComment} { + for _, line := range strings.Split(s, "\n") { + if commentsHave(line, marker) { + count++ + } + } + } + if n.Kind == yaml.MappingNode { + keys := map[string]bool{} + for i := 0; i < len(n.Content); i += 2 { + k := n.Content[i] + if k.Kind != yaml.ScalarNode || keys[k.Value] { + return 0, fmt.Errorf("non-scalar or duplicate mapping key at line %d", k.Line) + } + if k.Value == "<<" || k.Tag == "!!merge" { + return 0, fmt.Errorf("merge key at line %d", k.Line) + } + keys[k.Value] = true + } + } + for _, child := range n.Content { + c, err := validate(child) + if err != nil { + return 0, err + } + count += c + } + return count, nil +} + +func field(n *yaml.Node, name string) (*yaml.Node, *yaml.Node) { + if n.Kind == yaml.MappingNode { + for i := 0; i < len(n.Content); i += 2 { + if n.Content[i].Value == name { + return n.Content[i], n.Content[i+1] + } + } + } + return nil, nil +} + +func asciiEqualFold(a, b string) bool { + for i := 0; i < len(a); i++ { + if a[i] >= 0x80 { + return false + } + } + return strings.EqualFold(a, b) +} + +var ( + quoted = regexp.MustCompile(`'(?:[^']|'')*'`) + identifier = regexp.MustCompile(`(?:^|[^A-Za-z0-9_.])([A-Za-z_][A-Za-z0-9_-]*)`) + resume = regexp.MustCompile(`(?i)\balways\s*\(|!\s*cancelled\s*\(`) +) + +// condition reads the if of a job or step. An if that is empty or YAML null +// is no condition: GitHub runs it as success(). never is true when the +// condition is a constant other than true: it names no context and calls no +// function, so the runner can only ever skip it. GitHub treats false, 0, -0, +// the empty string, null and NaN as false and accepts any JSON number form, +// so every constant except true (keywords are case-insensitive) is treated as +// never running instead of being evaluated here. A condition that reads a +// context or calls a function is the runner's to evaluate (the canary jobs +// all depend on their plan job's output). resumes is true when the condition +// calls always() or !cancelled(), which can run a job after a job it needs +// was skipped; success() and failure() are both false then. +func condition(n *yaml.Node) (never, resumes bool, err error) { + _, cond := field(n, "if") + if cond == nil { + return false, false, nil + } + if cond.Kind != yaml.ScalarNode { + return false, false, fmt.Errorf("if must be a scalar at line %d", cond.Line) + } + v := strings.TrimSpace(cond.Value) + if cond.Tag == "!!null" || v == "" { + return false, false, nil + } + if strings.HasPrefix(v, "${{") && strings.HasSuffix(v, "}}") { + v = strings.TrimSpace(v[3 : len(v)-2]) + } + code := quoted.ReplaceAllString(v, "''") + for _, m := range identifier.FindAllStringSubmatch(code, -1) { + switch strings.ToLower(m[1]) { + case "true", "false", "null", "nan", "infinity": + default: + return false, resume.MatchString(code), nil + } + } + return !strings.EqualFold(v, "true"), false, nil +} + +// checkEnv requires the workflow-level env to bind name to want, and no job or +// step env to rebind it to anything else. +func checkEnv(root, jobs *yaml.Node, name, want string) error { + binding := func(n *yaml.Node) (*yaml.Node, error) { + _, env := field(n, "env") + if env == nil { + return nil, nil + } + if env.Kind != yaml.MappingNode { + return nil, fmt.Errorf("env must be a mapping at line %d", env.Line) + } + _, v := field(env, name) + if v != nil && v.Kind != yaml.ScalarNode { + return nil, fmt.Errorf("%s must be a scalar at line %d", name, v.Line) + } + return v, nil + } + v, err := binding(root) + if err != nil { + return err + } + if v == nil { + return fmt.Errorf("absent from the workflow env") + } + if v.Value != want { + return fmt.Errorf("%q in the workflow env", v.Value) + } + for i := 1; i < len(jobs.Content); i += 2 { + scopes := []*yaml.Node{jobs.Content[i]} + if _, steps := field(jobs.Content[i], "steps"); steps != nil && steps.Kind == yaml.SequenceNode { + scopes = append(scopes, steps.Content...) + } + for _, n := range scopes { + v, err := binding(n) + if err != nil { + return err + } + if v != nil && v.Value != want { + return fmt.Errorf("rebound to %q at line %d", v.Value, v.Line) + } + } + } + return nil +} + +// needsGraph rejects needs that name no job, the job itself, or a cycle. +func needsGraph(needs map[string][]string, jobs map[string]bool) error { + state := map[string]int{} // 0 unvisited, 1 on the current path, 2 done + var visit func(string) error + visit = func(name string) error { + switch state[name] { + case 1: + return fmt.Errorf("needs cycle through job %q", name) + case 2: + return nil + } + state[name] = 1 + for _, dep := range needs[name] { + if _, ok := jobs[dep]; !ok { + return fmt.Errorf("job %q needs unknown job %q", name, dep) + } + if err := visit(dep); err != nil { + return err + } + } + state[name] = 2 + return nil + } + for name := range needs { + if err := visit(name); err != nil { + return err + } + } + return nil +} + +// cronField is one field of a five-field cron line: the values it matches, +// and whether it was "*" (which matters for the day-of-month/day-of-week +// rule). +type cronField struct { + match [60]bool + star bool +} + +var ( + monthNames = map[string]int{"jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6, "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12} + dayNames = map[string]int{"sun": 0, "mon": 1, "tue": 2, "wed": 3, "thu": 4, "fri": 5, "sat": 6} +) + +func digits(s string) bool { return s != "" && strings.Trim(s, "0123456789") == "" } + +// parseCronField accepts what GitHub documents for schedule crons: *, a +// value, a range a-b, a step */n, a-b/n or a/n, comma lists, and names +// (JAN-DEC, SUN-SAT) only in their own field. Anything else is refused rather +// than guessed. As in cron, a field that starts with * is unrestricted for +// the day-of-month/day-of-week rule. +func parseCronField(s string, lo, hi int, names map[string]int) (cronField, error) { + var f cronField + f.star = strings.HasPrefix(s, "*") + value := func(v string) (int, error) { + if n, ok := names[strings.ToLower(v)]; ok { + return n, nil + } + if !digits(v) { + return 0, fmt.Errorf("cron value %q", v) + } + n, err := strconv.Atoi(v) + if err != nil || n < lo || n > hi { + return 0, fmt.Errorf("cron value %q outside %d-%d", v, lo, hi) + } + return n, nil + } + for _, part := range strings.Split(s, ",") { + base, stepText, hasStep := strings.Cut(part, "/") + step := 1 + if hasStep { + n, err := strconv.Atoi(stepText) + if !digits(stepText) || err != nil || n < 1 || n > hi { + return f, fmt.Errorf("cron step %q", part) + } + step = n + } + first, last := lo, hi + switch a, b, isRange := strings.Cut(base, "-"); { + case base == "*": + case isRange: + var err error + if first, err = value(a); err != nil { + return f, err + } + if last, err = value(b); err != nil { + return f, err + } + if first > last { + return f, fmt.Errorf("cron range %q", part) + } + default: + n, err := value(base) + if err != nil { + return f, err + } + first = n // a/n runs from a to the field's end + if !hasStep { + last = n + } + } + for n := first; n <= last; n += step { + f.match[n] = true + } + } + return f, nil +} + +type cronSpec [5]cronField + +func parseCron(line string) (cronSpec, error) { + var c cronSpec + fields := strings.Fields(line) + if len(fields) != 5 { + return c, fmt.Errorf("cron %q does not have 5 fields", line) + } + bounds := [5][2]int{{0, 59}, {0, 23}, {1, 31}, {1, 12}, {0, 6}} + names := [5]map[string]int{3: monthNames, 4: dayNames} + for i, field := range fields { + f, err := parseCronField(field, bounds[i][0], bounds[i][1], names[i]) + if err != nil { + return c, err + } + c[i] = f + } + return c, nil +} + +// fires reports whether the cron fires at minute t (UTC). As in cron, when +// both day of month and day of week are restricted either one may match. +func (c *cronSpec) fires(t time.Time) bool { + if !c[0].match[t.Minute()] || !c[1].match[t.Hour()] || !c[3].match[int(t.Month())] { + return false + } + dom, dow := c[2].match[t.Day()], c[4].match[int(t.Weekday())] + if c[2].star || c[4].star { + return dom && dow + } + return dom || dow +} + +// checkSchedule requires the workflow's on.schedule to fire inside each timed +// canary window the plan job accepts: 20-30 and 68-78 whole hours after +// publication, that is [T0+20h, T0+31h) and [T0+68h, T0+79h). A cron firing +// at the window's start leaves no room for schedule delay; that margin is the +// operator's choice and is not judged here. +func checkSchedule(root *yaml.Node, published string) error { + t0, err := time.Parse(time.RFC3339, published) + if err != nil { + return fmt.Errorf("publication time %q: %w", published, err) + } + _, on := field(root, "on") + if on == nil { + return fmt.Errorf("no on.schedule list") + } + _, schedule := field(on, "schedule") + if schedule == nil || schedule.Kind != yaml.SequenceNode { + return fmt.Errorf("no on.schedule list") + } + var crons []cronSpec + for _, item := range schedule.Content { + // Only cron is evaluated, in UTC; any other key (timezone, for one) + // would change when GitHub fires it, so it is refused, not ignored. + if item.Kind != yaml.MappingNode { + return fmt.Errorf("schedule entry is not a mapping at line %d", item.Line) + } + for i := 0; i < len(item.Content); i += 2 { + if k := item.Content[i]; k.Value != "cron" { + return fmt.Errorf("schedule entry key %q at line %d is not checked; only cron, in UTC", k.Value, k.Line) + } + } + _, line := field(item, "cron") + if line == nil || line.Kind != yaml.ScalarNode { + return fmt.Errorf("schedule entry without a cron string at line %d", item.Line) + } + c, err := parseCron(line.Value) + if err != nil { + return err + } + crons = append(crons, c) + } + for _, w := range [][2]int{{20, 31}, {68, 79}} { + start, end := t0.UTC().Add(time.Duration(w[0])*time.Hour), t0.UTC().Add(time.Duration(w[1])*time.Hour) + t := start.Truncate(time.Minute) + if t.Before(start) { + t = t.Add(time.Minute) + } + found := false + for ; t.Before(end) && !found; t = t.Add(time.Minute) { + for i := range crons { + found = found || crons[i].fires(t) + } + } + if !found { + return fmt.Errorf("no schedule cron fires in [T0+%dh, T0+%dh)", w[0], w[1]) + } + } + return nil +} + +func check(r io.Reader, mode, commit, tag string) error { + if mode != "all" && mode != "candidate" && mode != "env" && mode != "schedule" { + return fmt.Errorf("unknown mode %q", mode) + } + d := yaml.NewDecoder(r) + var doc yaml.Node + if err := d.Decode(&doc); err != nil { + return err + } + var extra yaml.Node + if err := d.Decode(&extra); !errors.Is(err, io.EOF) { + return fmt.Errorf("expected one YAML document") + } + markers, err := validate(&doc) + if err != nil { + return err + } + if len(doc.Content) != 1 { + return fmt.Errorf("expected workflow mapping") + } + if err := keywordCase(doc.Content[0]); err != nil { + return err + } + if mode == "schedule" { + return checkSchedule(doc.Content[0], commit) + } + _, jobs := field(doc.Content[0], "jobs") + if jobs == nil || jobs.Kind != yaml.MappingNode { + return fmt.Errorf("expected jobs mapping") + } + // A job can never run when its own if is a constant other than true, or + // when a job it needs can never run and its if does not resume after a + // skipped need. Job IDs compare case-insensitively, as GitHub resolves + // needs; a needs graph GitHub would reject is rejected here too. + never, resumes, needs := map[string]bool{}, map[string]bool{}, map[string][]string{} + for i := 0; i < len(jobs.Content); i += 2 { + name, job := strings.ToLower(jobs.Content[i].Value), jobs.Content[i+1] + if job.Kind != yaml.MappingNode { + return fmt.Errorf("expected job mapping") + } + if _, dup := never[name]; dup { + return fmt.Errorf("job IDs differ only by case at line %d", jobs.Content[i].Line) + } + if err := keywordCase(job); err != nil { + return err + } + if _, steps := field(job, "steps"); steps != nil && steps.Kind == yaml.SequenceNode { + for _, step := range steps.Content { + if err := keywordCase(step); err != nil { + return err + } + } + } + off, res, err := condition(job) + if err != nil { + return err + } + never[name], resumes[name] = off, res + switch _, n := field(job, "needs"); { + case n == nil: + case n.Kind == yaml.ScalarNode: + needs[name] = []string{strings.ToLower(n.Value)} + case n.Kind == yaml.SequenceNode: + for _, dep := range n.Content { + if dep.Kind != yaml.ScalarNode { + return fmt.Errorf("needs must list job names at line %d", dep.Line) + } + needs[name] = append(needs[name], strings.ToLower(dep.Value)) + } + default: + return fmt.Errorf("needs must be a job name or a list at line %d", n.Line) + } + } + if err := needsGraph(needs, never); err != nil { + return err + } + for changed := true; changed; { + changed = false + for name, deps := range needs { + for _, dep := range deps { + if !never[name] && !resumes[name] && never[dep] { + never[name], changed = true, true + } + } + } + } + if mode == "env" { + return checkEnv(doc.Content[0], jobs, commit, tag) + } + if mode == "candidate" && markers != 1 { + return fmt.Errorf("expected exactly one candidate marker, got %d", markers) + } + found := 0 + inspect := func(n *yaml.Node, off bool) error { + key, value := field(n, "uses") + if key == nil { + return nil + } + selected := mode == "candidate" && commentsHave(key.HeadComment+"\n"+value.HeadComment, marker) + if value.Kind != yaml.ScalarNode || value.Tag != "!!str" { + return fmt.Errorf("uses must be a string at line %d", value.Line) + } + ref := strings.ToLower(value.Value) + if mode == "all" { + selected = ref == target || strings.HasPrefix(ref, target+"@") || strings.HasPrefix(ref, target+"/") + } + if !selected { + return nil + } + found++ + if off { + return fmt.Errorf("candidate uses at line %d can never run (a constant if other than true, here or on a needed job)", value.Line) + } + // The owner and repository match ASCII case-insensitively, as GitHub + // resolves them; the commit must be spelled exactly. + repo, sha, _ := strings.Cut(value.Value, "@") + if !asciiEqualFold(repo, target) || sha != commit { + return fmt.Errorf("candidate uses is not @%s at line %d", commit, value.Line) + } + note := strings.Fields(strings.TrimPrefix(value.LineComment, "#")) + if len(note) == 0 || note[0] != tag { + return fmt.Errorf("candidate uses lacks # %s at line %d", tag, value.Line) + } + return nil + } + for i := 0; i < len(jobs.Content); i += 2 { + name, job := strings.ToLower(jobs.Content[i].Value), jobs.Content[i+1] + if err := inspect(job, never[name]); err != nil { + return err + } + _, steps := field(job, "steps") + if steps == nil { + continue + } + if steps.Kind != yaml.SequenceNode { + return fmt.Errorf("expected steps sequence") + } + for _, step := range steps.Content { + if step.Kind != yaml.MappingNode { + return fmt.Errorf("expected step mapping") + } + off, _, err := condition(step) + if err != nil { + return err + } + if err := inspect(step, off || never[name]); err != nil { + return err + } + } + } + if found == 0 || (mode == "candidate" && found != 1) { + return fmt.Errorf("expected active candidate uses, got %d", found) + } + return nil +} diff --git a/internal/tools/rcpreflight/main_test.go b/internal/tools/rcpreflight/main_test.go new file mode 100644 index 0000000..c9e97fd --- /dev/null +++ b/internal/tools/rcpreflight/main_test.go @@ -0,0 +1,271 @@ +package main + +import ( + "bytes" + "os" + "strings" + "testing" +) + +func TestActiveWorkflowPins(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag + workflow := func(steps string) string { return "jobs:\n test:\n steps:\n" + steps } + tests := []struct { + name, mode, text string + ok bool + }{ + {"plain", "all", workflow(" - uses: " + good + "\n"), true}, + {"quoted", "all", workflow(" - uses: 'kernel-guard/bpfcompat@" + sha + "' # " + tag + "\n"), true}, + {"marker", "candidate", workflow(" - name: candidate\n # bpfcompat-pin: candidate\n uses: " + good + "\n"), true}, + {"comment-only", "all", workflow(" # - uses: " + good + "\n - run: true\n"), false}, + {"hidden-ref", "all", workflow(" - uses: " + good + "\n - uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"folded-ref", "all", workflow(" - uses: " + good + "\n - uses: >-\n Kernel-Guard/bpfcompat@main\n"), false}, + {"unicode-ref", "all", workflow(" - uses: " + good + "\n - uses: \"Kernel-\\u0047uard/bpfcompat@main\"\n"), false}, + {"run-text", "all", workflow(" - run: |\n uses: " + good + "\n"), false}, + {"run-marker", "candidate", workflow(" - run: |\n # bpfcompat-pin: candidate\n uses: " + good + "\n - uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"flow-stale", "all", workflow(" - uses: " + good + "\n - {uses: 'Kernel-Guard/bpfcompat@main'}\n"), false}, + {"flow-correct", "all", "jobs: {test: {steps: [{uses: 'Kernel-Guard/bpfcompat@" + sha + "'}]}}\n", false}, // missing tag comment + {"quoted-key", "all", workflow(" - uses: " + good + "\n - 'uses': Kernel-Guard/bpfcompat@main\n"), false}, + {"wrong-tag", "all", workflow(" - uses: Kernel-Guard/bpfcompat@" + sha + " # v0.4.0-rc.4\n"), false}, + {"duplicate-key", "all", workflow(" - uses: " + good + "\n uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"duplicate-marker", "candidate", workflow(" - name: first\n # bpfcompat-pin: candidate\n uses: " + good + "\n - name: second\n # bpfcompat-pin: candidate\n uses: " + good + "\n"), false}, + {"alias", "all", workflow(" - uses: &pin Kernel-Guard/bpfcompat@" + sha + " # " + tag + "\n - uses: *pin\n"), false}, + {"multi-document", "all", workflow(" - uses: "+good+"\n") + "---\njobs: {}\n", false}, + {"no-jobs", "all", "run: 'uses: " + good + "'\n", false}, + {"wrong-type", "all", workflow(" - uses: [invalid]\n"), false}, + {"subpath", "all", workflow(" - uses: " + good + "\n - uses: Kernel-Guard/bpfcompat/sub@" + sha + " # " + tag + "\n"), false}, + {"disabled-step", "all", workflow(" - if: false\n uses: " + good + "\n"), false}, + {"disabled-expression", "all", workflow(" - if: ${{ false }}\n uses: " + good + "\n"), false}, + {"disabled-job", "candidate", "jobs:\n decoy:\n if: ${{ false }}\n steps:\n - name: decoy\n # bpfcompat-pin: candidate\n uses: " + good + "\n real:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\n", false}, + {"disabled-marked-step", "candidate", workflow(" - name: decoy\n if: 0\n # bpfcompat-pin: candidate\n uses: " + good + "\n - uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"conditional-step", "all", workflow(" - if: needs.plan.outputs.should_run == 'true'\n uses: " + good + "\n"), true}, + {"conditional-job", "all", "jobs:\n test:\n if: ${{ needs.plan.outputs.should_run == 'true' }}\n uses: " + good + "\n", true}, + {"merge-key", "all", workflow(" - uses: " + good + "\n - <<: {uses: Kernel-Guard/bpfcompat@main}\n"), false}, + {"case-variant-uses", "all", workflow(" - uses: " + good + "\n - Uses: Kernel-Guard/bpfcompat@main\n"), false}, + {"case-variant-steps", "all", "jobs:\n test:\n Steps:\n - uses: Kernel-Guard/bpfcompat@main\n ok:\n steps:\n - uses: " + good + "\n", false}, + {"kelvin-sign-owner", "all", workflow(" - uses: \"\u212Aernel-Guard/bpfcompat@" + sha + "\" # " + tag + "\n"), false}, + {"uppercase-sha", "all", workflow(" - uses: Kernel-Guard/bpfcompat@" + strings.ToUpper(sha) + " # " + tag + "\n"), false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), tt.mode, sha, tag) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + +// A pin on a step or job that can never run must not count. In an expression +// GitHub treats false, 0, -0, the empty string, null and NaN as falsy and +// accepts any JSON number form. An if that is empty or YAML null is no +// condition at all, and keywords are case-insensitive. +func TestNeverRunningPins(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag + for _, cond := range []string{"false", "${{ false }}", "-0", "${{ -0 }}", "-0.0", "0.0", "${{ 0.0 }}", "0e0", ".0", "00", + "0x0", "${{ 0x0 }}", "0x00", "NaN", "${{ NaN }}", "\"''\"", "${{ '' }}", "${{ null }}", "False", "false && true", "${{ !true }}", "1 == 2"} { + t.Run("step if "+cond, func(t *testing.T) { + text := "jobs:\n test:\n steps:\n - if: " + cond + "\n uses: " + good + "\n" + if err := check(strings.NewReader(text), "all", sha, tag); err == nil { + t.Fatalf("pin under if: %s accepted", cond) + } + }) + t.Run("job if "+cond, func(t *testing.T) { + text := "jobs:\n test:\n if: " + cond + "\n uses: " + good + "\n" + if err := check(strings.NewReader(text), "all", sha, tag); err == nil { + t.Fatalf("job pin under if: %s accepted", cond) + } + }) + } + for _, cond := range []string{"true", "${{ true }}", "True", "TRUE", "''", "null", "~", "", "needs.plan.outputs.should_run == 'true'", "${{ github.event_name == 'schedule' }}", "always()"} { + t.Run("runs "+cond, func(t *testing.T) { + text := "jobs:\n test:\n steps:\n - if: " + cond + "\n uses: " + good + "\n" + if err := check(strings.NewReader(text), "all", sha, tag); err != nil { + t.Fatalf("pin under if: %s rejected: %v", cond, err) + } + }) + } + needs := []struct { + name, mode, text string + ok bool + }{ + {"needs-disabled", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n steps:\n - uses: " + good + "\n", false}, + {"needs-list-disabled-candidate", "candidate", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: [a]\n steps:\n - name: x\n # bpfcompat-pin: candidate\n uses: " + good + "\n c:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\n", false}, + {"needs-chain", "all", "jobs:\n c:\n needs: b\n uses: " + good + "\n b:\n needs: [a]\n steps:\n - run: true\n a:\n if: ${{ -0 }}\n steps:\n - run: true\n", false}, + {"needs-always", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: ${{ always() }}\n steps:\n - uses: " + good + "\n", true}, + {"needs-running", "all", "jobs:\n plan:\n steps:\n - run: true\n b:\n needs: plan\n if: needs.plan.outputs.should_run == 'true'\n steps:\n - uses: " + good + "\n", true}, + {"needs-mapping", "all", "jobs:\n a:\n steps:\n - run: true\n b:\n needs: {a: x}\n steps:\n - uses: " + good + "\n", false}, + {"needs-quoted-always", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: github.ref != 'always()'\n steps:\n - uses: " + good + "\n", false}, + {"needs-failure", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: success() || failure()\n steps:\n - uses: " + good + "\n", false}, + {"needs-not-cancelled", "all", "jobs:\n a:\n if: false\n steps:\n - run: true\n b:\n needs: a\n if: ${{ !cancelled() }}\n steps:\n - uses: " + good + "\n", true}, + {"needs-case", "all", "jobs:\n plan:\n if: false\n steps:\n - run: true\n b:\n needs: PLAN\n steps:\n - uses: " + good + "\n", false}, + {"job-ids-differ-by-case", "all", "jobs:\n plan:\n steps:\n - run: true\n Plan:\n steps:\n - uses: " + good + "\n", false}, + {"needs-unknown", "all", "jobs:\n b:\n needs: nope\n steps:\n - uses: " + good + "\n", false}, + {"needs-self", "all", "jobs:\n b:\n needs: b\n steps:\n - uses: " + good + "\n", false}, + {"needs-cycle", "all", "jobs:\n a:\n needs: b\n steps:\n - run: true\n b:\n needs: a\n steps:\n - uses: " + good + "\n", false}, + } + for _, tt := range needs { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), tt.mode, sha, tag) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + +// Case variants are rejected only where GitHub reads its own keywords; user +// data such as env names, inputs and matrix keys may use any spelling. +func TestKeywordPositions(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + good := "Kernel-Guard/bpfcompat@" + sha + " # " + tag + tests := []struct { + name, text string + ok bool + }{ + {"env-name", "env:\n ENV: prod\njobs:\n test:\n steps:\n - uses: " + good + "\n", true}, + {"input-name", "jobs:\n test:\n steps:\n - uses: " + good + "\n with:\n Steps: x\n", true}, + {"matrix-key", "jobs:\n test:\n strategy:\n matrix:\n Jobs: [1]\n steps:\n - uses: " + good + "\n", true}, + {"job-needs-variant", "jobs:\n a:\n if: false\n steps:\n - run: true\n test:\n Needs: a\n steps:\n - uses: " + good + "\n", false}, + {"step-if-variant", "jobs:\n test:\n steps:\n - If: false\n uses: " + good + "\n", false}, + {"root-jobs-variant", "Jobs:\n x:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\njobs:\n test:\n steps:\n - uses: " + good + "\n", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), "all", sha, tag) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + +// The env check names what it found, so the report can show the stale value. +func TestEnvBindingDetail(t *testing.T) { + text := "env:\n CANDIDATE_TAG: v0.4.0-rc.4\njobs:\n x:\n steps:\n - run: true\n" + err := check(strings.NewReader(text), "env", "CANDIDATE_TAG", "v0.4.0-rc.5") + if err == nil || !strings.Contains(err.Error(), "v0.4.0-rc.4") { + t.Fatalf("check() = %v, want an error naming the found value", err) + } +} + +func TestWorkflowEnvBindings(t *testing.T) { + head := "env:\n CANDIDATE_TAG: v0.4.0-rc.5\n CANDIDATE_PUBLISHED_AT: \"2026-11-02T10:00:00Z\"\n" + tests := []struct { + name, text, key, want string + ok bool + }{ + {"top-level", head + "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", true}, + {"quoted", head + "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_PUBLISHED_AT", "2026-11-02T10:00:00Z", true}, + {"stale", head + "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.6", false}, + {"run-text-before-stale-env", "jobs:\n x:\n steps:\n - run: |\n CANDIDATE_TAG: v0.4.0-rc.6\nenv:\n CANDIDATE_TAG: v0.4.0-rc.5\n", "CANDIDATE_TAG", "v0.4.0-rc.6", false}, + {"absent", "jobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"job-shadow", head + "jobs:\n x:\n env:\n CANDIDATE_TAG: v0.4.0-rc.4\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"step-shadow", head + "jobs:\n x:\n steps:\n - run: true\n env:\n CANDIDATE_TAG: v0.4.0-rc.4\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"step-env-variant", head + "jobs:\n x:\n steps:\n - run: true\n Env:\n CANDIDATE_TAG: v0.4.0-rc.4\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + {"duplicate", head + " CANDIDATE_TAG: v0.4.0-rc.4\njobs:\n x:\n steps:\n - run: true\n", "CANDIDATE_TAG", "v0.4.0-rc.5", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), "env", tt.key, tt.want) + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } +} + +// The timed canaries count only from a schedule run whose plan job starts +// 20-30 or 68-78 whole hours after publication, so the designated crons must +// fire inside [T0+20h, T0+31h) and [T0+68h, T0+79h). T0 below is synthetic. +func TestScheduleWindows(t *testing.T) { + t0 := "2026-11-02T10:00:00Z" // a Monday + wf := func(crons ...string) string { + s := "on:\n workflow_dispatch:\n schedule:\n" + for _, c := range crons { + s += " - cron: \"" + c + "\"\n" + } + return s + "jobs:\n plan:\n steps:\n - run: true\n" + } + tests := []struct { + name, text string + ok bool + }{ + {"precedent-offsets", wf("14 10 3 11 *", "14 10 5 11 *"), true}, + {"window-starts", wf("0 6 3 11 *", "0 6 5 11 *"), true}, + {"last-minutes", wf("59 16 3 11 *", "59 16 5 11 *"), true}, + {"first-window-end-excluded", wf("0 17 3 11 *", "14 10 5 11 *"), false}, + {"second-window-end-excluded", wf("14 10 3 11 *", "0 17 5 11 *"), false}, + {"before-first-window", wf("59 5 3 11 *", "14 10 5 11 *"), false}, + {"previous-candidate-crons", wf("44 17 15 10 *", "44 17 17 10 *"), false}, + {"only-first", wf("14 10 3 11 *"), false}, + {"every-five-minutes", wf("*/5 * * * *"), true}, + {"names-and-ranges", wf("14 10 * NOV TUE", "14 10 * nov thu"), true}, + {"dom-or-dow", wf("14 10 1 11 TUE", "14 10 1 11 THU"), true}, + {"dow-seven-refused", wf("14 10 3 11 *", "14 10 5 11 7"), false}, + {"start-step-form", wf("14 10 3/2 11 *"), true}, + {"star-step-day-is-unrestricted", wf("14 10 */2 11 SUN"), false}, + {"month-name-in-dow", wf("14 10 * 11 NOV"), false}, + {"dow-name-in-month", wf("14 10 3 MON *", "14 10 5 11 *"), false}, + {"plus-sign", wf("14 10 +3 11 *", "14 10 5 11 *"), false}, + {"huge-step", wf("14 10 1-5/9223372036854775807 11 *", "14 10 5 11 *"), false}, + {"timezone-key", "on:\n schedule:\n - cron: \"14 10 3 11 *\"\n timezone: \"Asia/Tokyo\"\n - cron: \"14 10 5 11 *\"\njobs:\n plan:\n steps:\n - run: true\n", false}, + {"list-and-step", wf("0,30 8-12/2 3,5 11 *"), true}, + {"invalid-field", wf("14 10 3 11 *", "61 10 5 11 *"), false}, + {"too-few-fields", wf("14 10 3 11"), false}, + {"no-schedule", "on:\n workflow_dispatch:\njobs:\n plan:\n steps:\n - run: true\n", false}, + {"cron-not-string", "on:\n schedule:\n - cron: [1]\njobs:\n plan:\n steps:\n - run: true\n", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := check(strings.NewReader(tt.text), "schedule", t0, "-") + if (err == nil) != tt.ok { + t.Fatalf("check() = %v, want accepted=%v", err, tt.ok) + } + }) + } + if err := check(strings.NewReader(wf("14 10 3 11 *", "14 10 5 11 *")), "schedule", "not-a-time", "-"); err == nil { + t.Fatal("an unparsable publication time was accepted") + } +} + +func TestRunExitCodes(t *testing.T) { + sha := strings.Repeat("a", 40) + tag := "v0.4.0-rc.5" + dir := t.TempDir() + good := dir + "/good.yml" + stale := dir + "/stale.yml" + if err := os.WriteFile(good, []byte("jobs:\n t:\n steps:\n - uses: Kernel-Guard/bpfcompat@"+sha+" # "+tag+"\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(stale, []byte("jobs:\n t:\n steps:\n - uses: Kernel-Guard/bpfcompat@main\n"), 0o600); err != nil { + t.Fatal(err) + } + tests := []struct { + name string + args []string + code int + out string + }{ + {"verified", []string{"all", sha, tag, good}, 0, "VERIFIED\n"}, + {"pending", []string{"all", sha, tag, stale}, 0, "PENDING\n"}, + {"usage", []string{"all", sha, tag}, 2, ""}, + {"unknown-mode", []string{"some", sha, tag, good}, 2, ""}, + {"env-mode", []string{"env", "CANDIDATE_TAG", tag, good}, 0, "PENDING\n"}, + {"schedule-mode", []string{"schedule", "2026-11-02T10:00:00Z", "-", good}, 0, "PENDING\n"}, + {"unreadable", []string{"all", sha, tag, dir + "/missing.yml"}, 2, ""}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var stdout, stderr bytes.Buffer + if code := run(tt.args, &stdout, &stderr); code != tt.code || stdout.String() != tt.out { + t.Fatalf("run() = %d, %q; want %d, %q (stderr %q)", code, stdout.String(), tt.code, tt.out, stderr.String()) + } + }) + } +} diff --git a/release.yaml b/release.yaml index f61109e..08f0a52 100644 --- a/release.yaml +++ b/release.yaml @@ -1,5 +1,5 @@ stable_version: 0.3.7 -release_version: 0.4.0-rc.4 +release_version: 0.4.0-rc.5 release_channel: prerelease release_operator: ErenAri approval_mode: solo-maintainer diff --git a/scripts/rc-preflight.sh b/scripts/rc-preflight.sh new file mode 100755 index 0000000..128cd7b --- /dev/null +++ b/scripts/rc-preflight.sh @@ -0,0 +1,343 @@ +#!/usr/bin/env bash +# Read-only inventory of a release candidate's prerequisites and evidence. +# +# rc-preflight.sh --tag vX.Y.Z-rc.N [--commit <40-hex>] [--repo owner/repo] +# +# Prints one line per item: VERIFIED, MISSING or PENDING, the item, and the +# evidence behind the status. It only reads: GitHub REST GETs through `gh api`, +# `gh release download` of two text files (SHA256SUMS and +# release-candidate-evidence.json), and files in this checkout. It never dispatches or reruns a workflow, never writes to GitHub, +# never executes a downloaded file and never writes operator evidence. +# Active workflow pins, the canary's workflow env and its schedule are parsed +# by a helper +# compiled from this checkout's Go sources and vendored YAML library into the +# temporary directory (writing only Go's build cache). The build cannot +# download modules or a toolchain: GOPROXY=off, GONOPROXY and GONOSUMDB set to +# a pattern nothing matches (so a caller GOPRIVATE cannot reopen direct +# fetches), GOWORK=off, -mod=vendor. Verifying a cached toolchain may still +# query the checksum database. GOTOOLCHAIN is left as the caller set it, so the +# Go on PATH, or a toolchain at go.mod's version already in the module cache, +# can build it. +# +# Status meanings: +# VERIFIED observed directly (a ref, a run, a file, a hash) and as required. +# MISSING observed and wrong, or its window has closed without it. +# PENDING not observable yet (no tag, no run, operator-held evidence, an +# open decision). Elapsed time alone never makes an item VERIFIED. +# Canary and campaign runs count only when they started after publication and +# after main was designated to this candidate; a canary run counts only if a +# job after its planning job succeeded. A rollback drill run stays PENDING: +# which candidate it drilled is not visible here. Run-level items say only +# that a run of the right kind exists; binding its evidence to the candidate is the job of +# scripts/production-readiness-report.sh, which this does not replace. Canary +# windows (20-30h, 68-78h after publication) are judged from each run's start +# time as a proxy for its plan job, which enforces the exact window itself. +# +# Exit: 0 no MISSING, 1 at least one MISSING, 2 usage, 3 a lookup, download, +# parse or decode failed (the report is then incomplete and nothing is inferred +# from the failure). +# Test hook: BPFCOMPAT_PREFLIGHT_NOW= fixes "now". +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +repo="Kernel-Guard/bpfcompat" tag="" want_commit="" +while [[ $# -gt 0 ]]; do + case "$1" in + --tag) tag="${2:-}"; shift 2 ;; + --commit) want_commit="${2:-}"; shift 2 ;; + --repo) repo="${2:-}"; shift 2 ;; + *) echo "usage: $0 --tag vX.Y.Z-rc.N [--commit ] [--repo owner/repo]" >&2; exit 2 ;; + esac +done +[[ "$tag" =~ ^v([0-9]+\.[0-9]+\.[0-9]+)-rc\.[1-9][0-9]*$ ]] || { echo "rc-preflight: --tag must be vX.Y.Z-rc.N" >&2; exit 2; } +[[ -z "$want_commit" || "$want_commit" =~ ^[0-9a-f]{40}$ ]] || { echo "rc-preflight: --commit must be a full SHA" >&2; exit 2; } +[[ "$repo" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]] || { echo "rc-preflight: invalid --repo" >&2; exit 2; } +for t in gh jq git; do command -v "$t" >/dev/null || { echo "rc-preflight: $t is required" >&2; exit 2; }; done +version="${tag#v}" +now="${BPFCOMPAT_PREFLIGHT_NOW:-$(date -u +%s)}" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +missing=0 +say() { # + printf '%-8s %-44s %s\n' "$1" "$2" "$3" + [[ "$1" != MISSING ]] || missing=1 +} +# fetch : GET JSON into . Returns 0, or 4 for HTTP 404; any +# other failure, or a body that is not JSON, aborts the whole report (exit 3). +# Status decisions call it in the main shell, so the abort cannot be mistaken +# for "not found"; inside last_change's $(...) the abort still ends the report +# through set -e. +abort() { echo "rc-preflight: $*; report incomplete" >&2; exit 3; } +fetch() { + if gh api "$1" >"$2" 2>"$tmp/err"; then + jq -e . "$2" >/dev/null 2>&1 || abort "GET $1 returned a body that is not JSON" + return 0 + fi + if grep -q 'HTTP 404' "$tmp/err"; then return 4; fi + abort "lookup failed: GET $1: $(head -1 "$tmp/err")" +} +# need : like fetch, but a 404 is a lookup failure too. +need() { fetch "$1" "$2" || abort "lookup failed: GET $1: not found"; } +# decode : base64 file contents from the contents API. +decode() { jq -r .content "$1" | base64 -d >"$2" 2>/dev/null && [[ -s "$2" ]] || abort "cannot decode file contents ($1)"; } +# asset : download one release asset (text only, never executed). +asset() { gh release download "$tag" --repo "$repo" --pattern "$1" --dir "$2" >/dev/null 2>"$tmp/err" || abort "cannot download $1 from the $tag release: $(head -1 "$tmp/err")"; } +epoch() { date -u -d "$1" +%s; } + +echo "rc-preflight: ${repo} ${tag} at $(date -u -d "@$now" +%Y-%m-%dT%H:%M:%SZ) (read-only)" + +# ------------------------------------------------------------ source metadata +rv="$(awk -F': ' '$1 == "release_version" { print $2 }' "$ROOT/release.yaml")" +rc="$(awk -F': ' '$1 == "release_channel" { print $2 }' "$ROOT/release.yaml")" +if [[ "$rv" == "$version" && "$rc" == prerelease ]]; then say VERIFIED "release.yaml names the candidate" "release_version=$rv release_channel=$rc" +else say MISSING "release.yaml names the candidate" "release_version=$rv release_channel=$rc (checkout $(git -C "$ROOT" rev-parse --short HEAD))"; fi +if grep -Fq "## [${version}]" "$ROOT/CHANGELOG.md"; then say VERIFIED "CHANGELOG has the candidate heading" "## [${version}]" +else say MISSING "CHANGELOG has the candidate heading" "no '## [${version}]' heading"; fi +expected_validator="$(awk -F= '$1 == "reproduces" { split($2, a, " "); print a[2] }' "$ROOT/validator/build-inputs.lock")" + +policy="$ROOT/distribution/policy.json" +if ! bash "$ROOT/scripts/check-distribution-policy.sh" "$policy" >/dev/null 2>&1; then + say MISSING "distribution policy is well-formed" "scripts/check-distribution-policy.sh refused $policy" +else + open="$(jq -r '[.blocks_stable[] as $k | select(.decisions[$k].status != "decided") | $k] | join(",")' "$policy")" + pre="$(jq -r '.decisions.prerelease_with_open_decisions | "\(.status) \(.value)"' "$policy")" + case "$pre" in + "decided true") say VERIFIED "recorded: prerelease with open decisions" "operator recorded true (a policy value, not a legal finding)" ;; + "decided false") + if [[ -n "$open" ]]; then say MISSING "recorded: prerelease with open decisions" "recorded false while open: $open" + else say VERIFIED "recorded: prerelease with open decisions" "recorded false; no stable-blocking decision is open"; fi ;; + *) say PENDING "recorded: prerelease with open decisions" "undecided (Q17); open stable-blocking decisions: ${open:-none}" ;; + esac + if [[ -z "$open" ]]; then say VERIFIED "recorded: stable-blocking decisions" "all recorded in distribution/policy.json" + else say PENDING "recorded: stable-blocking decisions" "undecided: $open (needed before a stable tag, not before this RC)"; fi +fi + +# ------------------------------------------------------------ tag +if ! fetch "repos/${repo}/git/ref/tags/${tag}" "$tmp/ref.json"; then + say PENDING "tag exists" "$tag not created" + for item in "tag build (release-artifacts)" "release staged with required assets" "SHA256SUMS lists required assets" \ + "validator identity" "optional assets match tagged policy" "RC published as prerelease"; do + say PENDING "$item" "requires $tag" + done + for item in "designated: campaign-candidate.json" "designated: consumer-canary candidate pin" "designated: release-candidate-canary" \ + "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)" "rollback drill run"; do + say PENDING "$item" "requires a published $tag" + done + say PENDING "Falco expanded entry" "operator-held; bound by production-readiness-report.sh" + say PENDING "fail-closed promotion incident note" "operator-held; bound by production-readiness-report.sh" + say PENDING "operator promotion confirmation" "operator-held; never written by tooling" + exit "$missing" +fi +obj_type="$(jq -r .object.type "$tmp/ref.json")" obj_sha="$(jq -r .object.sha "$tmp/ref.json")" +commit="$obj_sha" +if [[ "$obj_type" == tag ]]; then need "repos/${repo}/git/tags/${obj_sha}" "$tmp/tagobj.json"; commit="$(jq -r .object.sha "$tmp/tagobj.json")"; fi +if [[ -z "$want_commit" ]]; then say VERIFIED "tag exists" "$tag -> $commit (no --commit to compare)" +elif [[ "$commit" == "$want_commit" ]]; then say VERIFIED "tag exists" "$tag -> $commit (as approved)" +else say MISSING "tag exists" "$tag -> $commit, approved commit is $want_commit"; fi + +# ------------------------------------------------------------ tag build +need "repos/${repo}/actions/workflows/release-artifacts.yml/runs?event=push&head_sha=${commit}&per_page=50" "$tmp/runs.json" +run="$(jq -c --arg t "$tag" '[.workflow_runs[] | select(.head_branch == $t)] | sort_by(.run_attempt) | last // empty' "$tmp/runs.json")" +if [[ -z "$run" ]]; then say PENDING "tag build (release-artifacts)" "no push run for $tag at $commit yet" +else + rid="$(jq -r .id <<<"$run")" ra="$(jq -r .run_attempt <<<"$run")" st="$(jq -r .status <<<"$run")" co="$(jq -r .conclusion <<<"$run")" + if [[ "$st" != completed ]]; then say PENDING "tag build (release-artifacts)" "run $rid attempt $ra is $st" + elif [[ "$co" == success ]]; then say VERIFIED "tag build (release-artifacts)" "run $rid attempt $ra success" + else say MISSING "tag build (release-artifacts)" "run $rid attempt $ra concluded $co"; fi +fi + +# ------------------------------------------------------------ release +need "repos/${repo}/releases?per_page=100" "$tmp/releases.json" +rel="$(jq -c --arg t "$tag" '[.[] | select(.tag_name == $t)] | first // empty' "$tmp/releases.json")" +required=(bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-validator-static-linux-amd64 + bpfcompat-third-party-licenses.tar.gz bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json) +signed=(SHA256SUMS SHA256SUMS.sig SHA256SUMS.crt bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt release-candidate-evidence.json) +published_at="" digest="" +if [[ -z "$rel" ]]; then + for item in "release staged with required assets" "SHA256SUMS lists required assets" "validator identity" \ + "optional assets match tagged policy" "RC published as prerelease"; do say PENDING "$item" "no release for $tag yet"; done +else + names="$(jq -r '.assets[].name' <<<"$rel")" + absent=() + for a in "${required[@]}" "${signed[@]}"; do grep -qxF "$a" <<<"$names" || absent+=("$a"); done + if [[ ${#absent[@]} -eq 0 ]]; then say VERIFIED "release staged with required assets" "$(jq '.assets | length' <<<"$rel") assets" + else say MISSING "release staged with required assets" "absent: ${absent[*]}"; fi + if grep -qxF SHA256SUMS <<<"$names"; then + asset SHA256SUMS "$tmp/sums" + sums="$tmp/sums/SHA256SUMS" + unlisted=() + for a in "${required[@]}"; do awk -v f="$a" '$2 == f && $1 ~ /^[0-9a-f]{64}$/ { n++ } END { exit n != 1 }' "$sums" || unlisted+=("$a"); done + if [[ ${#unlisted[@]} -eq 0 ]]; then say VERIFIED "SHA256SUMS lists required assets" "${#required[@]} required entries, $(grep -c . "$sums") lines" + else say MISSING "SHA256SUMS lists required assets" "missing or duplicated: ${unlisted[*]}"; fi + vs="$(awk '$2 == "bpfcompat-validator-static-linux-amd64" { print $1 }' "$sums")" + if [[ -n "$vs" && "$vs" == "$expected_validator" ]]; then say VERIFIED "validator identity" "$vs (validator/build-inputs.lock reproduces=)" + else say MISSING "validator identity" "SHA256SUMS ${vs:-none}, lock expects $expected_validator"; fi + if fetch "repos/${repo}/contents/distribution/policy.json?ref=${tag}" "$tmp/policy-contents.json"; then + decode "$tmp/policy-contents.json" "$tmp/policy.json" + bad="" + for pair in "ship_link_inputs_asset:bpfcompat-validator-link-inputs-linux-amd64.tar.gz" "ship_native_sources_asset:bpfcompat-validator-native-sources.tar"; do + d="${pair%%:*}" f="${pair#*:}" + ship="$(bash "$ROOT/scripts/check-distribution-policy.sh" --ship "$d" "$tmp/policy.json" 2>/dev/null || echo error)" + listed=false; awk -v f="$f" '$2 == f { n++ } END { exit !n }' "$sums" && listed=true + [[ "$ship" == "$listed" ]] || bad+="$d=$ship but $f listed=$listed; " + done + if [[ -z "$bad" ]]; then say VERIFIED "optional assets match tagged policy" "policy at $tag agrees with SHA256SUMS" + else say MISSING "optional assets match tagged policy" "$bad"; fi + else + say MISSING "optional assets match tagged policy" "distribution/policy.json absent at $tag" + fi + else + for item in "SHA256SUMS lists required assets" "validator identity" "optional assets match tagged policy"; do + say MISSING "$item" "the $tag release has no SHA256SUMS asset" + done + fi + draft="$(jq -r .draft <<<"$rel")" prerel="$(jq -r .prerelease <<<"$rel")" + if [[ "$draft" == true ]]; then say PENDING "RC published as prerelease" "draft (promotion pending)" + elif [[ "$prerel" == true ]]; then + published_at="$(jq -r .published_at <<<"$rel")" + say VERIFIED "RC published as prerelease" "published $published_at" + digest="" + if grep -qxF release-candidate-evidence.json <<<"$names"; then + asset release-candidate-evidence.json "$tmp/ev" + jq -e . "$tmp/ev/release-candidate-evidence.json" >/dev/null 2>&1 || abort "release-candidate-evidence.json of $tag is not JSON" + digest="$(jq -r '.image // empty' "$tmp/ev/release-candidate-evidence.json" | sed -n 's/.*@\(sha256:[0-9a-f]\{64\}\)$/\1/p')" + fi + else say MISSING "RC published as prerelease" "published but not marked prerelease"; fi +fi + +# ------------------------------------------------------------ designation and qualification +# Runs only count when they started after publication AND after main was +# designated to this candidate; before that, the workflows still named the +# previous candidate. +rcnum="${tag##*-rc.}" +if [[ -z "$published_at" ]]; then + for item in "designated: campaign-candidate.json" "designated: consumer-canary candidate pin" "designated: release-candidate-canary" \ + "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)" "rollback drill run"; do + say PENDING "$item" "requires a published $tag" + done +else + main_file() { need "repos/${repo}/contents/$1?ref=main" "$tmp/contents.json"; decode "$tmp/contents.json" "$2"; } + # last_change : epoch at which the newest change to landed on + # main: the merge time of the pull request that brought that commit in, or + # the commit's own date when it was pushed directly. + last_change() { + need "repos/${repo}/commits?sha=main&path=$1&per_page=1" "$tmp/commits.json" + local sha d + sha="$(jq -r '.[0].sha // empty' "$tmp/commits.json")" + [[ -n "$sha" ]] || abort "no main commit touches $1" + need "repos/${repo}/commits/${sha}/pulls" "$tmp/pulls.json" + d="$(jq -r '[.[] | select(.merged_at != null and .base.ref == "main") | .merged_at] | max // empty' "$tmp/pulls.json")" + [[ -n "$d" ]] || d="$(jq -r '.[0].commit.committer.date' "$tmp/commits.json")" + epoch "$d" + } + ok_designations=0 + main_file .github/campaign-candidate.json "$tmp/cc.json" + jq -e 'type == "object"' "$tmp/cc.json" >/dev/null 2>&1 || abort "campaign-candidate.json on main is not a JSON object" + # The scheduled campaigns also bind the two digests recorded there, so they + # must be this release's SHA256SUMS and candidate evidence. + cc_problems="" + [[ "$(jq -r .tag "$tmp/cc.json")" == "$tag" && "$(jq -r .commit_sha "$tmp/cc.json")" == "$commit" ]] || + cc_problems+="binds $(jq -r .tag "$tmp/cc.json") $(jq -r .commit_sha "$tmp/cc.json"); " + for pair in "checksums_sha256=$tmp/sums/SHA256SUMS" "candidate_evidence_sha256=$tmp/ev/release-candidate-evidence.json"; do + key="${pair%%=*}" file="${pair#*=}" + if [[ ! -f "$file" ]]; then cc_problems+="$key: the release has no ${file##*/} to compare; " + elif [[ "$(jq -r --arg k "$key" '.[$k] // empty' "$tmp/cc.json")" != "$(sha256sum "$file" | cut -d' ' -f1)" ]]; then + cc_problems+="$key is not this release's ${file##*/}; " + fi + done + if [[ -z "$cc_problems" ]]; then + say VERIFIED "designated: campaign-candidate.json" "main binds $tag $commit and this release's SHA256SUMS and candidate evidence digests"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: campaign-candidate.json" "main differs: ${cc_problems}retarget after verification"; fi + main_file .github/workflows/consumer-canary.yml "$tmp/consumer.yml" + command -v go >/dev/null || abort "Go is required to parse active workflow pins" + (cd "$ROOT" && GOWORK=off GOPROXY=off GONOPROXY=none.invalid GONOSUMDB=none.invalid go build -mod=vendor -o "$tmp/pincheck" ./internal/tools/rcpreflight) || + abort "cannot build the vendored workflow-pin parser without downloads; it needs Go at go.mod's version on PATH, or that toolchain already in the module cache with checksum verification on (GOSUMDB not off)" + pin_result="$("$tmp/pincheck" candidate "$commit" "$tag" "$tmp/consumer.yml")" || abort "workflow-pin parser failed" + if [[ "$pin_result" == VERIFIED ]]; then + say VERIFIED "designated: consumer-canary candidate pin" "one active candidate marker pinned to $commit # $tag"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: consumer-canary candidate pin" "requires one '# bpfcompat-pin: candidate' marker followed by an active @$commit # $tag uses scalar"; fi + main_file .github/workflows/release-candidate-canary.yml "$tmp/rcc.yml" + # envis : the workflow-level env binds it (parsed, not grepped, + # so run text cannot stand in for it) and no job or step rebinds it. When it + # does not, why= holds what the parser found instead. + envis() { + local r + r="$("$tmp/pincheck" env "$1" "$2" "$tmp/rcc.yml" 2>"$tmp/envis.err")" || + abort "workflow-pin parser failed: $(cat "$tmp/envis.err")" + [[ "$r" == VERIFIED ]] && return 0 + why="$(sed -n 's/^rc-preflight pins: //p' "$tmp/envis.err" | tail -1)" + return 1 + } + problems="" + envis CANDIDATE_TAG "$tag" || problems+="CANDIDATE_TAG=($why) " + envis CANDIDATE_COMMIT "$commit" || problems+="CANDIDATE_COMMIT=($why) " + envis CANDIDATE_PUBLISHED_AT "$published_at" || problems+="CANDIDATE_PUBLISHED_AT=($why) " + if [[ -z "$digest" ]]; then problems+="(release-candidate-evidence.json names no image digest) " + elif ! envis CANDIDATE_DIGEST "$digest"; then problems+="CANDIDATE_DIGEST=($why) "; fi + grep -q "rc${rcnum}-canary-component-" "$tmp/rcc.yml" || problems+="no rc${rcnum}-canary-component names " + # The published-action job must use exactly this candidate's commit. + pin_result="$("$tmp/pincheck" all "$commit" "$tag" "$tmp/rcc.yml")" || abort "workflow-pin parser failed" + if [[ "$pin_result" != VERIFIED ]]; then + problems+="published-action pin is not @${commit} " + fi + # A timed canary counts only from a schedule run inside the plan job's + # windows, so the dated crons must fire inside both of them. + sched="$("$tmp/pincheck" schedule "$published_at" - "$tmp/rcc.yml" 2>"$tmp/sched.err")" || + abort "workflow-pin parser failed: $(cat "$tmp/sched.err")" + [[ "$sched" == VERIFIED ]] || problems+="crons ($(sed -n 's/^rc-preflight pins: //p' "$tmp/sched.err" | tail -1)) " + if grep -Eo 'rc[0-9]+-canary-component-|reports/rc[0-9]+-action' "$tmp/rcc.yml" | grep -vqE "^(rc${rcnum}-|reports/rc${rcnum}-)"; then + problems+="other candidates' artifact/report names remain " + fi + if [[ -z "$problems" ]]; then say VERIFIED "designated: release-candidate-canary" "tag, commit, digest $digest, published $published_at, rc${rcnum} names, crons inside both timed windows"; ok_designations=$((ok_designations + 1)) + else say PENDING "designated: release-candidate-canary" "main still differs: $problems"; fi + + pub="$(epoch "$published_at")" + if [[ "$ok_designations" -ne 3 ]]; then + for item in "canary manual" "canary t-plus-24h" "canary t-plus-72h" "scheduled campaigns (4)"; do + say PENDING "$item" "requires main to be designated to $tag" + done + else + since="$pub" + for f in .github/campaign-candidate.json .github/workflows/consumer-canary.yml .github/workflows/release-candidate-canary.yml; do + c="$(last_change "$f")"; (( c > since )) && since="$c" + done + since_iso="$(date -u -d "@$since" +%Y-%m-%dT%H:%M:%SZ)" + need "repos/${repo}/actions/workflows/release-candidate-canary.yml/runs?created=%3E%3D${since_iso}&per_page=100" "$tmp/cruns.json" + # A canary run counts only if a job after the planning job succeeded: + # that proves the plan job accepted its window (out-of-window runs skip + # every later job and still conclude "success"). + executed() { + need "repos/${repo}/actions/runs/$1/jobs?per_page=100" "$tmp/jobs-$1.json" + jq -e '[.jobs[] | select(.name != "Select RC observation window" and .conclusion == "success")] | length > 0' "$tmp/jobs-$1.json" >/dev/null + } + canary() { # + local id st h hit="" + while IFS=$'\t' read -r id st; do + [[ -n "$id" ]] || continue + h=$(( ($(epoch "$st") - pub) / 3600 )) + # +-1h: the run may start just before its plan job crosses the hour; + # executed() below proves the plan job accepted the window. + (( h >= $3 - 1 && h <= $4 + 1 )) || continue + if executed "$id"; then hit="$id started $st"; break; fi + done < <(jq -r --arg e "$2" '.workflow_runs[] | select(.event == $e and .status == "completed" and .conclusion == "success") | "\(.id)\t\(.run_started_at)"' "$tmp/cruns.json") + if [[ -n "$hit" ]]; then say VERIFIED "canary $1" "$2 run $hit, past its plan job (candidate binding: readiness report)" + elif (( now < pub + ($4 + 2) * 3600 )); then say PENDING "canary $1" "window ${3}-${4}h after publication open or not reached" + else say MISSING "canary $1" "no executed $2 run ${3}-${4}h after publication"; fi + } + canary manual workflow_dispatch 0 100000 + canary t-plus-24h schedule 20 30 + canary t-plus-72h schedule 68 78 + need "repos/${repo}/actions/workflows/latest-kernel-compatibility.yml/runs?event=schedule&created=%3E%3D${since_iso}&per_page=100" "$tmp/camp.json" + n="$(jq '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success")] | length' "$tmp/camp.json")" + if [[ "$n" -ge 4 ]]; then say VERIFIED "scheduled campaigns (4)" "$n successful scheduled runs since designation (spacing and binding: readiness report)" + else say PENDING "scheduled campaigns (4)" "$n of 4 successful scheduled runs since designation ($since_iso)"; fi + fi + need "repos/${repo}/actions/workflows/rollback-drill.yml/runs?created=%3E%3D${published_at}&per_page=50" "$tmp/drill.json" + drill="$(jq -r '[.workflow_runs[] | select(.status == "completed" and .conclusion == "success") | .id] | first // empty' "$tmp/drill.json")" + if [[ -n "$drill" ]]; then say PENDING "rollback drill run" "run $drill succeeded since publication; which candidate it drilled is checked only by the readiness report" + else say PENDING "rollback drill run" "no successful run since publication"; fi +fi +say PENDING "Falco expanded entry" "operator-held; bound by production-readiness-report.sh" +say PENDING "fail-closed promotion incident note" "operator-held; bound by production-readiness-report.sh" +say PENDING "operator promotion confirmation" "operator-held; never written by tooling" +exit "$missing" diff --git a/scripts/rc-preflight_test.sh b/scripts/rc-preflight_test.sh new file mode 100755 index 0000000..94f244a --- /dev/null +++ b/scripts/rc-preflight_test.sh @@ -0,0 +1,383 @@ +#!/usr/bin/env bash +# Regression cover for scripts/rc-preflight.sh, the read-only RC inventory. +# +# A fake `gh` answers `gh api ` from a route table (glob -> HTTP status -> +# fixture file) and `gh release download` from a fixture SHA256SUMS, and logs +# every call so the test can prove nothing but reads happened. "Now" is fixed +# with BPFCOMPAT_PREFLIGHT_NOW. The checkout's own release.yaml, CHANGELOG, +# lock and distribution policy are read as they are. +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +PRE="$ROOT_DIR/scripts/rc-preflight.sh" +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +failures=0 +fail() { echo "[rc-preflight-test] FAIL: $*" >&2; failures=$((failures + 1)); } +ok() { echo "[rc-preflight-test] ok: $*"; } + +version="$(awk -F': ' '$1 == "release_version" { print $2 }' "$ROOT_DIR/release.yaml")" +TAG="v${version}" +[[ "$TAG" =~ -rc\.[0-9]+$ ]] || { echo "[rc-preflight-test] skip: release.yaml is not a prerelease ($version)"; exit 0; } +VSHA="$(awk -F= '$1 == "reproduces" { split($2, a, " "); print a[2] }' "$ROOT_DIR/validator/build-inputs.lock")" +C1="$(printf 'a%.0s' {1..40})" C2="$(printf 'b%.0s' {1..40})" TAGOBJ="$(printf 'c%.0s' {1..40})" +DIG="sha256:$(printf 'd%.0s' {1..64})" +RCN="${TAG##*-rc.}" +PUB="2026-11-02T10:00:00Z" +PUB_EPOCH="$(date -u -d "$PUB" +%s)" +at() { date -u -d "@$((PUB_EPOCH + $1 * 3600 + 600))" +%Y-%m-%dT%H:%M:%SZ; } # -> ISO, 10 min past the hour +cron_at() { date -u -d "@$((PUB_EPOCH + $1))" '+%-M %-H %-d %-m *'; } # -> dated cron line +DESIGNATED="$(date -u -d "@$((PUB_EPOCH + 300))" +%Y-%m-%dT%H:%M:%SZ)" # designation merged 5 min after T0 + +mkdir -p "$tmp/bin" +cat >"$tmp/bin/gh" <<'EOF' +#!/usr/bin/env bash +printf '%s\n' "$*" >>"$GH_LOG" +if [[ "$1" == api ]]; then + while IFS=$'\t' read -r glob status file; do + [[ -z "$glob" || "$glob" == \#* ]] && continue + # shellcheck disable=SC2053 + if [[ "$2" == $glob ]]; then + case "$status" in + 200) cat "$FIX/$file"; exit 0 ;; + *) echo "gh: HTTP $status (https://api.github.com/$2)" >&2; exit 1 ;; + esac + fi + done <"$FIX/routes" + echo "gh: HTTP 404 (unrouted $2)" >&2; exit 1 +fi +if [[ "$1 $2" == "release download" ]]; then + dir="" pat="" + while [[ $# -gt 0 ]]; do case "$1" in --dir) dir="$2";; --pattern) pat="$2";; esac; shift; done + if [[ -f "$FIX/download-fail" ]]; then echo "gh: HTTP 403" >&2; exit 1; fi + mkdir -p "$dir"; cp "$FIX/$pat" "$dir/$pat"; exit 0 +fi +echo "gh shim: refusing non-read command: $*" >&2; exit 9 +EOF +chmod +x "$tmp/bin/gh" + +b64() { base64 -w0 "$1"; } +contents() { jq -n --arg c "$(b64 "$1")" '{content: $c}'; } + +# scenario : a fixture dir with a published release by default. +scenario() { + F="$tmp/$1"; mkdir -p "$F" + jq -n --arg s "$TAGOBJ" '{object: {type: "tag", sha: $s}}' >"$F/ref.json" + jq -n --arg s "$C1" '{object: {type: "commit", sha: $s}}' >"$F/tagobj.json" + jq -n --arg t "$TAG" '{workflow_runs: [{id: 900, run_attempt: 1, head_branch: $t, status: "completed", conclusion: "success"}]}' >"$F/runs.json" + names=(bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-validator-static-linux-amd64 bpfcompat-third-party-licenses.tar.gz + bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json SHA256SUMS SHA256SUMS.sig SHA256SUMS.crt + bpfcompat.sbom.cdx.json bpfcompat.sbom.cdx.sig bpfcompat.sbom.cdx.crt release-candidate-evidence.json) + printf '%s\n' "${names[@]}" | jq -R '{name: .}' | jq -s --arg t "$TAG" --arg p "$PUB" \ + '[{tag_name: $t, draft: false, prerelease: true, published_at: $p, assets: .}]' >"$F/releases.json" + { for n in bpfcompat-linux-amd64 bpfcompat-linux-arm64 bpfcompat-third-party-licenses.tar.gz bpfcompat-validator-relink-kit-linux-amd64.tar.gz validator-build-inputs.json; do + printf '%064d %s\n' 1 "$n"; done + printf '%s bpfcompat-validator-static-linux-amd64\n' "$VSHA"; } >"$F/SHA256SUMS" + contents "$ROOT_DIR/distribution/policy.json" >"$F/policy.json" + printf 'jobs:\n test:\n steps:\n - name: x\n # bpfcompat-pin: candidate\n uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C1" "$TAG" >"$F/consumer-raw.yml"; contents "$F/consumer-raw.yml" >"$F/consumer.json" + printf 'on:\n schedule:\n - cron: "%s"\n - cron: "%s"\nenv:\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\njobs:\n x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' \ + "$(cron_at $((24 * 3600 + 14 * 60)))" "$(cron_at $((72 * 3600 + 14 * 60)))" \ + "$TAG" "$C1" "$DIG" "$PUB" "$RCN" "$C1" "$TAG" "$RCN" >"$F/rcc-raw.yml"; contents "$F/rcc-raw.yml" >"$F/rcc.json" + jq -n --arg i "ghcr.io/kernel-guard/bpfcompat@$DIG" '{image: $i}' >"$F/release-candidate-evidence.json" + jq -n '[{sha: "e000000000000000000000000000000000000000", commit: {committer: {date: "2020-01-01T00:00:00Z"}}}]' >"$F/commits.json" + jq -n --arg m "$DESIGNATED" '[{merged_at: $m, base: {ref: "main"}}]' >"$F/pulls.json" + jq -n '{jobs: [{name: "Select RC observation window", conclusion: "success"}, {name: "Clean install", conclusion: "success"}]}' >"$F/jobs.json" + jq -n --arg m "$(at 0)" --arg a "$(at 25)" --arg b "$(at 70)" '{workflow_runs: [ + {id: 1, run_attempt: 1, event: "workflow_dispatch", status: "completed", conclusion: "success", run_started_at: $m}, + {id: 2, run_attempt: 1, event: "schedule", status: "completed", conclusion: "success", run_started_at: $a}, + {id: 3, run_attempt: 1, event: "schedule", status: "completed", conclusion: "success", run_started_at: $b}]}' >"$F/cruns.json" + jq -n '{workflow_runs: [range(4) | {status: "completed", conclusion: "success"}]}' >"$F/camp.json" + jq -n '{workflow_runs: [{id: 77, status: "completed", conclusion: "success"}]}' >"$F/drill.json" + cat >"$F/routes" <: campaign-candidate.json binds the candidate and the digests +# of the fixture's current SHA256SUMS and candidate evidence, unless the +# scenario pinned its own (cc-fixed). +bind_cc() { + [[ -e "$1/cc-fixed" ]] && return 0 + jq -n --arg t "$TAG" --arg c "$C1" --arg s "$(sha256sum "$1/SHA256SUMS" | cut -d' ' -f1)" \ + --arg e "$(sha256sum "$1/release-candidate-evidence.json" | cut -d' ' -f1)" \ + '{tag: $t, commit_sha: $c, checksums_sha256: $s, candidate_evidence_sha256: $e}' >"$1/cc-raw.json" + contents "$1/cc-raw.json" >"$1/cc.json" +} +run() { # [args...] + local s="$1" h="$2"; shift 2 + bind_cc "$tmp/$s" + : >"$tmp/gh.log" + set +e + PATH="$tmp/bin:$PATH" FIX="$tmp/$s" GH_LOG="$tmp/gh.log" BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + h * 3600)) \ + bash "$PRE" --tag "$TAG" "$@" >"$tmp/out" 2>&1 + rc=$? + set -e +} +line() { grep -E "^$1 +$2 " "$tmp/out" >/dev/null; } +show() { sed 's/^/ | /' "$tmp/out" >&2; } + +# 1. Before the tag: everything after it is PENDING, nothing is MISSING. +scenario pretag; route pretag $'repos/*/git/ref/tags/*\t404\tnone' +run pretag 0 +if [[ $rc -eq 0 ]] && line PENDING "tag exists" && line PENDING "validator identity" && line PENDING "canary t-plus-72h" && + ! grep -q '^MISSING' "$tmp/out" && ! grep -q '^VERIFIED tag' "$tmp/out"; then ok "no tag: later items PENDING, exit 0" +else fail "no tag (rc=$rc)"; show; fi + +# 2. A lookup failure is not "not found": the report aborts (exit 3). +scenario apierr; route apierr $'repos/*/git/ref/tags/*\t502\tnone' +run apierr 0 +if [[ $rc -eq 3 ]] && ! grep -q 'tag exists' "$tmp/out" && grep -q 'lookup failed' "$tmp/out"; then ok "API failure aborts instead of reporting PENDING" +else fail "API failure (rc=$rc)"; show; fi +scenario apierr2; route apierr2 $'repos/*/releases?per_page=100\t500\tnone' +run apierr2 0 +if [[ $rc -eq 3 ]] && ! grep -q 'release staged' "$tmp/out"; then ok "a later lookup failure aborts too" +else fail "later API failure (rc=$rc)"; show; fi + +# 3. Published release, all evidence observable, inside every window. +scenario full +run full 200 --commit "$C1" +if [[ $rc -eq 0 ]] && line VERIFIED "tag exists" && grep -q "as approved" "$tmp/out" && line VERIFIED "tag build" && + line VERIFIED "release staged" && line VERIFIED "SHA256SUMS lists required assets" && line VERIFIED "validator identity" && + line VERIFIED "optional assets match tagged policy" && line VERIFIED "RC published as prerelease" && + line VERIFIED "designated: campaign-candidate.json" && line VERIFIED "designated: consumer-canary candidate pin" && + line VERIFIED "designated: release-candidate-canary" && line VERIFIED "canary manual" && line VERIFIED "canary t-plus-24h" && + line VERIFIED "canary t-plus-72h" && line VERIFIED "scheduled campaigns" && line PENDING "rollback drill run" && + grep -q 'which candidate it drilled' "$tmp/out" && + line PENDING "operator promotion confirmation"; then ok "annotated tag dereferenced; every observable item VERIFIED" +else fail "full scenario (rc=$rc)"; show; fi +if awk '!/^api / && !/^release download .*--pattern (SHA256SUMS|release-candidate-evidence\.json) /' "$tmp/gh.log" | grep -q .; then fail "non-read gh call: $(cat "$tmp/gh.log")" +elif grep -qE -- '(-X|--method|-f |-F |--field|--input)' "$tmp/gh.log"; then fail "gh api called with a method or body" +else ok "only GET api calls and the two text downloads were made"; fi + +# 4. Wrong identities are MISSING. +run full 200 --commit "$C2" +if [[ $rc -eq 1 ]] && line MISSING "tag exists"; then ok "tag on another commit than approved: MISSING"; else fail "commit mismatch (rc=$rc)"; show; fi +scenario badval; sed -i "s/^$VSHA /$(printf '%064d' 7) /" "$tmp/badval/SHA256SUMS" +run badval 200 +if [[ $rc -eq 1 ]] && line MISSING "validator identity"; then ok "different validator digest: MISSING"; else fail "validator mismatch (rc=$rc)"; show; fi +scenario noasset; jq '.[0].assets |= map(select(.name != "bpfcompat-third-party-licenses.tar.gz"))' "$tmp/noasset/releases.json" >"$tmp/x" && mv "$tmp/x" "$tmp/noasset/releases.json" +run noasset 200 +if [[ $rc -eq 1 ]] && line MISSING "release staged" && grep -q 'absent: bpfcompat-third-party-licenses.tar.gz' "$tmp/out"; then ok "missing release asset: MISSING" +else fail "missing asset (rc=$rc)"; show; fi +scenario optional; printf '%064d bpfcompat-validator-native-sources.tar\n' 5 >>"$tmp/optional/SHA256SUMS" +run optional 200 +if [[ $rc -eq 1 ]] && line MISSING "optional assets match tagged policy"; then ok "optional asset shipped against the tagged (undecided) policy: MISSING" +else fail "optional asset (rc=$rc)"; show; fi + +# 5. Time: windows open are PENDING, closed and empty are MISSING; elapsed time never VERIFIES. +scenario early; jq '.workflow_runs = [.workflow_runs[0]]' "$tmp/early/cruns.json" >"$tmp/x" && mv "$tmp/x" "$tmp/early/cruns.json" +jq '.workflow_runs = []' "$tmp/early/camp.json" >"$tmp/x" && mv "$tmp/x" "$tmp/early/camp.json" +jq '.workflow_runs = []' "$tmp/early/drill.json" >"$tmp/x" && mv "$tmp/x" "$tmp/early/drill.json" +run early 50 +if line PENDING "canary t-plus-72h" && line MISSING "canary t-plus-24h" && line PENDING "scheduled campaigns" && + grep -q '0 of 4' "$tmp/out" && line PENDING "rollback drill run"; then ok "50h: T+24h window closed empty is MISSING, T+72h still PENDING" +else fail "time windows at 50h (rc=$rc)"; show; fi +run early 400 +if line MISSING "canary t-plus-72h" && line PENDING "scheduled campaigns" && ! line VERIFIED "scheduled campaigns"; then + ok "400h with no runs: T+72h MISSING, campaigns still PENDING (never VERIFIED by time)" +else fail "time windows at 400h (rc=$rc)"; show; fi +scenario late; jq --arg l "$(at 35)" '.workflow_runs[1].run_started_at = $l' "$tmp/late/cruns.json" >"$tmp/x" && mv "$tmp/x" "$tmp/late/cruns.json" +run late 200 +if line MISSING "canary t-plus-24h"; then ok "a schedule run outside 20-30h does not count as T+24h"; else fail "out-of-window run counted (rc=$rc)"; show; fi +scenario failedrun; jq '.workflow_runs[2].conclusion = "failure"' "$tmp/failedrun/cruns.json" >"$tmp/x" && mv "$tmp/x" "$tmp/failedrun/cruns.json" +run failedrun 200 +if line MISSING "canary t-plus-72h"; then ok "a failed in-window run does not count"; else fail "failed run counted (rc=$rc)"; show; fi + +# 6. Draft release: published-only items wait. +scenario draft; jq '.[0].draft = true | .[0].published_at = null' "$tmp/draft/releases.json" >"$tmp/x" && mv "$tmp/x" "$tmp/draft/releases.json" +run draft 200 +if [[ $rc -eq 0 ]] && line PENDING "RC published as prerelease" && line PENDING "canary manual" && line VERIFIED "validator identity"; then + ok "draft: verified assets, publication-dependent items PENDING" +else fail "draft (rc=$rc)"; show; fi + +# 7. Designation must cover every candidate-bound value; runs before it do not count. +scenario stale; sed -i "s/rc${RCN}-canary-component-clean-install-1/rc4-canary-component-clean-install-1/" "$tmp/stale/rcc-raw.yml" +contents "$tmp/stale/rcc-raw.yml" >"$tmp/stale/rcc.json" +run stale 200 +if line PENDING "designated: release-candidate-canary" && grep -q "artifact/report names" "$tmp/out" && line PENDING "canary manual" && + grep -q 'requires main to be designated' "$tmp/out"; then ok "leftover previous-candidate artifact names: not designated, canaries not counted" +else fail "stale designation (rc=$rc)"; show; fi +scenario olddigest; sed -i "s/CANDIDATE_DIGEST: .*/CANDIDATE_DIGEST: sha256:$(printf 'e%.0s' {1..64})/" "$tmp/olddigest/rcc-raw.yml" +contents "$tmp/olddigest/rcc-raw.yml" >"$tmp/olddigest/rcc.json" +run olddigest 200 +if line PENDING "designated: release-candidate-canary" && grep -q "CANDIDATE_DIGEST=(\"sha256:$(printf 'e%.0s' {1..64})\" in the workflow env)" "$tmp/out"; then + ok "previous candidate digest left in the canary: not designated, and the stale value is shown" +else fail "stale digest (rc=$rc)"; show; fi +scenario runtext +{ printf 'jobs:\n notes:\n steps:\n - run: |\n CANDIDATE_TAG: %s\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$TAG" "$C1" "$DIG" "$PUB" + printf ' x:\n name: rc%s-canary-component-clean-install-1\n uses: Kernel-Guard/bpfcompat@%s # %s\n out: reports/rc%s-action.json\n' "$RCN" "$C1" "$TAG" "$RCN" + printf 'env:\n CANDIDATE_TAG: v0.0.0-rc.1\n CANDIDATE_COMMIT: %s\n CANDIDATE_DIGEST: %s\n CANDIDATE_PUBLISHED_AT: "%s"\n' "$C1" "$DIG" "$PUB"; } >"$tmp/runtext/rcc-raw.yml" +contents "$tmp/runtext/rcc-raw.yml" >"$tmp/runtext/rcc.json" +run runtext 200 +if line PENDING "designated: release-candidate-canary" && grep -q 'CANDIDATE_TAG=' "$tmp/out"; then ok "current values in run text do not hide a stale workflow env" +else fail "run-text env (rc=$rc)"; show; fi +scenario ccdigest; touch "$tmp/ccdigest/cc-fixed" +jq --arg s "$(printf 'f%.0s' {1..64})" '.checksums_sha256 = $s' "$tmp/ccdigest/cc-raw.json" >"$tmp/x" && mv "$tmp/x" "$tmp/ccdigest/cc-raw.json" +contents "$tmp/ccdigest/cc-raw.json" >"$tmp/ccdigest/cc.json" +run ccdigest 200 +if line PENDING "designated: campaign-candidate.json" && grep -q "checksums_sha256 is not this release's SHA256SUMS" "$tmp/out" && line PENDING "canary manual"; then + ok "campaign pin naming another release's SHA256SUMS digest: not designated" +else fail "stale campaign digest (rc=$rc)"; show; fi +scenario stalecron +sed -i 's/^ - cron: .*/ - cron: "44 17 15 10 *"/' "$tmp/stalecron/rcc-raw.yml"; contents "$tmp/stalecron/rcc-raw.yml" >"$tmp/stalecron/rcc.json" +run stalecron 200 +if line PENDING "designated: release-candidate-canary" && grep -qF 'crons (no schedule cron fires in [T0+20h, T0+31h))' "$tmp/out"; then + ok "previous candidate's dated crons left in the canary: not designated" +else fail "stale crons (rc=$rc)"; show; fi +scenario prerun; LATE_DES="$(date -u -d "@$((PUB_EPOCH + 3 * 3600))" +%Y-%m-%dT%H:%M:%SZ)" +jq --arg d "$LATE_DES" '.[0].merged_at = $d' "$tmp/prerun/pulls.json" >"$tmp/x" && mv "$tmp/x" "$tmp/prerun/pulls.json" +# The API filter is what excludes earlier runs: serve only what created>=designation would return. +jq '.workflow_runs |= map(select(.event == "schedule"))' "$tmp/prerun/cruns.json" >"$tmp/prerun/cruns-filtered.json" +route prerun $'repos/*/actions/workflows/release-candidate-canary.yml/runs*\t200\tcruns-filtered.json' +run prerun 200 +if grep -qF "created=%3E%3D${LATE_DES}" "$tmp/gh.log" && line PENDING "canary manual" && line VERIFIED "canary t-plus-24h"; then + ok "runs are listed from the designation merge time, not from publication" +else fail "pre-designation run (rc=$rc)"; show; fi +scenario planonly; jq '.jobs = [.jobs[0], {name: "Clean install", conclusion: "skipped"}]' "$tmp/planonly/jobs.json" >"$tmp/x" && mv "$tmp/x" "$tmp/planonly/jobs.json" +run planonly 200 +if line MISSING "canary t-plus-24h" && line MISSING "canary t-plus-72h"; then ok "a run whose later jobs the plan job skipped does not count" +else fail "plan-only run counted (rc=$rc)"; show; fi + +scenario oldaction; printf ' uses: Kernel-Guard/bpfcompat@%s # v0.4.0-rc.4\n' "$C2" >>"$tmp/oldaction/rcc-raw.yml" +contents "$tmp/oldaction/rcc-raw.yml" >"$tmp/oldaction/rcc.json" +run oldaction 200 +if line PENDING "designated: release-candidate-canary" && grep -q 'published-action pin' "$tmp/out"; then ok "canary Action pin left on another commit: not designated" +else fail "stale action pin (rc=$rc)"; show; fi +scenario nomarker; printf 'jobs: {}\n' >"$tmp/nomarker/consumer-raw.yml"; contents "$tmp/nomarker/consumer-raw.yml" >"$tmp/nomarker/consumer.json" +run nomarker 200 +if line PENDING "designated: consumer-canary candidate pin" && grep -q "requires one '# bpfcompat-pin: candidate' marker" "$tmp/out"; then ok "missing pin marker is reported, not a silent exit" +else fail "missing pin marker (rc=$rc)"; show; fi + +# An actual stale action cannot be hidden by a correct commented line or an +# alternative YAML scalar spelling. Keep these end-to-end with the fake API. +for variant in commented commentonly quoted lowercase tabbed ref subpath wrongtag; do + scenario "pin-$variant" + file="$F/rcc-raw.yml" + case "$variant" in + commented) + sed -i "s/uses: Kernel-Guard\/bpfcompat@$C1/# uses: Kernel-Guard\/bpfcompat@$C1/" "$file" + printf ' uses: Kernel-Guard/bpfcompat@v0.4.0-rc.4\n' >>"$file" ;; + commentonly) sed -i "s/uses: Kernel-Guard/# uses: Kernel-Guard/" "$file" ;; + quoted) printf ' uses: "Kernel-Guard/bpfcompat@%s" # %s\n' "$C2" "$TAG" >>"$file" ;; + lowercase) printf ' uses: kernel-guard/bpfcompat@%s # %s\n' "$C2" "$TAG" >>"$file" ;; + tabbed) printf ' uses:\tKernel-Guard/bpfcompat@%s # %s\n' "$C2" "$TAG" >>"$file" ;; + ref) printf ' uses: Kernel-Guard/bpfcompat@main\n' >>"$file" ;; + subpath) printf ' uses: Kernel-Guard/bpfcompat/sub@%s # %s\n' "$C1" "$TAG" >>"$file" ;; + wrongtag) sed -i "s/# $TAG/# v9.9.9-rc.1/" "$file" ;; + esac + contents "$file" >"$F/rcc.json" + run "pin-$variant" 200 + if line PENDING "designated: release-candidate-canary" && line PENDING "canary manual"; then + ok "Action $variant cannot establish candidate designation" + else fail "Action $variant falsely designated (rc=$rc)"; show; fi +done +for variant in commented duplicate stale-after-comment; do + scenario "consumer-$variant" + file="$F/consumer-raw.yml" + case "$variant" in + commented) sed -i 's/ uses:/ # uses:/' "$file" ;; + duplicate) cat "$file" >>"$F/second"; cat "$F/second" >>"$file" ;; + stale-after-comment) + sed -i 's/ uses:/ # uses:/' "$file" + printf ' uses: Kernel-Guard/bpfcompat@%s # %s\n' "$C2" "$TAG" >>"$file" ;; + esac + contents "$file" >"$F/consumer.json" + run "consumer-$variant" 200 + if line PENDING "designated: consumer-canary candidate pin" && line PENDING "canary manual"; then + ok "consumer $variant cannot establish candidate designation" + else fail "consumer $variant falsely designated (rc=$rc)"; show; fi +done +scenario quotedgood +sed -i "s|uses: Kernel-Guard/bpfcompat@$C1|uses: 'kernel-guard/bpfcompat@$C1'|" "$F/rcc-raw.yml" +sed -i "s|uses: Kernel-Guard/bpfcompat@$C1|uses: \"Kernel-Guard/bpfcompat@$C1\"|" "$F/consumer-raw.yml" +contents "$F/rcc-raw.yml" >"$F/rcc.json"; contents "$F/consumer-raw.yml" >"$F/consumer.json" +run quotedgood 200 +if line VERIFIED "designated: release-candidate-canary" && line VERIFIED "designated: consumer-canary candidate pin"; then + ok "correct quoted pins and case-insensitive repository names are accepted" +else fail "valid quoted pins refused (rc=$rc)"; show; fi + +# The workflow's plan job can start after the run crosses its hour boundary. +scenario hour19 +jq --arg t "$(at 19)" '.workflow_runs[1].run_started_at = $t' "$F/cruns.json" >"$tmp/x" && mv "$tmp/x" "$F/cruns.json" +run hour19 200 +if line VERIFIED "canary t-plus-24h"; then ok "hour-19 run with successful executed jobs remains observable" +else fail "hour-19 executed run refused (rc=$rc)"; show; fi +jq '.jobs = [.jobs[0], {name: "Clean install", conclusion: "skipped"}]' "$F/jobs.json" >"$tmp/x" && mv "$tmp/x" "$F/jobs.json" +run hour19 200 +if line MISSING "canary t-plus-24h"; then ok "hour-19 plan-only run never becomes VERIFIED" +else fail "hour-19 plan-only run counted (rc=$rc)"; show; fi + +# 8. Malformed answers abort; nothing is inferred from them. +scenario badjson; printf 'not json' >"$tmp/badjson/releases.json" +run badjson 200 +if [[ $rc -eq 3 ]] && grep -q 'not JSON' "$tmp/out" && ! grep -q 'release staged' "$tmp/out"; then ok "non-JSON API body aborts"; else fail "non-JSON body (rc=$rc)"; show; fi +scenario badb64; printf '{"content": "%%%%not-base64%%%%"}' >"$tmp/badb64/policy.json" +run badb64 200 +if [[ $rc -eq 3 ]] && grep -q 'cannot decode' "$tmp/out"; then ok "undecodable file contents abort"; else fail "bad base64 (rc=$rc)"; show; fi +scenario dlfail; touch "$tmp/dlfail/download-fail" +run dlfail 200 +if [[ $rc -eq 3 ]] && grep -q 'cannot download SHA256SUMS' "$tmp/out" && ! grep -q '^MISSING validator identity' "$tmp/out"; then + ok "a failed asset download aborts instead of reporting MISSING" +else fail "download failure (rc=$rc)"; show; fi + +scenario badev; printf 'garbage' >"$tmp/badev/release-candidate-evidence.json" +run badev 200 +if [[ $rc -eq 3 ]] && grep -q 'release-candidate-evidence.json of .* is not JSON' "$tmp/out"; then ok "unparsable candidate evidence aborts" +else fail "bad evidence (rc=$rc)"; show; fi +scenario ccarray; touch "$tmp/ccarray/cc-fixed"; printf '[]' >"$tmp/ccarray/cc-raw.json"; contents "$tmp/ccarray/cc-raw.json" >"$tmp/ccarray/cc.json" +run ccarray 200 +if [[ $rc -eq 3 ]] && grep -q 'campaign-candidate.json on main is not a JSON object' "$tmp/out"; then ok "a campaign pin that is not a JSON object aborts" +else fail "non-object campaign pin (rc=$rc)"; show; fi + +# 9. Building the pin parser: no module or toolchain download, and the caller's +# toolchain choice is kept (an older Go on PATH may switch to a cached +# go.mod toolchain). A caller GOPRIVATE would otherwise let Go fetch a +# toolchain directly despite GOPROXY=off. +REAL_GO="$(command -v go)" +mkdir -p "$tmp/goshim" "$tmp/gofail" +cat >"$tmp/goshim/go" <}" "\${GOPROXY-}" "\${GONOPROXY-}" "\${GONOSUMDB-}" "\${GOWORK-}" "\$*" >>"$tmp/go.log" +exec "$REAL_GO" "\$@" +SHIM +printf '#!/usr/bin/env bash\necho "go: go.mod requires go >= 9.9 (running go 1.0)" >&2\nexit 1\n' >"$tmp/gofail/go" +chmod +x "$tmp/goshim/go" "$tmp/gofail/go" +scenario gobuild; : >"$tmp/go.log" +set +e +PATH="$tmp/bin:$tmp/goshim:$PATH" FIX="$tmp/gobuild" GH_LOG="$tmp/gh.log" GOTOOLCHAIN="${GOTOOLCHAIN:-auto}" GOPRIVATE=golang.org GONOPROXY=golang.org/toolchain \ + BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + 200 * 3600)) bash "$PRE" --tag "$TAG" >"$tmp/out" 2>&1 +rc=$? +set -e +if [[ $rc -eq 0 ]] && grep -q "^GOTOOLCHAIN=${GOTOOLCHAIN:-auto} GOPROXY=off GONOPROXY=none.invalid GONOSUMDB=none.invalid GOWORK=off ARGS=build -mod=vendor " "$tmp/go.log"; then + ok "parser build keeps the caller's GOTOOLCHAIN and overrides GOPRIVATE/GONOPROXY, with GOPROXY=off, GOWORK=off and -mod=vendor" +else fail "parser build environment (rc=$rc): $(cat "$tmp/go.log")"; show; fi +scenario gofails +set +e +PATH="$tmp/bin:$tmp/gofail:$PATH" FIX="$tmp/gofails" GH_LOG="$tmp/gh.log" \ + BPFCOMPAT_PREFLIGHT_NOW=$((PUB_EPOCH + 200 * 3600)) bash "$PRE" --tag "$TAG" >"$tmp/out" 2>&1 +rc=$? +set -e +if [[ $rc -eq 3 ]] && grep -q 'cannot build the vendored workflow-pin parser' "$tmp/out" && ! grep -q 'designated: consumer-canary' "$tmp/out"; then + ok "an unusable Go aborts the report instead of reporting a pin status" +else fail "unusable Go (rc=$rc)"; show; fi + +# 10. Usage. + +if bash "$PRE" --tag v0.4.0 >/dev/null 2>&1; then fail "stable tag accepted"; else ok "non-RC tag refused"; fi + +if [[ $failures -ne 0 ]]; then echo "[rc-preflight-test] $failures failure(s)" >&2; exit 1; fi +echo "rc-preflight_test: all checks passed"