Skip to content

CI: run the legs a change can affect, by path class; the release PR, main and the release run everything #118

Closed

Description

@CaffeinatedCoder

Problem

Every pull request into release/2.3.0 runs the whole matrix: three test legs, Pack (with the smoke test against the packed feed and the SBOMs), and the smoke test packing its own feed. A docs-only change pays for all of it. Measured over the 11 successful runs since the prober left CI (2026-10-28 21:08 to 2026-10-29 08:21):

Job Median Max
Test (ubuntu-latest) 6.3 min 7.0
Test (windows-latest) 8.5 min 9.3
Test (macos-latest) 7.2 min 15.2
Pack 6.2 min 6.6
Consumer smoke test 5.7 min 6.3
Changes 0.1 min 0.1

Wall clock is 15.0 min median (12.6 to 21.4), 33 runner-minutes, because Pack and Consumer wait for the whole test matrix (#117). The Changes job skips only Pack and Consumer, and only when no listed path changed. It fails open and misses renames (#70). Nothing tells a deliberate skip from an accidental one (#71). Nothing runs the full matrix on its own schedule, or before the release publishes (#74).

Change

A change-class model, recorded as decision 113:

  • Lanes. Four: Linux tests (always), Windows, macOS, and Pack plus both smoke-test modes (pack). Over 150 runs, a Windows leg failed alone once for a src/ defect (the analyzer's code fix, CRLF, f9f64e5), and macOS never did.
  • Path classes. Prose (docs/, design/, any *.md outside src/) selects Linux, and when every path is prose the Linux leg builds and runs only test/Prose.slnf: Conventions, Samples, DocsQuery and Oracles tests. CHANGELOG.md adds pack. The Analyzers and Generators select Linux, Windows and the pack, and the other run-time packages Linux and the pack. test/, spikes/ and the other tools select Linux and Windows. The smoke test and its inputs, the selection tool, every build file and workflow, and any unlisted path select Linux, Windows and the pack.
  • Events. A PR into release/**, or a push to one, is selected by its diff. Only a PR into main (the release PR), a push to main, a dispatch, the nightly and the release run macOS, and they run every lane.
  • Selection in tools/CiLanes, a BCL tool (decision 92). It uses git diff --no-renames, and a conventions test holds it to what MSBuild says the pack reads (The Changes job fails open, misses renames, and does not list tools/Shared or .config/dotnet-tools.json #70).
  • A gate job, CI, over every job. It passes when the selected jobs succeeded and the others were skipped (CI: a gate job over every job, and which legs it requires #71), and it becomes the required check.
  • Portable file names, held by a Linux convention test before the OS legs may skip (CI: OS legs skip inert-only diffs, after a convention test for portable file names #73).
  • release.yml calls the full workflow before it verifies, and a nightly dispatches it on main and every release branch (Release: call the full CI workflow before publishing, and a nightly full run #74).

#70, #71, #73 and #74 are the parts. This issue is the model that ties them together, and #117 is in it.

Done when

The decision record exists. One PR implements it. That PR shows a docs-only change that runs only the Linux leg, a src/ change that runs every leg, and before/after run times for both.

Activity

  1. added this to the 2.3.0 milestone on Oct 29, 2026
  2. added
    choreBuild, CI, tooling, packaging, refactoring with no behaviour change
    area/releaseVersioning, changelogs, packaging and release wiring
    area/toolingThe repository's tools: prober, verdict, guard, docs-query, smoke test
    on Oct 29, 2026
  3. CaffeinatedCoder commented on Oct 29, 2026

    @CaffeinatedCoder
    OwnerAuthor

    Before and after

    The same one-line change, in throwaway draft PRs. "Before" PRs went into release/2.3.0 (the old workflow), and "after" PRs into release/ci-lanes-demo, cut from this branch at fce2663. Wall clock runs from the first job's start to the last job's end. Runner-minutes add up every job that ran.

    Change Before: jobs Before: wall Before: runner-min After: jobs After: wall After: runner-min
    docs-only (docs/closed-sets.md) #124: Linux, Windows, macOS 8.7 min 22.0 #126: Linux with the prose suite, gate 2.4 min 2.4
    src/ (src/CodoMetis.TypeKit/Fault.cs) #125: Linux, Windows, macOS, then Pack and Consumer 16.5 min 38.4 #127: Linux, Pack and Consumer side by side, gate 7.3 min 17.7
    • Docs-only: the prose suite's Linux leg is restore 0.2, build 0.9 and test 0.7 min (1,884 tests: Conventions, Samples, DocsQuery, Oracles).
    • src/: a core package change runs neither Windows nor macOS. The pack now starts beside the tests, not after them.
    • Changes and the gate: together they add 0.4 to 0.8 min to every run.
    • This PR's own run (40853425202): it touches workflows and tools/CiLanes, so Linux, Windows and the pack ran, without macOS. It took 9.3 min and 27.6 runner-minutes.
    • The demo base's push (40853528584): the same selection, 9.7 min.

    An Analyzers or Generators change adds the Windows leg (about 8.5 min). A PR into main, a push to main, the nightly run and the release run every leg, macOS included.

  4. CaffeinatedCoder commented on Oct 29, 2026

    @CaffeinatedCoder
    OwnerAuthor

    Done in #123, merged into release/2.3.0 as a92e5bc (decision 113). Closed by hand: Closes closes nothing on a merge into a release branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/releaseVersioning, changelogs, packaging and release wiringarea/toolingThe repository's tools: prober, verdict, guard, docs-query, smoke testchoreBuild, CI, tooling, packaging, refactoring with no behaviour change

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions