Repository navigation
ci: CodSpeed label gate and daily main, base-tier and full-size jobs, strict alloc budgets, Python tool pins, probe logs (v1.6.0, Refs #890) - #1124
Merged
Conversation
…the test/integration skip guard (Refs #890) Failing-first arm of three items of #890. Each gate is added with the fix left out, so that CI shows what it catches: - item 91: TESTING_STRICT_ALLOC_BUDGETS=1 on the allocation-guard step. The postgres Query_1col_1row guard measures 14 allocs/op against a budget of 4. - item 93: tool-pins-check.sh check 4. No workflow installs or runs a Python tool with a name==version of its own, and every requirements file under .github/ is in a Dependabot pip entry. ci.yml still runs `pipx run --spec zizmor==1.30.0`, so Lint is red, and the "good" fixtures of the self-test, which copy the real tree, are red with it. - item 96: TestAdaptiveSkipRoutingIntegration890 with an empty registry lists the 11 raw skips of test/integration/adaptive_*_test.go that bypass the gate; a CI step in the adaptive job runs it by name.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…and on main once a day (Refs #890) Items 7 and 97 of #890. The workflow's job minutes passed a 600-minute monthly count on 2026-10-12 and no job has had a macro runner since; the header now carries the numbers. A pull request runs the benchmarks while it carries the `performance` label (the run that adding any other label starts has a concurrency group of its own, so it cannot cancel the one in flight); main is measured by a daily schedule instead of every push, and a scheduled run replaces an older one still waiting for a runner. Adds a temporary hosted twin of the trigger, group and condition (l18-gate-proof.yml, label `spike` in place of `performance`) so that the gate can be shown on GitHub without a macro-runner minute. It is deleted in a later commit.
… retry logs, epoll arms fail, BenchmarkCloseChurn warning (Refs #890 #885) - 890.87: job iouring-base-tier runs ./internal/engine/iouring and the root package's Shutdown/Detach/SSE/Stream/WebSocket/Hijack/Drain tests with CELERIS_MAX_IOURING_TIER=base, asserts the cap took effect and tallies skips against an allow-list (empty in this commit: the first run lists them). - 890.92: job large-response builds ./test/server without -race and coverage and runs the eight tests that read lean761(), n836() or raceOn761 by name, printing wall time and peak RSS. - 890.88: the six keptProbeIOUring copies take a testing.TB and log one line per retry round, as the c714ProbeIOUring copies do. - 890.95: the epoll arms of pipeline_test.go and slowloris_synthetic_test.go fail where they skipped when epoll.New fails. - 885.21: the warning about the outlier samples on BenchmarkCloseChurn.
…t 14, zizmor pinned in a requirements file, the integration skip registry (Refs #890) - item 91: driver/postgres Query_1col_1row has the budget the driver costs, 14 allocs/op (measured 14 on linux/arm64 in 20 of 20 runs and on the GitHub amd64 runner), and the allocation-guard step stays strict. - item 93: zizmor==1.30.0 moves to .github/tools/requirements.txt, installed by `pip install -r` in a venv; dependabot.yml gets a pip entry for /.github/tools. The Dependabot CLI (v1.96.0, pip updater) proposes zizmor 1.30.0 to 1.31.1 from that file. Check 4 of tool-pins-check.sh passes and so do its fixtures. - item 96: skipOrFailUpswitch890 is the one definition of the gate in test/integration (the standby gate and the io_uring skip of adaptive_standby_886 use it); the ten raw skips of adaptive_hybrid_test.go are registered with their count and why, which item 95 leaves to the adaptive lane. - item 92: the large-response job also asserts that the 64 MiB and 16 MiB bodies ran and the lean ones did not. - item 87: base-tier-tally.sh has a self-test in the Lint job, and its allow-lists are data files.
…Refs #890 item 87) Run 40414278698 on b90001a: ./internal/engine/iouring took 473 s under -race with 554 top-level tests passed and 12 SKIP lines, the root package and ./test/server 4.6 s and 239 s with 107 passed and 19 SKIP lines, and TestShutdownDetachedPartialWriteIsFlushed806 passed on the capped probe. Each skip is named in .github/base-tier-skips-{iouring,root}.txt with its reason, the tally has a min-pass floor, and its self-test covers it.
…ed tier cap, a -race build of the full-size job); reverted by the next commit (Refs #890)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…item 7) Its runs are cited in the pull request: 40414920862 (label spike, ran, not cancelled by the run another label started, 40414979324 skipped), 40415458668 (a push with the label ran) and 40416207215 (cancelled by the run of the next push, 40416230527).
…rn (Refs #890 item 87) Run 40419578127 (the head after a merge with main) failed the job: main had brought in the TestH1Unclosed...889 family, nine tests whose io_uring, adaptive and adaptive-iouring arms skip with 'io_uring tier=base ...: no usable io_uring' (117 top-level PASS, 46 SKIP lines, 27 not named). An exact-name list would turn the job red for every test of that kind that a later pull request adds. The root list now has a pattern entry for the three arm names, honoured only where the log says tier=base under the skipped arm; the self-test covers an arm that logged it, one that skipped for another reason, one the pattern does not name and a top-level test named like an arm.
…er step timeouts, the CodSpeed schedule caveat (Refs #890 items 7, 87) - The pattern entry no longer takes the engine's startup line (tier=base) as evidence: an arm that started an engine and skipped for another reason fails. Self-test case added (17 cases). - -timeout 900 s and 600 s (about twice the measured 473 s and 239 s), job limit 30 minutes: a run with the cap lost ended after 23 minutes in run 40416230192. - codspeed.yml says that CodSpeed documents push as the baseline and does not mention schedule, and what to do if the first scheduled run is not recorded.
…1124, m1 m2 m4 n1) base-tier-tally-test.sh: a probe message of one parallel arm must not excuse another arm's skip (=== NAME / === CONT lines), parallel arms that each logged it pass, another tier=base message is no evidence, and a regular expression character in an allowed name is literal. tool-pins-check-test.sh: uvx name@version, uv tool run name@version and range pins (>=, ~=, <) are flagged, an @ that is no version and a redirection are not. Run against the previous scripts: 3 and 5 rows fail (Refs #890).
…check 4 sees range and uv pins, the integration guard scans helpers_test.go, closed PRs do not start CodSpeed, discovery for the full-size job (review r1 of #1124) - m1, m2, n1: base-tier-tally.sh credits a probe message to the test that is current on RUN, NAME and CONT lines, the evidence is the 'no usable io_uring' message the comments name, allowed names are compared as strings. The head's real root log (117 PASS, 46 SKIP, 100 NAME/CONT/PAUSE lines) still passes. - m4: tool-pins-check.sh check 4 flags >=, ~=, <, > pins and uvx/uv tool name@version. - m3, n2: the skip guard scans helpers_test.go (startEngine, three raw skips, registered as debt) and does not take a method for the gate function. - m6: the CodSpeed job runs only for an open pull request. - m5: the comments of the two new jobs say they gate releases. - m7: the full-size job fails when a Test function that reads the lean-shape helpers is not in its list. - M2: codspeed.yml's header names the fallbacks and the date of the quota reset. (Refs #890)
…the merged tree (Refs #890 item 87) Main 8936e89 (with #1123 and #1127) run through the job's logic in docker at CELERIS_MAX_IOURING_TIER=base: the iouring package gives 577 top-level PASS and 13 SKIP lines, the new one being TestListenOnAZonedLinkLocalServes885 (the container has no fe80:: address); root and test/server give 118 PASS and 46 SKIP lines, all allowed. The zoned test fails instead of skipping under CELERIS_REQUIRE_ZONED_PEER=1, as in the unit job, so both steps set it and the list stays as it was. min-pass floors 540 and 105.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane L18 CI-TESTS of v1.6.0. Workflow files, scripts and tests only; no production code. Refs #890, Refs #885, Refs #888.
Method: every item was checked on main (baa43d9) first; "premise" below is that check. Items are quoted by their opening words, with the ordinal that counts every checkbox in the body of the issue (the numbers of #1099 count only unchecked boxes and differ). The gates were added first with the fix left out, so a run shows what each catches (commit 1, run A), then the fixes (run D, head b56c8fd, was the first fully green run; the head is green too).
performancelabel gate + daily schedule for main (the loadgen half is not in this repo). Positive arm and cron cannot be shown before merge, see belowtest/server/race761_off_test.go... makes the large-response tests run their full-size shape"large-response, measured on hosted runners.github/tools/go.mod"skipOrFailIfStandbyAborted886... keeps its own inline" gategrep MAX_IOURING_TIER .github/found nothing)iouring-base-tierinternal/engine/epoll/adaptive_abort_844_linux_test.go(thepkg844 == "iouring"arm"const pkg844 = "epoll"(:44), so the arm and itst.Fatalf(:273-285) are dead code there; nothing can fail or skip on a host without io_uringkeptProbeIOUring"Logfper retry round, as thec714ProbeIOUringcopiest.Fatalfon the epoll arms, io_uring arms keep their skipsdriver/postgresTestAllocBudgets/Query_1col_1row"TESTING_STRICT_ALLOC_BUDGETS=1on the stepLogfatdetach_capture_alias_linux_test.go:333and the other copies)noteReadErrfeedsserverErrs,inbound_sequence_linux_test.go:197-204); the in-tree comment (backpressure_oracle_linux_test.go:1099) already matchesh2_continuation_streams_981_linux_test.go:248,:354); the controls half was owed (#1055: "control A must be rebuilt against the pre-#984 tree")-raceon a hosted runner (run A, 6.4 s in the item)Commits: 1
a61d910gates without their fixes; 2e0379d2CodSpeed; 3b90001athe two jobs, probe logs, epoll arms, warning; 4234e18bthe fixes for 1; 5b56c8fdthe base-tier allow-lists; thene7a797c(DO NOT MERGE, bad inputs for items 87 and 92) and its revert, the CONTRIBUTING sentence, the removal of the temporary CodSpeed twin,5556e7f(pattern entry in the root base-tier list),3438221(probe message as the evidence, shorter timeouts, the CodSpeed schedule caveat),a136253anda979806(review round 1: self-test rows first, then the fixes), the merge commitc804f3dandbc707c6(zoned peer required in the base-tier job, floors from the merged tree). Squash on merge.CI on the head
Head
3438221(before review round 1 and the merge): CI run 40422726424 success (all jobs, including the two new ones:iouring-base-tier12m36s, job 121920842823;large-response1m32s, job 121920842684), Coverage 40422726408 success, Drivers 40422726407 success, CodSpeed 40422726405 skipped (draft, no label).gh pr checkslists all 11 required names below as pass on this head.Required checks, before and after
The ruleset "Main — CI" requires these 11 names (
gh api repos/goceleris/celeris/rulesets/22358875), read before the change and unchanged after:Lint, Unit (root + middleware sub-modules), Conformance, Driver Conformance (Postgres + Redis), Build (ubuntu-latest), Build (macos-latest), Vulnerability Check, Cross-compile (windows/amd64), Cross-compile (windows/arm64), Portable fallback (windows-latest), Portable fallback (macos-latest).
name:of every job of ci.yml on main against this head: two jobs added (iouring-base-tier,large-response), none renamed or removed;diffof the two lists is those two lines. Both are NOT required checks (likeadaptive,iouring,zc-window): they report on a pull request but do not stop its merge. Making them required is an org-admin ruleset change that this PR does not make.890.69: controls A and E re-run
The item says: replace the sleeps with server-side conditions (done by #1055), "then re-run controls A and E to confirm the wire limit rows still fail". Control A is the pre-#984 processor (the defect: the CONTINUATION path opened its stream with
GetOrCreateStream, which checks neither the stream-ID order, nor the stream's state, nor SETTINGS_MAX_CONCURRENT_STREAMS); control E is no MAX_CONCURRENT_STREAMS admission in the header path. #1055 could not build the oldprocessor.goagainst main (OnHandlerPanic,MaxHeaderListSizeundefined). I rebuilt control A on current main instead of reverting the file: inhandleContinuationthe call toserveHeaderBlockis replaced byGetOrCreateStream+SetState(StateOpen)+completeHeaderBlock, which is what the pre-#984 code did; control E replacesTryOpenStreaminadmitHeadersbyGetOrCreateStream. Both are scratch worktrees, nothing committed. Docker golang:1.27.3,--cpus 2,--security-opt seccomp=unconfined --ulimit memlock=-1(so that the io_uring and adaptive arms run, not skip),CELERIS_REQUIRE_IOURING_WORKERS=1,-race -count=1, the two testsTestH2MaxConcurrentStreamsHeadersContinuation981andTestH2StreamIDOrderHeadersContinuation981:headers+continuationrows of epoll, io_uring and adaptive: 6 of the MaxConcurrentStreams rows (slots-come-back,refused-id-reused) and the 6 reused-id and lower-id rows of the stream-ID test; everyplain-headersrow and every std row passes. The defect is detected, on the one path it lives in.plain-headersandheaders+continuation), the 6 stream-ID rows pass: the same 12 as #1055's control E.The sleeps are gone and both controls still fail the rows they are for. Logs: scratchpad
l18citests/logs/ctl-{celeris,ctlA,ctlE}.log. The item can be ticked for the controls half.890.7 and 890.97: CodSpeed
Data (
gh api repos/goceleris/celeris/actions/workflows/codspeed.yml/runs, then the jobs of each completed run; 682 runs, 250 jobs): the 163 jobs that got a macro runner (2026-09-26 to 10-12) add up to 880 minutes rounded up per job (786 raw); 644 fall in October and the count passes 600 at 13:45Z on 10-12. The last job ran at 20:37Z. None of the 76 jobs created after it got a runner: 35 were cancelled within 3 hours (replaced by the next run of their group), 41 after about 24 hours (the limit for a queued job), and 7 runs are still queued. So the item's "superseded by the next push" is the shorter half of the story: the queue itself stalled. That fits an exhausted monthly quota, and the header of codspeed.yml says it is not proof; the real figure is in CodSpeed's organization settings, which only a maintainer reads.What changed:
pull_requestgainslabeled, and the job runs only with theperformancelabel (the draft and fork conditions stay); main is measured by a daily cron (17 3 * * *) and no longer bypush; a scheduled run cancels an older one still waiting for a runner; the run that adding any other label starts has a concurrency group of its own (a labeled event of another label would otherwise cancel the benchmark run theperformancelabel started a minute before). At 4.8 minutes a job a day costs about 144 minutes a month, against about 440 for the 92 pushes to main of the 30 days to 10-26. CONTRIBUTING.md says theperformancelabel now also starts the run.Evidence.
codspeed.ymltargetscodspeed-macro-arm64-graviton-ubuntu-22-04, which is not a GitHub-hosted label (the file calls it a self-hosted runner outside GitHub's sandbox), so under the lane's rule I did not dispatch it, did not putperformanceon this PR and did not run it. What I could show on GitHub without it:skipped: 40414278753 (no label), 40414920128 (labelspikejust added, alabeledevent), 40414979246 (labelneeds-infoadded), and the pushes 40415458833, 40419553248 and later. This is a draft PR, so the draft clause also applies; the label clause is what the twin shows.l18-gate-proof.yml, same triggers, same group expression and sameif, withspikein place ofperformance, no draft clause, a 90 s sleep; deleted in the last commit): no label, run 40414278847 skipped. Labelspikeadded: run 40414920862 ran (success). Labelneeds-infoadded 27 s later while it ran: run 40414979324 skipped and 40414920862 was not cancelled (completed success). A push with the label present: 40415458668 ran; a push with the label present 12 s after another: the run of the first, 40416207215, was cancelled by the run of the second, 40416230527.performancelabel starts and runs; that the cron fires (a schedule runs only from the default branch). actionlint accepts both (dockerrhysd/actionlint:latest, clean on every head).workflow_dispatchfor backtests; it does not mentionschedule. If the first scheduled run is not recorded as the main baseline in the CodSpeed dashboard,push: branches: [main]goes back beside the cron (the header of the workflow says so).890.92: the full-size shape on a hosted runner
Job
large-response(a separate non-required job, not a step of an existing one: it needs a no-race build, memlock and its own timing, and a step inunitwould add its 80 s to a required check) builds./test/serverwithout-raceand coverage and runs the eight tests that readlean761(),n836()orraceOn761by name with memlock raised,-v, a tally by name and a check that the 64 MiB HTTP/1 bodies (12 runs) and the 16 MiB HTTP/2 ones (8 runs) ran and the lean ones did not. Hostedubuntu-latest, 4 CPUs, 16 GB:| run | wall | peak RSS of the test process | tally |
|---|---|---|---|---|
| B, job 121892520456 | 66.1 s | 570 MiB | 8 of 8 PASS, 0 SKIP |
| D, job 121899046921 | 78.9 s | 548 MiB | 8 of 8 PASS, 0 SKIP, shape 12/8/0/0 |
| E (bad input, below), job 121899124643: the same binary built with
-race| 76.3 s | 1128 MiB | 8 of 8 PASS, shape 0/0/12/8: job RED |The job's discovery (review r1, m7) fails when a Test function of
./test/serverthat readslean761(),raceOn761,n836()ordur836()is not in the list; it does not see a test that reads them only through another helper that is not itself a Test function (the awk looks at the enclosing top-level function). The size run was taken from thepull_requestruns of this PR (jobs above), because no dispatchable hosted workflow builds./test/serverwithout race or coverage (ci.ymlhas noworkflow_dispatch;test-coverage.ymlhas one but runs with-cover, the lean shape).TestH2CUnmatchedThroughTimeoutSingleflight836runs its three arms withAsyncHandlers: truehere (28.3 s in D against 9.7 s lean under-racein run A). The item's open question was whether the shape is affordable: it is, at about 80 s and 0.55 GiB. Nothing in the full-size shape failed, so no production defect came out of it.890.87: the base tier
Job
iouring-base-tier,CELERIS_MAX_IOURING_TIER=base, memlock raised, two steps:./internal/engine/iouringunder-race(473 s in run B, 471 s in D; step 504 s and 500 s) and the root package plus./test/serverfor the namesShutdown|Detach|SSE|Stream|WebSocket|Hijack|Drainunder-race(4.6 s and 239 s in B, 4.4 s and 236 s in D; step 258 s and 254 s).base-tier-tally.shfails on: a SKIP line not allowed by.github/base-tier-skips-{iouring,root}.txt(12 and 19 SKIP lines on the first run: the iouring list names each, with its reason; the root list names three top-level tests and has one pattern entry for theio_uring,adaptiveandadaptive-iouringarms of probe-gated tests, honoured only where the arm itself logged the probe'sio_uring tier=base kernel=...skip message, not the engine's startup line), fewer top-level PASS thanmin-pass(554 and 107 measured on the first run; 578 and 118 on the head after the merge with main; floors now 540 and 105), a named test that did not run once and PASS (TestShutdownDetachedPartialWriteIsFlushed806,TestReportTier844), and a cap that did not take effect (the capped probe lineio_uring_tier=base CELERIS_MAX_IOURING_TIER="base"must appear once and no engine arm may reporttier=high). Its self-test (22 cases after review round 1) is a Lint step.TestShutdownDetachedPartialWriteIsFlushed806passed on the capped probe, the thing the item says only local runs covered.base-tier-tally iouring: top-level PASS 554 (min 500), SKIP lines 12, not in the allow-list 0, capped probe lines 1, engine arms at tier=base 15 and tier=high 0; root:PASS 107, SKIP lines 19, not in the allow-list 0.TestH1Unclosed...889tests, so the root step saw 117 top-level PASS and 46 SKIP lines, 27 of them not named. A list of names would go red for every test of that kind that a later pull request adds, so the root list got the pattern entry above (the iouring list stays exact names: a new skip in that package needs a line, the repo's usual rule for allow-lists). A first version of the pattern took anytier=baseline as evidence; the engine's own startup line says that too (264 such lines in that log), so it now needs the probe's message, with a self-test case (22 cases in all after review r1: parallel-subtest interleaving, the exact probe message, literal names). The root log of that run passes the tally with the pattern (checked under mawk, the awk of ubuntu-latest).CELERIS_MAX_IOURING_TIER: bse(the probe counts an unknown value as "none"): the package step is red, 47 top-level tests FAIL, the probe line readsio_uring_tier=none CELERIS_MAX_IOURING_TIER="bse". The root step then ran 23.7 minutes (against about 255 s) and was cancelled whileTestShutdownDetachedPushDoesNotHoldTheDrain806/io_uring/Shutdown-backgroundran: with io_uring "none" the Shutdown tests fail slowly (TestShutdownSendDrainIsBoundedtook 96.8 s to fail), they do not skip. Step timeouts are now 900 s and 600 s and the job limit 30 minutes. go test failed before the tally ran, so the tally's own cap check is shown by the self-test fixtures (an arm attier=high, a probe line of an uncapped run), not by a CI run.TestAcceptedFDIsNonBlockingAndCloseOnExec884) is tier-independent and already in the Unit job. What the job adds is the engine tests running at the base tier.c714ProbeIOUring,keptProbeIOUring) calls io_uring "usable" only at the high tier with provided buffers, so at the base tier they skip with "no usable io_uring". The job therefore covers the base tier for the tests that start an io_uring engine without that probe, not for all root io_uring arms.890.93: zizmor
.github/tools/requirements.txtholdszizmor==1.30.0; the Lint step makes a venv and runspip install -r(pipx runhas--specand no requirements-file option, checked with pipx 1.18.4);dependabot.ymlhas apipentry for/.github/tools; check 4 oftool-pins-check.shfails when a workflow or composite action installs or runs a Python tool (pipx run|install,pip(3) install,python -m pip install,uvx,uv tool run|install) with its ownname==version, and when arequirements*.txtunder.github/has no pip entry for its directory (directory:ordirectories:). 20 new fixtures intool-pins-check-test.sh(38 in all; review r1 m4 addeduvx name@version,uv tool run name@versionand the>=,~=,<pins, with two that must not be flagged)..github/workflows/ci.yml:144:run: pipx run --spec zizmor==1.30.0 zizmor .... Locally the same commit's self-test showed the check itself right (every1-expecting fixture ok) and every "good"-expecting fixture red, because they copy the real tree.-d /.github/tools:created | zizmor ( from 1.30.0 to 1.31.1 )(log: scratchpadl18citests/logs/dependabot-pip.log). The CLI builds its job from its arguments and does not readdependabot.yml; what it shows is that Dependabot's pip updater reads.github/tools/requirements.txtin that directory and proposes a bump. That the YAML entry names that directory is what check 4 enforces.--local, Pinned-Dependencies, Token-Permissions, Dangerous-Workflow): 10, 10, 10 on main and on this branch; thepip install -rin a venv adds no finding.890.91: allocation budgets
Gate without the fix (run A, job 121891700334):
alloc_guard_test.go:60: allocs/op=14 (budget 4),allocs/op 14 exceeds budget 4 (strict mode),alloc guards: top-level PASS 14 (want 15), packages with a wrong tally 1, SKIP lines 0, FAIL lines 2; that is the only failing step of the job and the only failing guard of the 15. WithQuery_1col_1rowat 14 (runs C and D):top-level PASS 15 (want 15) ... FAIL lines 0, and the same in the head run. The other guards sit at 3/4, 4/4, 26/26, 0/0 and so on, exactly at budget in several; none moved in the runs above. The budget is raised to what the driver costs (item's first option; bringing the allocations down isdriver/postgres's).890.96 and 890.95: test/integration
./test/integrationruns only throughmage, not in CI, so its changes are shown by local runs (docker golang:1.27.3,--cpus 2) and the one guard that can run in CI is run by name in theadaptivejob.TestAdaptiveSkipRoutingIntegration890(the counterpart ofTestAdaptiveSkipRouting684, AST-based, with a 21-case self-test and a planted regression in the real gate file): with an empty registry it lists 11 raw skips in the threeadaptive_*_test.gofiles, 10 inadaptive_hybrid_test.goand one inadaptive_standby_886_linux_test.go(run A, Adaptive job 121891700701, step red). The one in 886 is an io_uring-availability skip and now goes throughskipOrFailUpswitch890, this package's single definition of the gate (skipOrFailIfStandbyAborted886calls it too). The ten inadaptive_hybrid_test.goare registered with their count and why, because item 95 leaves the adaptive skips to their lane; review r1 (m3) addedhelpers_test.goto the scan, becausestartEngineskips on a Listen failure and the adaptive tests and the epoll arms of the slowloris and pipeline tests share it: its three raw skips are registered too (13 in all, debt, same reason); they are the debt, and a further site with a registered message fails the guard (checked by planting one). With the registry (run C, Adaptive job 121896544783):0 raw skips not admitted, 5 registered(6 entries with the helpers after review r1), step green.epoll.Newonly fails on config validation and the skip never fires. Control by hand, same mutation on main and on the branch (cfg.MaxHeaderBytes = 1, rejected byValidate):TestHTTP1PipeliningAsyncPASS with two SKIPs on main, FAIL on the branch;TestSlowlorisSyntheticepoll arm SKIP on main, FAIL on the branch, iouring arm SKIP on both. Unmutated on the branch:TestHTTP1PipeliningAsync,TestSlowlorisSynthetic,TestSlowlorisSyntheticConcurrentPASS (the iouring arms skip in docker's seccomp). A deadminSkipfield of the slowloris table (never set) is removed.890.88: probe retry logs
CELERIS_REQUIRE_IOURING_WORKERS=1 CELERIS_MAX_IOURING_TIER=none, one test per copy in docker, branch against main: main prints 0 lines of retry (logger copy); the six copies on the branch print 942 to 957keptProbeIOUring: retry round N, no usable io_uring ring yet (tier none, provided buffers false)lines in the 10 s of the retry (logger 942, recovery 953, adapters 941, metrics 943, otel 944, test/server 957). The tests then fail as designed ("forbids dropping the io_uring arms"). Logs: scratchpadl18citests/logs/probe-*.log. No CI run can show this: the probe does not retry on a runner that has io_uring.Files beyond
ci.yml,codspeed.ymland testsThe items need these:
.github/dependabot.yml(890.93 pip entry),.github/tools/requirements.txt(890.93),.github/scripts/tool-pins-check{,-test}.sh(890.93 check 4),.github/scripts/base-tier-tally{,-test}.shand.github/base-tier-skips-{iouring,root}.txt(890.87: the tally, its self-test and its allow-lists as data),driver/postgres/alloc_guard_test.go(890.91, a test file),CONTRIBUTING.md(one paragraph: theperformancelabel now also starts the CodSpeed run; no item asks for it, but the coupling is invisible without it; #1092 does not touch the file). Overlap with open PRs checked on exact file lists: #1119, #1123, #1092 share no file with this PR; #1122 sharesci.yml(its hunks are the root-step allow-list, lines 222-240) and #1126 sharescodspeed.yml(twouses:pins);git merge-treeof this head with each of #1122, #1126, #1123 and #1119 reports no conflict. Main moved by one commit while this was open (#1118, no shared file);keptProbeIOUring()has no zero-argument call site left on main or in the open PRs' diffs.Review round 1 (head 3438221), what changed
M1 and M2 as above and below; m1
=== NAME/=== CONThandled, with a crafted interleaved log that fails the old tally (4 of the 22 rows fail on the old script, 5 of the 38 on the old check 4); m2 the evidence is now theno usable io_uringmessage (the head's real root log, 46 SKIP lines, still passes under mawk); m3helpers_test.goscanned; m4 range and uv pins; m5 below; m6state == 'open'in theif:of the CodSpeed job; m7 a discovery in thelarge-responsestep (it lists the Test functions of./test/serverthat readlean761(),raceOn761,n836()ordur836(); checked locally to give exactly the eight, and to report a planted ninth; a helper that is not a Test function and reads them is not seen); nits n0c (counts in this body), n1 (names compared as strings, with a row), n2 (a method of the gate's name is not the gate). Not taken: n0 (the min-pass floors stay weak: they catch a-runthat stops matching, not the quiet loss of 50 tests; said here), n3 (the postgres budget has no headroom: intended for a guard).Release gate (m5). The two new jobs do not block merges, but they do block releases:
release.ymlcallsci.yml(ci:job) andtag-submodulesruns only ifneeds.ci.result == 'success', so a rediouring-base-tierorlarge-responseblocks the tag. The same holds for the existing non-requiredadaptive,iouring,zc-windowandrecv-theftjobs. A merge with--admincan therefore land a red one unnoticed and show it at release time.release.ymlis not changed.Strict allocation budgets (n0b). Several guards sit exactly at budget (4/4, 26/26, 100/100, 1/1 and now postgres 14/14) and the allocation step is part of the required
Unitcheck throughunit-gate: a Go or dependency bump that adds one allocation to one of them turns Unit red. That is item 91's intent; the budget is then raised or the allocation removed in the PR that causes it.Main moved: merge commit
c804f3d(origin/main 8936e89, with #1118, #1122, #1123, #1125, #1126 and #1127). Head 3438221 had been tested on a merge ref that predates #1122. The merge was clean, with no conflict marker: #1126 changed one line ofcodspeed.yml(and one ofscorecard.yml), outside every hunk of this PR, and #1127's step change inci.yml(the "eight top-level PASS" comment of the multi-worker step, and the #885 allow-list names) is outside the hunks of this PR too. The codspeed.yml of the merge keeps both: the diff of the merge commit against its first parent (this branch) in that file is onlyand the gate is as before:
schedule: - cron: "17 3 * * *",pull_requesttypes withlabeled, the concurrency group with thelabeled-by-another-label clause, and the jobif:(fork,state == 'open', not draft, labelperformance,labeledevent only forperformance).scorecard.ymlis #1126's (upload-sarif9b32540, v4.39.1).Base tier on the merged tree (the job's own two commands, docker golang:1.27.3,
--cpus 2,CELERIS_MAX_IOURING_TIER=base, seccomp unconfined, memlock unlimited,-race, thenbase-tier-tally.shunder mawk):./internal/engine/iouringTestListenOnAZonedLinkLocalServes885(#1127), because the container has no fe80:: address; capped probe line 1,tier=basearms 15,tier=high0;TestShutdownDetachedPartialWriteIsFlushed806andTestReportTier844PASS./test/serverThe tests that #1123 and #1127 add run at the base tier without a skip, except that one. The zoned tests already fail instead of skipping under
CELERIS_REQUIRE_ZONED_PEER=1(the unit job sets it, a hosted runner's eth0 has an fe80:: address), so the two base-tier steps now set it too (bc707c6) and the allow-list does not name the test. Themin-passfloors move to 540 and 105 (577 and 118 measured). Discovery of the full-size job still lists exactly the eight tests.Post-merge verification (890.7, 890.97)
The main baseline of CodSpeed now comes from
scheduleruns. CodSpeed documentspushto the default branch as the baseline and does not mentionschedule; the pinned runner versions (v5.5.0 now, v5.6.0 in #1126) accept a scheduled run (RunEvent::Schedule), but whether the server records it as the main baseline is undocumented. The macro-runner quota is exhausted until 2026-10-31, so the first scheduled run cannot get a runner before then. Steps, once the quota is back:gh run list -R goceleris/celeris -w codspeed.yml -e schedule -L 3shows a run, and its jobBenchmarks (arm64 macro runner)got a runner and succeeded.mainand is the baseline: open a pull request that touches a benchmark path, addperformance, and check that CodSpeed compares it with that run (not with "no baseline" or an older report).push: branches: [main](with thepaths:list) back beside the schedule, or keep the cron and have it dispatch the workflow (gh workflow run codspeed.yml --ref main, aworkflow_dispatchrun, which CodSpeed documents). Both fallbacks are named in the header ofcodspeed.yml. The dispatch fallback has two costs: each dispatched run gets a concurrency group of its own, so a stalled run is no longer replaced by the next one, and CodSpeed documentsworkflow_dispatchfor backtests, not as a baseline source, so it is no better documented than the schedule; restoringpushis the only documented baseline trigger.Ready to paste into the #890 tracking list (not edited here):
Release-note lines
None.
Findings not fixed here
ubuntu-latestjobs here are advisory.iouring-base-tieradds about 13 minutes of job time to every pull request run; if that is too much it can move to a label or a schedule.CELERIS_MAX_IOURING_TIER=none(or a value the probe does not know) the Shutdown tests of the root package and./test/serverdo not skip, they fail slowly (97 s for one): a misconfigured cap costs minutes of CI, not a skip. Not changed here.adaptive_abort_844_linux_test.gocarries threepkg844 == "iouring"branches that can never run. Removing them is a cleanup in a file this lane does not need to touch.test/integrationis not in CI at all, and its ten adaptive_hybrid skips are the same trap as item 95's epoll arms (adaptive.Newfails on config validation and port holds only).For the reviewer to attack
performance; a PR labelled for the notes on a benchmark path starts a macro run (intended, documented in CONTRIBUTING). Dropping the push trigger means a PR compares with a main run up to a day old, and aworkflow_dispatchis the manual refresh. The positive arm (real label starts the real job) and the cron are unproven before merge; the twin proves the expression and the group, not the runner label. Is it right to also cancel an older scheduled run, and is17 3 * * *a good time?!cancelled()even when step 1 failed, on purpose (both reports in one run), bounded by its timeout.scheduleas a baseline trigger (see "Post-merge verification").adaptive.New's ast.Fatalfand the Listen ones through the gate; that is the adaptive lane's file.python3 -m venvonubuntu-latestinstead ofpipx; check 4 is a regex overpipx|pip|uvxlines and will miss other installers (pip install -c, arequirementsURL,conda).