Skip to content

engines: OnDetach rollback first, io_uring pause wake, zoned bind address, one-dump NIC table, MaxEvents by allowed nodes, upgrade-carry count, backlog hook (v1.6.0, Refs #885 #886 #889 #890) - #1127

Merged
FumingPower3925 merged 9 commits into
mainfrom
l7-xeng
Oct 27, 2026

Conversation

@FumingPower3925

@FumingPower3925 FumingPower3925 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Lane L7 XENG of v1.6.0, cross-engine small fixes. Refs #885, Refs #886, Refs #889, Refs #890.

Review r1 (Opus) approved with fixes; commits 6 and 7 are its fixes (tests first), and main (a5fb8f1 and later, #1123 included) is merged in with a merge commit, no conflict.

Method: each item's premise was checked on main (baa43d9) first; the branch is rebased onto 3f420c6 (#1122 and #1118 merged). Commit 1 is the tests (and the no-op seams they need), red on main; commit 2 is the fixes; commit 3 is comments and godoc, no behaviour; commits 4 and 5 are the tests and the fix of 886.23 alone, so that pair can be dropped or rebased on its own (it was the one item that conflicted with #1122, see below). Item numbers are the ordinal of the checkbox in the issue body, counting every box, ticked or not (not the numbers of #1099, which counts only the unticked ones); each row quotes the item's opening words.

item opening words premise on main result commit
889.48 "OnDetach hook rollback (epoll loop.go ~2324, io_uring worker.go ~3066 ..." TRUE: the rollback defer sits after the detachMu allocation and the count; the io_uring rollback test did not assert the detachedConns gauge fixed, both engines 1, 2
886.23 "The request count of an h2c upgrade differs between the recv path and the transplant replay" TRUE (the replay counts 1, the recv path 0, pinned by Test{Epoll,Iouring}AdoptRequestCountUpgradeCarry886) fixed: the replay counts none, as the recv path does; 4 sites; pins changed 4, 5
890.22 "Replace sleep-as-synchronisation in #805's tests" PARTIAL: the 2 ms / 100 ms sleeps of TestSplitBodyResponsesKeepTheConnection and TestConcurrentResponsesOwnTheirBodies were already replaced by #1055; TestBackloggedPeerIsClosed still slept 100 ms (backlogGrace761) fixed: waits for the engine's refusal. "can fail falsely" NOT reproduced (below) 1, 2
885.54 "epoll and io_uring parseAddr refuses a zoned IPv6 bind address" TRUE (invalid addr: [fe80::1234%dz0]:0; epoll, io_uring and adaptive fail to start through celeris.New(...).Start()) fixed, both engines (adaptive follows) 1, 2
886.24 "BeginPauseAccept ... only stores acceptPaused and does not signal the wake eventfd" io_uring TRUE (lag up to 99 ms); epoll FALSE (lag 0.8 to 5.9 ms: a listening loop waits at most 4 ms) io_uring fixed; epoll deliberately NOT changed (declined with evidence: a listening epoll loop waits at most 4 ms, measured lag 0.8 to 5.9 ms), so the tick of 886.24 is a change on io_uring only 1, 2
886.11 "The listener wait of #683 gives up at 250 ms" its remaining sub-point (a flap inside the pause latency) is 886.24's settled by 886.24; the bound stays (decision U-14, A). NOT ticked here: the box is ticked on merge none
885.53 "epoll and io_uring (shared internal/platform): the NIC interrupt detection of PR #1062 builds its interface table with net.Interfaces() then Addrs()" TRUE (311 interfaces: 313 netlink requests, 21.6 ms) fixed: one dump, 3 requests, 0.94 ms 1, 2
885.31 "The MaxEvents split divides by the host's NUMA node count" TRUE as code; the effect reaches len(events) only above 4 nodes (finding below) fixed, epoll 1, 2
885.14 "h2PoolSettled (epoll and io_uring) ignores writeRefused after GoAway" TRUE as stated, and not left open: traced below comment only (as the lane plan says); no code, no test 2
886.9 "OnError delivery on a conn with no detachMu differs by engine" TRUE, and the state is reachable (below) comments + two io_uring pin tests; the rule is NOT picked 3
U-12 the rx-port ("rule D") note - godoc in rxFreeCPUs only, no README 3

889.48 (OnDetach rollback)

OnDetach (epoll Loop.initProtocol, io_uring Worker.initProtocol) registered its rollback inside if !unlockDetachMu, which was computed after detachMu was allocated and the count made. A panic between the snapshot and that point left the count (and detachedConns) up, detachMu set and asyncDetachPending set. Now the gate is computed from the snapshot (prevMu != nil in place of cs.detachMu != nil: in sync mode l.async is false so the gate is false either way; in async mode detachMu was allocated at acquireConnState, so prevMu is not nil; the one state where they differ, async + promoted + no mutex, could not unlock a mutex it does not hold anyway) and the rollback is registered right after the snapshot. The sync path's reliance on the rollback leaving detachCounted false is commented.

Test: a new seam detachPostCountHook (nil in production, called once per Detach between the count and the install; same shape as detachPostInstallHook) lets TestOnDetachPanicLeavesTheConnAsIfNeverRan889 panic right after the count: red on main (detachedCount=1 counted=true, cs.detachMu is still set, asyncDetachPending is still set, and on io_uring the detachedConns gauge is 1), green now. The io_uring test also asserts the gauge in the early and late cases (the port of TestFinalGaugeRolledBack); that part passes on main, it is a stronger test, not failing-first.

886.23 (request count of an h2c upgrade carry): commits 4 and 5

Rule picked: the replay counts none for an upgrade carry (!errors.Is(perr, conn.ErrUpgradeH2C) at the epoll sync/async and io_uring sync/async replay sites). The other rule needs drainRead/handleRecv (not mine to edit here). h2 conns count no request on either engine, so the upgrade request is not counted anywhere afterwards; BytesPerReq moves by one request per upgraded replayed conn, as the issue says. The four pinned tests now expect 0 and read the 101 off the client first (expectCount886 returns at once on 0 and only holds 400 ms; without the 101 it could pass before the replay ran).

Conflict with #1122, resolved: attachAdoptedFD and replayCarriedAsync are functions #1122 rewrote (they were not on the list of functions to keep away from, but the same two reqBatch++ lines are involved), which is why this item is a commit pair of its own. #1122 and #1118 merged while this was open (main is now 3f420c6); this branch is rebased onto it, and the epoll condition is if !recvCounted && !errors.Is(perr, conn.ErrUpgradeH2C) { at both sites. ./internal/engine/epoll/... passes on the rebased tree, plain and -race (TestReclaimTransplantDoesNotCountTheCarriedRecvTwice886 included). One corner stays: a reclaim of an upgrade carry keeps the count drainRead made when it read the bytes (it counts before it transplants), so that one case counts 1; it needs drainRead. git merge-tree of this branch with #1123's head (fetched after the rebase): clean.

890.22 (sleep in TestBackloggedPeerIsClosed)

conn.BacklogRefusedHook (nil in production) is called in handleH1Request in the branch where WriteBacklogged() answers true and the request is refused (ErrWriteBacklog), with the peer address; the test watches its own client address and waits for it (std has no cap: no wait). The call is out of line (//go:noinline), in the taken branch only: the code of a request that is served is unchanged. The test went from 2.3 s to 0.34 s for the nine native arms.

Hot path (the branch sits beside the per-request WriteBacklogged query): BenchmarkProcessH1Backlogged890 (a GET with the query installed, answering false), docker --cpus 2, 12 rounds interleaved main / this PR / A/A twin of main, 1 s each, medians:

main this PR A/A twin
ns/op 49.86 49.66 49.91
allocs/op 0 0 0

this PR / main 0.996, twin / main 1.001 (the noise floor).

"Can fail falsely" was NOT reproduced: backlogGrace761 set to 0 on main, TestBackloggedPeerIsClosed 10 of 10 pass, and 10 of 10 pass under 8 busy-loop processes on the same 2 CPUs. So this is a removal of a timing guess for the 100 ms that cost, not a fix of an observed flake. Red state: with the seam defined but not called (commit 1) the test fails in nine arms (no request was refused within 10s).

885.54 (zoned bind address)

parseAddr (both engines): a host with a % is read with netip.ParseAddr, which keeps the zone, and the zone becomes SockaddrInet6.ZoneId through bindiag.ZoneID, std's rule (net's ipv6ZoneCache.index, read in go1.27.3): the interface of that name, else the decimal digits the zone starts with (12abc is 12; a number of 0xFFFFFF or more saturates at 0xFFFFFF, net's dtoi), else 0 (the bind then fails for a link-local address, as std's does). The first version returned 0 for 12abc and for a number past 32 bits while its godoc claimed std's rule (review nit); I matched std (the smaller change: eight lines, against narrowing the godoc and leaving a difference), with rows in TestZoneIDIsStdsRule and in both engines' parseAddr table. An IPv4-mapped host is IPv4 whatever its zone, and 1.2.3.4%lo, [fe80::1%], [%lo] are errors, all as net.ResolveTCPAddr does (checked in a container; [fe80::1%lo%lo] is a zone lo%lo for std too and here). The old table row that pinned the error ([fe80::1%eth0]:80) is replaced by TestParseAddrReadsAZonedIPv6Host885, which compares each row with net.ResolveTCPAddr. TestListenOnAZonedLinkLocalServes885 listens on [fe80::1234%dz0]:0 (a dummy link; --cap-add NET_ADMIN, ipv6 on), serves a request and checks Addr() prints the zone as net.Listen does; it skips without a link-local address and fails under CELERIS_REQUIRE_ZONED_PEER=1. bindiag.WithPort needed no change. TestZonedListenAddressServesOnEveryEngine885 (test/server) goes through the exported API: celeris.New(Config{Engine, Addr: "[fe80::1234%dz0]:port"}).Start() for std, epoll, epoll-async, io_uring, io_uring-async and adaptive (its port hold included), a request to the zoned address, and, for the native engines, Addr() printing the zone (std's own Addr() printed no zone on a hosted runner in the first CI run, so it is not asserted; std is the oracle here); on main five of the six arms fail with invalid addr (adaptive: sub-engine startup failed ... invalid addr), now all pass. The CI root step already runs ./test/server with CELERIS_REQUIRE_ZONED_PEER=1.

886.24 (BeginPauseAccept wakes the workers): io_uring changed, epoll declined

The item names both engines; only io_uring changed. Epoll was declined on measurement: its listening loop waits at most 4 ms (adaptiveTimeoutMs, base 1 ms times 4), and the measured pause-to-closed lag is 0.8 to 5.9 ms (table below). Ticking 886.24 must not read as a code change on both engines: add "epoll: not changed, measured 0.8-5.9 ms, the wait is capped at 4 ms" to the item when ticking.

Measured first, before the change: BeginPauseAccept (DisableDeferAccept, so the listener closes at once) to no listener in the kernel's table, 2 loops / workers, 3 runs of 8 cycles = 24 samples per arm, a 1.2 s idle settle then 0.3 s before each cycle (ms):

before: median / p90 / max after: median / p90 / max
io_uring, default ReadHeaderTimeout, 16 live conns 4.87 / 6.27 / 21.58 0.30 / 1.52 / 1.64
io_uring, ReadHeaderTimeout < 0, 16 live conns 4.21 / 5.65 / 96.87 0.20 / 1.50 / 4.12
io_uring, default, no conns 4.82 / 6.21 / 27.00 1.19 / 1.40 / 1.57
io_uring, ReadHeaderTimeout < 0, no conns 4.75 / 6.45 / 99.35 1.27 / 1.49 / 1.59
epoll (unchanged), the same four arms 1.62 / 3.05 / 5.58; 4.36 / 4.93 / 5.89; 1.31 / 2.47 / 2.77; 3.05 / 4.45 / 5.58 -

So on io_uring the tail the issue describes is real (up to 99 ms) and rare (the sweep tick keeps the median near 5 ms); epoll's is not: adaptiveTimeoutMs caps a listening loop's wait at 4 ms, and run()'s own comment says so ("while the listener is open the wait is already at most a few ms"). The signal costs epoll a write per loop per pause for nothing, so epoll's BeginPauseAccept is not changed (the issue asked for both; premise false for epoll). Its lag test is kept as a guard (bound 100 ms; it passes on main).

Fix (io_uring Engine.BeginPauseAccept): after the flag store, wakeFD.Signal() on every worker under e.mu (as ResumeAccept, after the store so a woken worker reads the flag set). Deterministic test, red on main: TestBeginPauseAcceptSignalsEveryWorkerWakeFD886 (three workers with real eventfds; the counter must be non-zero). The end-to-end lag test cannot fail reliably before the change (the tail is rare), so it is a guard with a loose bound (100 ms, every arm). Its 2-worker arm needs raised memlock, so TestBeginPauseAcceptReachesAnIdleWorker886 is added by name to the CI step that runs the multi-worker listener tests with CELERIS_REQUIRE_IOURING_WORKERS=1 (ci.yml: the name list, the comment and the count, now eight); lags there under -race in my container: median 0.5 to 1.6 ms, max 3.0 ms (24 samples per arm).

886.11: its flap-inside-pause-latency sub-point is this item; the 250 ms bound stays (U-14: keep as a safety net).

885.31 (MaxEvents split)

splitMaxEvents(maxEvents, topo, plan.Allowed) divides by the nodes whose CPU list meets the plan's allowed set; where that cannot be told (no allowed set, no per-node CPU lists, none of the allowed CPUs on a listed node) it uses the host's count, as before. Seams: detectNUMA. Tests (red on main): the table TestSplitMaxEventsFollowsTheNodesTheAllowedSetMeets885 and, through Listen, TestListenSplitsMaxEventsByTheNodesOfThePlan885 (8-node host stood in: a process kept to one node gets 2048 events per loop, one spanning the nodes 1024).

Finding: a loop's event array is min(MaxEvents, 2048) (maxEpollEvents) and the default MaxEvents is 8192, so the split shows in len(events) only on a host with more than 4 nodes (8192/5 < 2048), or with a configured MaxEvents under 2048 x nodes. On the usual 1 to 4 node hosts the fix changes nothing observable. The issue's "larger MaxEvents on a restricted host" is true for the resolved value, not for the array.

885.53 (interface table: one dump)

osIfaceAddrs is net.Interfaces() (one RTM_GETLINK) plus one RTM_GETADDR dump mapped by interface index (parseAddrRIB: IFA_LOCAL if present, else IFA_ADDRESS, as net's newAddr; mapped IPv6 unmapped; the 512 cap and the order are kept); if the dump cannot be read or parsed it builds the table the old way. That includes a panic of the std attribute walker (syscall.ParseNetlinkRouteAttr, which net calls on the same bytes) on a final attribute that is not padded to 4, which a kernel dump never has: addrsByIndex recovers and reports not-ok (review minor 1; the fuzz crasher and six hand-built unpadded dumps are table rows, and TestOSIfaceAddrsFallsBackOnAnUnparseableDump885 checks the fallback table; the test binary panics before the fix). An interrupted dump (NLM_F_DUMP_INTR) is read as net reads it, and the code comment says so. Seam: netlinkRIB. Equivalence with the old per-interface table on the host's own interfaces: TestOSIfaceAddrsIsOneAddressDump885 (red on main: the seam sees 0 dumps; it counts 1 now). TestParseAddrRIB885 builds dumps byte by byte (point-to-point in both attribute orders, IPv6, mapped, interleaved interfaces, another family, short or oversized messages, messages after NLMSG_DONE); TestHostDumpParses885 rejects the silent fallback.

Measured, docker --cpus 2, 311 interfaces (300 dummy links with an address each), 10 runs each, BenchmarkOSIfaceAddrs885 against BenchmarkRefIfaceAddrs885 (the old code verbatim, in the same binary):

old this PR
ns/op, median (min-max) 21,592,200 (21,234,900 - 24,640,800) 941,200 (855,100 - 1,013,800)
B/op 94,400,000 4,260,000
allocs/op 43,998 5,251
netlink sendto calls (strace, one call) 313 3

(The 3: the RTM_GETLINK of net.Interfaces, the one RTM_GETADDR, and the benchmark's own net.Interfaces for the interface count.) This is the startup cost of the detection, paid again at the adaptive engine's standby Listen. The first-cluster-run figure the issue suggested (irq_took) is not used.

885.14 and 886.9: comments

885.14: read from the code, not measured (the lane plan makes it comment-only; the issue also asked for an explicit close and a test, which this does not do). A refused GOAWAY (the conn's backlog is over the 64 MiB h2 cap, so its peer stopped reading) sets writeRefused; h2PoolSettled does not look at it. Traced: the next drainH2Writes with a frame queued closes the conn (epoll discards the queue and closeWhenFlushed; io_uring closeConn), a read of the conn finds writeRefused (epoll drainRead's tail; io_uring handleRecv) and ends it, and a conn with none of those is not busy in h2PoolSettled, so the pass settles and shutdown closes it. What is lost is the client's graceful notice from a peer that was not reading; the cost of a conn that stays busy (OutboundPending with a WINDOW_UPDATE that never comes) is the wait up to the drain's bound. Both h2PoolSettled docs say so.

886.9: the item asks "if it cannot [hold a callback without a detachMu], state that in both engines, otherwise pick one rule". It can, so no rule is stated as unreachable. A sync conn holds an OnError without a detachMu when a handler registers one (Context.SetWSErrorHandler, exported) and does not Detach, and after a Detach that panicked and was rolled back (the rollback puts detachMu back to nil, the callback stays; websocket and sse register before they Detach, so this is how they get there). epoll's endH1Verdict then delivers it, io_uring's does not (if mu != nil). TestEpollRecvPathOnErrorSync908 and TestEpollAdoptCarriedOnErrorSync908 pin epoll's side; the two new tests TestIouringRecvPathOnErrorSyncNoDetachMuIsDropped886 and TestIouringAdoptCarriedOnErrorSyncNoDetachMuIsDropped886 pin io_uring's (run 3 of 3). Both endH1Verdict docs now say it. Picking the rule is still open: ticking this box on the strength of comments alone is the maintainer's call.

U-12 (rx-port rule note)

rxFreeCPUs's godoc (internal/platform/cpuplan_rxfree_linux.go) now says when the rule does not engage (since-boot counters read once at the first Listen; msr1's two bond ports split each cell about 50/50: enp49s0 median 50.0%, range 39.2% to 61.8% over 64 cells, so no window reliably shows one port over the 60% of step 1) and that the rule is left as it is. No README change, no io512 claim.

Tests run

Local, docker golang:1.27.3, --cpus 2, one container at a time.

package plain -race
./internal/engine/epoll/... ok, 209 s (before the rebase) ok, 259 s (after the rebase onto #1122)
./internal/engine/iouring/... (memlock unlimited, seccomp unconfined) ok, 445 s ok, 476 s (both before the rebase, which changes no io_uring code; the new and touched tests pass after it)
./internal/platform/... (incl. pintest) ok, 13.7 s ok, 15.3 s
./internal/conn/... ok, 5.4 s ok, 13.5 s
./internal/engine/internal/bindiag ok ok
./test/server (TestBackloggedPeerIsClosed, TestStreamedResponseIsDelivered, TestPipelinedResponsesOwnTheirBodies, TestConcurrentResponsesOwnTheirBodies, the 761 group) ok, 4.7 s -
the new and touched tests of both engines, -count=5, in a container with a dummy link dz0 (CELERIS_REQUIRE_ZONED_PEER=1) ok, ok -
TestZonedListenAddressServesOnEveryEngine885, same container ok, 6 arms -
TestBeginPauseAcceptReachesAnIdleWorker886 + the eventfd test, -count=3 - ok

./internal/adaptive/... FAILS in this container, and fails the same on main: TestReverseTransplant, TestReverseTransplantAsync, TestBidirectionalFlap, TestBidirectionalFlapAsync (the controller switches by itself under the test's own load: "engine switch recommended ... sustained high load", then reverse transplant did not converge). On baa43d9 and on this branch, -run 'TestReverseTransplant|TestBidirectionalFlap' -count=3: 12 of 12 fail on each. Every other adaptive test passes (12,315 test events). Not caused by this PR, not investigated further here.

Red on main: the tests of commits 1 and 4 run on those commits (main plus the no-op seams): TestOnDetachPanicLeavesTheConnAsIfNeverRan889/count/{sync,async-unpromoted} (both engines), Test{Epoll,Iouring}AdoptRequestCountUpgradeCarry886 and ...Async886 (RequestCount after the adoption = 1, want 0; commit 4), TestParseAddrReadsAZonedIPv6Host885 and TestListenOnAZonedLinkLocalServes885 (both engines; invalid addr: [fe80::1234%dz0]:0), TestZonedListenAddressServesOnEveryEngine885 (five of six arms), TestBeginPauseAcceptSignalsEveryWorkerWakeFD886, TestSplitMaxEventsFollowsTheNodesTheAllowedSetMeets885, TestListenSplitsMaxEventsByTheNodesOfThePlan885/process-kept-to-one-node, TestOSIfaceAddrsIsOneAddressDump885, and TestBackloggedPeerIsClosed (nine native arms, no request was refused within 10s). Logs: scratchpad l7xeng/logs/red-*.log.

golangci-lint run ./internal/... ./test/... (arm64 and amd64): 3 issues, all errcheck in internal/engine/std/*_test.go, files this PR does not touch. mage apiCheck: matches (no exported API changed: the new identifiers conn.BacklogRefusedHook and bindiag.ZoneID are internal).

Mutants (each fails the named test, then restored)

mutation red
io_uring rollback without detachedConns.Add(-1) TestOnDetachPanicLeavesTheConnAsIfNeverRan889 (count, early and late, sync)
epoll rollback without cs.detachCounted = false TestOnDetachPanicLeavesTheConnAsIfNeverRan889 (count, early, late)
rollback registered after the count (commit 1's code) .../count/sync, .../count/async-unpromoted, both engines
replay counts an upgrade carry, each of the four sites Test{Epoll,Iouring}AdoptRequestCountUpgradeCarry886, ...Async886
BeginPauseAccept without the signal TestBeginPauseAcceptSignalsEveryWorkerWakeFD886
hook never called TestBackloggedPeerIsClosed (nine native arms)
ZoneId always 0, each engine TestParseAddrReadsAZonedIPv6Host885, TestListenOnAZonedLinkLocalServes885
MaxEvents split by the host's count TestSplitMaxEventsFollowsTheNodesTheAllowedSetMeets885, TestListenSplitsMaxEventsByTheNodesOfThePlan885
RIB parser ignores IFA_LOCAL TestParseAddrRIB885/p2p-address-first, /p2p-local-first
RIB parser always fails (silent fallback to per-interface calls) TestParseAddrRIB885 (all rows), TestHostDumpParses885

Release notes

  • io_uring: a pause of the listener (BeginPauseAccept, the adaptive engine's switch) now reaches an idle worker at once; before, a worker whose ring wait had not ended saw it up to 100 ms later (measured max 99 ms, median 4.8 ms; now max 4 ms).
  • epoll and io_uring: a zoned IPv6 listen address ([fe80::1%eth0]:8080) now binds, as it does on the std engine; before, both engines refused to start on it.
  • epoll and io_uring: the transplant replay of an h2c upgrade request counts no request in RequestCount, as the receive path does (it counted one).
  • epoll: MaxEvents is divided by the NUMA nodes the process may use, not by the host's node count (visible in the event array only above 4 nodes).
  • epoll and io_uring: the NIC interrupt detection at start reads the host's addresses in one netlink dump instead of one per interface (311 interfaces: 21.6 ms to 0.94 ms).

Found, not fixed here

For the reviewer to attack

  • 889.48: that prevMu != nil is the same gate as the old cs.detachMu != nil on every reachable state, and that registering the rollback before the mutex allocation changes nothing for the promoted path (it is registered only for !unlockDetachMu).
  • 886.23: that "count none" is the right rule (the alternative, counting on the recv path, needs drainRead/handleRecv), and that no consumer of RequestCount for an h2 conn expects the upgrade request counted.
  • 890.22: that the hook fires for every cap refusal and only there (handleH1Request's taken branch), keyed on H1State.RemoteAddr (the test relies on it equalling the client's LocalAddr().String(); a mismatch, a v4-mapped peer on a dual-stack listener, fails loudly after 10 s and cannot pass falsely, as the test's comment says), and that //go:noinline keeps the served path unchanged (the benchmark says so).
  • 885.54: scope-id 0 for a zone that names nothing is std's rule, not an error here; a link-local bind then fails at bind(2) with EINVAL instead of at parse.
  • 886.24: that skipping epoll is right: the 4 ms bound is adaptiveTimeoutMs with base 1 ms; a loop with a driver conn or a dirty list waits less.
  • 885.53: parity with net's newAddr on a message whose attributes are in an unusual order (the typed pick differs from net's "first attribute" only there); the fallback path.
  • 885.31: nodesMet counts the nodes that hold an allowed CPU, so on a host with CPU-less memory nodes (CXL) even an unrestricted process now divides by fewer nodes than NumNodes and gets a larger MaxEvents than before (only visible above 4 nodes, see the finding).
  • 885.14 and 886.9: the traces are code reading; run nothing to confirm the 885.14 one.

@FumingPower3925 FumingPower3925 added this to the v1.6.0 milestone Oct 26, 2026
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@FumingPower3925
FumingPower3925 force-pushed the l7-xeng branch 2 times, most recently from eb3e9e8 to c18146b Compare October 27, 2026 00:15
… Refs #885 #886 #889 #890)

Failing-first half of the lane. Every behaviour change below has a test here
that fails on baa43d9; the fixes follow in the next commit. 886.23 has its own
test and fix commits at the end, so that it can be dropped on its own.

- 889.48: OnDetach panic injected right after the count (detachPostCountHook,
  a no-op seam called between the rollback's snapshot and the place the
  rollback is registered); io_uring rollback test also asserts the
  detachedConns gauge.
- 890.22: TestBackloggedPeerIsClosed waits for the engine's refusal
  (conn.BacklogRefusedHook, defined here, called in the next commit) instead
  of sleeping.
- 885.54: parseAddr of a zoned IPv6 host, Listen on a zoned link-local address
  on both engines, and a Server start on one for every engine including
  adaptive.
- 886.24: BeginPauseAccept must signal the io_uring workers' wake eventfd;
  pause-to-closed lag measurements for both engines.
- 885.31: the MaxEvents split follows the nodes the allowed set meets
  (splitMaxEvents, detectNUMA seam; the split still uses the host count here).
- 885.53: osIfaceAddrs is one RTM_GETADDR dump (netlinkRIB seam).
The fixes for the tests of the previous commit.

- 889.48: OnDetach's rollback is registered right after the snapshot, before
  detachMu is allocated and the count is made, and the promoted-path gate is
  decided there from the snapshot's detachMu; the sync path's reliance on the
  rollback leaving detachCounted false is commented. Both engines.
- 890.22: conn.BacklogRefusedHook is called where a request is refused for the
  back-pressure cap, and TestBackloggedPeerIsClosed waits for it instead of
  sleeping 100 ms.
- 885.54: parseAddr reads a zoned IPv6 host with netip and sets the scope id as
  net.Listen does (bindiag.ZoneID). Both engines.
- 886.24: io_uring's BeginPauseAccept signals every worker's wake eventfd after
  it stores the flag. epoll's is not changed: its listening loop waits at most
  4 ms (measured 0.8 to 5.9 ms). The 2-worker pause test joins the CI step that
  runs the multi-worker listener tests by name (ci.yml).
- 885.31: epoll divides MaxEvents by the NUMA nodes the plan's allowed set
  meets, not the host's node count.
- 885.53: osIfaceAddrs reads all addresses from one RTM_GETADDR dump, mapped by
  interface index, instead of one dump per interface (311 interfaces: 21.6 ms to
  0.94 ms, 313 netlink requests to 3).
- 885.14: h2PoolSettled's doc says what becomes of a refused GOAWAY.
…886)

No behaviour change.

- 886.9: endH1Verdict of both engines states the OnError difference on a conn
  with no detachMu (epoll delivers, io_uring does not), how a sync conn gets
  into that state, and that the rule is an open decision. The io_uring side is
  pinned by two tests (the epoll side already was).
- U-12: the rx-port rule's doc says when it does not engage (since-boot
  counters at the first Listen; an evenly split network) and that it is left
  as it is. Godoc only; nothing in README.
…uest (v1.6.0, Refs #886)

The four h2c-upgrade-carry replay tests expect 0 requests and read the 101 off
the client first (expectCount886 returns at once on 0 and only holds 400 ms, so
without the 101 it could pass before the replay ran). They fail on baa43d9
(the replay counts 1).

Kept apart from the rest of the lane on purpose: the fix touches the two
reqBatch++ lines of epoll/adopt.go that #1122 rewrites (recvCounted), so this
pair is the one to drop or rebase.
…lay (v1.6.0, Refs #886)

886.23: the recv path (drainRead, handleRecv) returns on the upgrade verdict
before it counts, so a conn that upgraded on a recv counted its upgrade request
as none, while the replay of a carry that is an upgrade request counted one.
The replay now counts none, at the four sites (epoll sync and async, io_uring
sync and async). The other rule needs drainRead/handleRecv. h2 conns count no
request on either engine, so BytesPerReq moves by one request per upgraded conn
that was replayed.

On epoll the condition sits beside #1122's: `!recvCounted &&
!errors.Is(perr, conn.ErrUpgradeH2C)`. A reclaim of an upgrade carry keeps the
count drainRead made when it read the bytes (it counts before it transplants),
so that one case still counts 1; it needs drainRead.
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.27891% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/platform/nicrx_linux.go 92.72% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

… malformed zone (v1.6.0, Refs #885)

- 885.53: a dump whose last attribute is not padded to 4 makes the std
  attribute walker panic (slice bounds out of range [12:9]); rows through
  addrsByIndex for the review's fuzz crasher and six hand-built ones, and
  TestOSIfaceAddrsFallsBackOnAnUnparseableDump885. The binary panics before the
  fix.
- 885.54: ZoneID rows for a malformed zone ("12abc" is 12, a number of
  0xFFFFFF or more saturates), in bindiag and in both engines' parseAddr table,
  whose oracle now follows net's rule.
…a malformed zone as std does (v1.6.0, Refs #885)

Fixes for the tests of the previous commit (review of #1127).

- 885.53: addrsByIndex recovers from the std attribute walker's panic on an
  unpadded final attribute and reports not-ok, so osIfaceAddrs builds the table
  the old way, as its doc says. A comment says an interrupted dump
  (NLM_F_DUMP_INTR) is read as net reads it.
- 885.54: bindiag.ZoneID reads the digits a zone starts with and saturates at
  0xFFFFFF, as net's dtoi does, instead of returning 0 for "12abc" and for a
  number past 32 bits. The smaller change of the two options (matching std
  rather than narrowing the godoc).
- the backlog test's comment says a peer-address mismatch fails after 10 s and
  cannot pass falsely.
@FumingPower3925
FumingPower3925 marked this pull request as ready for review October 27, 2026 02:25
@FumingPower3925
FumingPower3925 merged commit 8936e89 into main Oct 27, 2026
28 checks passed
@FumingPower3925
FumingPower3925 deleted the l7-xeng branch October 27, 2026 02:26
@FumingPower3925 FumingPower3925 mentioned this pull request Oct 27, 2026
19 of 31 tasks
FumingPower3925 added a commit that referenced this pull request Oct 27, 2026
…the merged tree (Refs #890 item 87)

Main 8936e89 (with #1123 and #1127) run through the job's logic in docker at
CELERIS_MAX_IOURING_TIER=base: the iouring package gives 577 top-level PASS and
13 SKIP lines, the new one being TestListenOnAZonedLinkLocalServes885 (the
container has no fe80:: address); root and test/server give 118 PASS and 46 SKIP
lines, all allowed. The zoned test fails instead of skipping under
CELERIS_REQUIRE_ZONED_PEER=1, as in the unit job, so both steps set it and the
list stays as it was. min-pass floors 540 and 105.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant