[FumingPower3925]

Lane L20 GATE-UNREAD of v1.6.0. Refs #885, Refs #884, Refs #1104.

The shutdown gate of the pool wait (epoll closeIdleH1, #1116; io_uring closeIdleH1, #1123) closes an HTTP/1 conn it judges idle. It looked at the engine's own state and at the kernel's SEND queue, never at the kernel's RECEIVE queue, so a request that had reached the kernel before the gate but that the engine had not read yet was cut by the close. Found by the flakes lane while chasing the CI failure of TestShutdownPoolWaitGateRequestsInProgressAndIdleConns885/sync (#1135).

Method: commit 1 is the tests (red on main), commit 2 is the fix. Every number below is from docker golang:1.27.3, --cpus 2, -race, non-root, kernel 7.0.12-linuxkit.

item	verdict	evidence
epoll: the gate closes a conn whose request is unread in the kernel	premise TRUE (the flakes lane's test, 10 of 10 on main; now four shapes, 40 of 40 red)	tests 1 to 4 below
io_uring twin: same defect; where the request sits depends on the ring's task work (socket queue on DEFER_TASKRUN, an unreaped recv CQE on the base tier and under COOP_TASKRUN)	TRUE on both, a reset where epoll gives an EOF (20 of 20 red on the default tier; red at CELERIS_MAX_IOURING_TIER=base after the first push, found by CI)	tests 5 to 9
the silent conn's reset in the CI failure	NOT the gate: stopAccepting's close of the listener resets the accept queue (reproduced)	test 8
#1135's wait (waitEngineBusy1104) is unnecessary once this lands	NO, see "What this does not fix"	test 8

Commits

1. tests: the shutdown gate closes a conn whose request is unread in the kernel, on epoll and io_uring; the accept queue's reset is the listener's (red on main)
2. epoll, io_uring: the shutdown gate reads, and answers close-after, a request that reached the kernel before it, instead of closing the conn
3. merge of main (carries tests: test/server start retry takes a fresh port; the shutdown gate test waits for the engine (Refs #1104) #1135)
4. tests: io_uring gate pass: a conn whose recv CQE is posted and unreaped holds a request, whichever tier (red on commit 2)
5. io_uring: the gate's first pass holds a conn whose recv CQE is posted and not reaped, as well as one with bytes in its socket queue
6. io_uring: the first pass asks the sockets, then the completion queue; a later pass keeps a held conn while its bytes are unread (review r1)

Commits 4 to 6 answer a failure CI showed on the first push: the job io_uring base tier (CELERIS_MAX_IOURING_TIER=base) failed both io_uring end-to-end tests of commit 2 (kernel 6.17, tier base: socket-queue-nonempty=0 cq-unreaped=19, 18 conns closed with their request delivered). My local kernel (7.0.12-linuxkit, DEFER_TASKRUN) kept the request in the socket's queue, so TIOCINQ found it; on the base tier the recv completes when the bytes arrive, the request is a posted CQE the pass has not reaped, and the socket queue is empty. Reproduced locally with CELERIS_MAX_IOURING_TIER=base on commit 2 before changing anything (3 of 3 runs red, socket-queue-nonempty=0 cq-unreaped=live).

Tests, and the failing-on-main numbers

A handler running inline on a loop thread holds that loop (clients' requests pile up in the kernel while the loop cannot read them), then runs the gate's pass the way the loop runs it at the top of an iteration (poolWaitGate on epoll; refuseRecv = true; closeIdleH1() on io_uring). -race -count=10, on main (commit 1 alone) against this PR:

test	main	this PR
epoll TestPoolWaitGateDoesNotCloseAConnWithAnUnreadRequest1104 (the flakes lane's test, with the accept emulated inside the pass; see the reviewer notes)	10 of 10 red, 12 to 22 of 32 clients closed	10 of 10 pass, 0 closed
epoll ...KeepAliveConnWithAnUnreadRequest1104 (served, idle conns; the next request in the kernel)	10 of 10 red, 13 to 21 closed	10 of 10 pass
epoll ...SilentConnWithAUnreadFirstRequest1104 (Protocol Auto, TCP_DEFER_ACCEPT off: accepted with no byte, first request arrives during the hold; no emulated accept)	10 of 10 red, 11 to 19 closed	10 of 10 pass
epoll ...ReadsAnUnreadRequestPastTheReadBudget1104 (a 24 KiB POST against a 1 KiB drainReadBudget, the shape of the CI upload)	10 of 10 red	10 of 10 pass
iouring TestGateDoesNotCloseAKeepAliveConnWithAnUnreadRequest1104 (default tier)	10 of 10 red, 10 to 19 closed, ECONNRESET or EOF	10 of 10 pass, at the default tier and at base
iouring TestGateDoesNotCloseAConnWithAnUnreadFirstRequest1104 (DEFER_ACCEPT off, Protocol Auto)	10 of 10 red	10 of 10 pass, both tiers
iouring unit tests of both shapes (CQE posted / bytes in the socket) and the rules around them, on the gate rig	the CQE one red on commit 4	all pass, both tiers, 10 of 10
epoll TestPoolWaitGateClosesIdleConnsWithAFIN1104 (control: idle conns that hold nothing still get a FIN)	pass	pass
epoll TestListenerCloseResetsTheAcceptQueue1104 (characterisation, below)	pass	pass
epoll TestPoolWaitGateRequestInTheWindowBeforeCloseIsCut1104 (commit 2: the residual race, below)	n/a (needs the hook)	10 of 10 pass

Every answer the fixed gate gives to a request it found in the kernel carries Connection: close (the tests assert the count equals the conns of the held loop, on both engines). Raw: scratchpad l20gateunread/out/{c1-epoll-main,c1-iouring-main,fix-epoll,fix-iouring}.log.

An EOF on epoll (and on io_uring when the bytes were already a CQE: the close is then a FIN), a reset on io_uring when they were in the socket: epoll's closeConn sends shutdown(SHUT_WR) before close, so the client reads the FIN first and the request, small, is simply lost; io_uring's close sends the RST for the unread bytes.

The fix

The gate's first pass (the one that raises refuseRecv) asks the kernel's receive queue of every conn it would otherwise close (TIOCINQ, ahead of the cap and the send-side tests, so a conn those leave for a later pass is not met unasked). A conn that holds bytes has a request in progress, which the shutdown contract answers (Connection: close, as std's Shutdown answers an active conn):

• the conn is marked gateRead (a bool in the padding of connState; size unchanged, TestConnStateStaysInItsSizeClass passes) and refusesRecv / io_uring's gateReadsOn return "read on" for it;
• epoll: holdUnreadAtGate also reads it once, now (drainRead, under the same 64 KiB budget): the epoll event for those bytes may already have been taken (a read refused at the gate consumes the edge, and with EPOLLET nothing reports bytes that were already there);
• io_uring: the recv armed on the conn delivers the bytes as a CQE at the next reap, and handleRecv now reads them on;
• initProtocol (both engines) creates the HTTP/1 state close-after when the gate is up, so a conn whose protocol is decided past the gate (a first request found by the pass; also the bytes of a decision begun) is answered close-after.

Later passes do not ask the receive queue: bytes that arrive after the first pass are a request that begins after the gate, refused as before (the #1116 tests for that pass unchanged). A conn that fails the ioctl is left to the next pass; an ended detached stream is not asked (its inbound bytes are the application's).

Hot path (read from the diff): nothing is added to a per-request or per-iteration path. poolWaitGate runs on its timer; the receive-queue question is asked on the first pass only, one ioctl per conn the pass would close. One bool test in refusesRecv / gateReadsOn, evaluated only while refuseRecv is set. One store at a conn's protocol decision (initProtocol: CloseAfter = refuseRecv), per conn, not per request, on both engines. One atomic pointer load per conn closed, on the gate's pass (the test hook). Function bodies touched outside the gate files: initProtocol on epoll (loop.go) and io_uring (worker.go), plus the io_uring Worker struct field gateSwept; none in checkTimeouts. io_uring's first pass additionally reads the unreaped CQ once (holdPostedRecvs).

The close/read race

close(2) with unread bytes is an RST, so a conn is asked about its receive queue before it is closed, and a conn that holds bytes is read, not closed. In the pass the question is asked ahead of the cap and the send-side tests (so no conn is left unasked), which puts the send-queue ioctl (TIOCOUTQ) and the cap test between the question and close(2): a few microseconds. What cannot be closed is the window between the question and close(2): a request that reaches the kernel in it meets a conn judged idle. TestPoolWaitGateRequestInTheWindowBeforeCloseIsCut1104 makes it deterministic with testHookGateBeforeClose (nil by default): a request sent in the window is cut (the client reads EOF; 10 of 10), the other conns of the pass are unaffected, nothing hangs. I chose not to close that window:

• shutdown(SHUT_WR) then close does not help: Linux close still resets a socket whose receive queue is not empty (and epoll already does it);
• asking after "stopping reads" is the same as asking before on a single-threaded loop: the kernel's queue fills independently of the loop;
• the only scheme that turns the RST into a FIN is a lingering close (half-close, keep the fd registered, discard what arrives until the peer's FIN or a bound). It answers nothing either, the client's request is lost in both and an HTTP client retries it on either (a keep-alive conn that fails before a response byte), std's Shutdown has the same window when it closes an idle conn, and it adds a per-conn closing state, an interaction with refusesRecv and a time bound to the hot-adjacent code. Not done; listed under findings.

io_uring twin

The flakes lane thought the window was different (recv armed before the gate sees the conn). It is the same defect, with a tier-dependent place for the request. The pass runs at the top of the worker iteration, before the iteration enters the ring and takes its CQ snapshot, so anything the kernel delivered since the last reap meets a conn that looks idle (detectAccum empty, nothing queued):

ring	where the request is at the pass	measured
DEFER_TASKRUN (default tier, kernel 6.1+)	in the socket's queue: completions are posted only inside the worker's own enter	local 7.0.12: live=19 socket-queue-nonempty=18 cq-unreaped=1; CI default tier 6.17: live=33 socket-queue-nonempty=32 cq-unreaped=1
base tier (flags 0)	a recv CQE already posted and not reaped; the socket queue is empty	CI base tier: live=19 socket-queue-nonempty=0 cq-unreaped=19; local with CELERIS_MAX_IOURING_TIER=base: live=22 socket-queue-nonempty=0 cq-unreaped=22
COOP_TASKRUN (high/optional tier on 5.19 to 6.0)	either: task work runs at any syscall return of the worker thread, including the pass's own ioctls	not measured (no such kernel here); handled by the order below

The fix asks both places, sockets first (sweepUnreadAtGate, first pass only):

1. a walk over the live conns that gateCloseable finds idle asks TIOCINQ (holdUnreadAtGate) and marks the conn gateRead;
2. then the unreaped completion queue is read in place (holdPostedRecvs: BeginCQ head to tail, nothing consumed) and the conn of every recv CQE with bytes (Res > 0, generation matching the live conn, not an ended detached stream) is marked;
3. the walk that closes then skips a marked conn; the reap that follows delivers the CQE, and handleRecv, with refuseRecv up, reads the marked conn on (gateReadsOn) and answers it close-after.

The order matters under COOP_TASKRUN and on the base tier: the ioctls of the first walk can themselves run task work that turns a later conn's bytes into a CQE; asking the sockets first and the queue second means a request is in one of the two places when each is asked (the review r1 point). An EOF or an error CQE is not a request (Res <= 0). A later pass closes a marked conn only once its socket is empty (a conn whose recv has not been re-armed leaves the bytes in the socket; without the rule the later pass would have closed it with them unread, an RST). This works for single-shot and multishot recv, with and without provided buffers, and for a recv into the body buffer (the CQE names the conn either way).

Nothing is added per request or per iteration: the CQ walk and the second socket walk run on the first pass only (O(CQ ring) and O(live conns), once).

What remains uncovered, with evidence:

• A CQE the kernel posts after the look, and a request that reaches the socket between its question and close(2): the race any close has (std has it), shown for epoll by the window test.
• Fixed-file conns cannot be asked for their socket queue (ownsDescriptorNumber); their posted CQEs are still seen. This is unreachable in a working configuration: with the development-only CELERIS_IOURING_FIXED_FILES=1 the engine does not serve even the first request (the warning says "this path is INCOMPLETE (celeris#541)"; measured: both new e2e tests fail at the setup request, first 0: read: connection reset by peer, 5 of 5, before the gate is involved).
• SQPOLL: no tier sets it (TestNoTierEnablesSQPoll884 pins that). holdPostedRecvs reads the CQ whatever the mode, so a request already posted would be seen; not run, there is no way to select it.
• COOP_TASKRUN itself: not run (no 5.19 to 6.0 kernel here); the order of the two questions is by reading.

Tests, deterministic on both shapes: TestGatePassHoldsAConnWithARecvCQEPosted1104 arms a real recv on a socketpair conn, lets the kernel post the CQE, leaves it unreaped and runs the pass (red on commit 4, 10 of 10 at both tiers); TestGatePassHoldsAConnWithBytesInTheSocketQueue1104 leaves the bytes in the socket; ...WhoseRecvCQEIsAnEOF1104 (control: an EOF CQE is not a request), ...IgnoresAPostedRecvCQEOfAnotherGeneration1104, TestGateLaterPassKeepsAHeldConnWhileItsBytesAreUnread1104 and TestGateReadsOnIgnoresTheMarkOfAnEndedDetachedStream1104 pin the rules above. The two e2e tests (held worker, 32 clients) run at the default tier and at CELERIS_MAX_IOURING_TIER=base; they log which shape the kernel gave.

The silent conn's reset

A conn that has sent nothing has no unread byte, and the gate's close of it is a FIN (TestPoolWaitGateClosesIdleConnsWithAFIN1104: 10 of 10, EOF). No SO_LINGER exists in the engines (grep; sockopts.ApplyFD sets none), so it is not that. The reset comes from stopAccepting: the first loop iteration that sees the cancelled context closes the listener, and the kernel resets every conn still in that listener's accept queue (its doc says so; std's Shutdown has the same). The CI test dials the silent conns, the download and the upload last and sleeps 1.3 s ("TCP_DEFER_ACCEPT hands the engine a conn with no byte after about a second"); a loop that is behind, serving 24 HTTP/2 downloads of 16 MiB under -race -cover, can have all three in its accept queue at the cancel. TestListenerCloseResetsTheAcceptQueue1104 (a held loop, 32 silent conns, 32 conns with a request, 2.5 s, then the cancel): the held loop's conns read ECONNRESET (22 of 32 silent, 13 of 32 with a request in the run shown; the free loop's silent ones got a FIN from its own shutdown). The timing in the CI log (257 ms) is the times the test acted, not the times the resets arrived. The same evidence points the other two CI failures there: a gate close on epoll gives an EOF for a small request (above), but the CI saw connection reset by peer on the download head's read and on the upload's write. I cannot prove from the log that the CI conns were in the accept queue; this is the cause consistent with all three, and the only one that explains the silent conn.

What this does not fix, and #1135

The lane description said this fix makes #1135's waitEngineBusy1104 unnecessary (#1135 has since merged; this branch carries it). It does only in part: it is unnecessary for conns the engine has accepted (the gate no longer closes them with their request unread), and still needed for conns still in the kernel's accept queue at the cancel, which stopAccepting resets whatever the gate does. The wait (until ActiveConnections and RequestCount show the engine has taken everything) is what addresses those; keep it. I did not touch #1135's files.

Mutants

Each reverted after its run (-race -count=1, the named tests):

mutant	result
epoll: the TIOCINQ hold removed (if false && first ...)	...UnreadRequest1104, ...KeepAlive...1104 red (the run predates the silent and budget tests, which main's code also fails)
epoll: refusesRecv ignores gateRead	the unread-request and keep-alive tests red (the inline read is refused at its top)
epoll: never close an idle conn (closeConn removed from closeIdleH1)	TestPoolWaitGateClosesIdleConnsWithAFIN1104 and the window test red; and the #1116 wire tests TestShutdownPoolWaitGateClosesIdleH1AndEndsDetached885 (4 arms) and TestShutdownPoolWaitGateRequestsInProgressAndIdleConns885 (sync, async-route) red
epoll: always hold (the question answers "bytes" for every conn)	the FIN control and the window test red
epoll: initProtocol does not mark close-after	...SilentConnWithAUnreadFirstRequest1104 red (0 of 14 answers carry Connection: close)
io_uring: hold removed	both iouring tests red
io_uring: gateReadsOn ignores gateRead	both red (71 s: the requests are dropped)
io_uring: initProtocol does not mark close-after	TestGateDoesNotCloseAConnWithAnUnreadFirstRequest1104 red
io_uring: the CQ look removed (holdPostedRecvs not called)	default tier: TestGatePassHoldsAConnWithARecvCQEPosted1104 red; base tier: that plus both e2e tests red
io_uring: an EOF CQE held (Res < 0 for Res <= 0)	...WhoseRecvCQEIsAnEOF1104 red
io_uring: the generation check removed	...IgnoresAPostedRecvCQEOfAnotherGeneration1104 red
io_uring: a later pass closes a held conn with unread bytes / never closes it	TestGateLaterPassKeepsAHeldConnWhileItsBytesAreUnread1104 red (both)
io_uring: gateReadsOn ignores writeEnded	TestGateReadsOnIgnoresTheMarkOfAnEndedDetachedStream1104 red
io_uring: never close an idle conn, at the default tier and at base	the three TestGatePass*884 tests and the new control/hold tests red, both tiers

Local results

Final head (after the second merge of main; the full runs below were on 5eb6bb7, then main's #1136 merged and the new io_uring tests were re-run, 3 of 3 at both tiers):

• new tests, -race -count=10: io_uring at the default tier and at CELERIS_MAX_IOURING_TIER=base, epoll at the default: all pass.
• io_uring red-first on commit 4 (tests only, with commit 2's code), -count=10: at base TestGatePassHoldsAConnWithARecvCQEPosted1104 and both e2e tests 10 of 10 red; at the default tier the CQE unit test 10 of 10 red (the e2e tests pass there: the request is in the socket).
• COOP_TASKRUN, forced (deferTaskrun: false in SelectTier's high tier, in a scratch worktree, not committed; kernel 7.0.12, tier=high): all io_uring tests 10 of 10 pass; the diag shows the CQE shape (socket-queue-nonempty=0 cq-unreaped=live). Review r1 measured the first push red there.
• existing gate/shutdown tests, -race -count=3: epoll TestShutdown|TestDrain|TestPoolWait ok (55 s); io_uring TestShutdown|TestDrain|TestGate ok at the default tier (46 s) and at base (47 s); test/server TestShutdown|TestDrain|TestIOUringShutdownGate ok (357 s).
• ./internal/engine/iouring/... -race once: ok at the default tier (485 s) and at base (488 s, the CI base-tier job's setting); ./internal/engine/epoll/... ok (265 s).
• golangci-lint (repo config) on both packages: only the three errcheck findings main already has in files this PR does not touch.
• No test of this PR skips. Container runs: docker --cpus 2, non-root, one process at a time.

Release note

shutdown: with the HTTP/2 pool wait's gate (epoll, io_uring), an HTTP/1 request that had reached the kernel before the gate but was not read yet is answered (with Connection: close) instead of being cut by the close of its connection.

Findings not fixed here

• stopAccepting resets the accept queue. A client whose request sits in the kernel's accept queue when the shutdown begins gets a reset, not an answer (as std). Draining the queue (accept, answer or close with a FIN) before closing the listener would remove that class; it is a behaviour decision, and the cause of the CI flake as far as I can tell.
• io_uring: a fixed-file conn cannot be asked for its socket queue (unreachable in a working configuration, measured above); COOP_TASKRUN not run.
• Lingering close (above) for the window between the question and close(2).
• Detached streams the gate has ended are closed without asking the receive queue (WebSocket frames the client sent in the last milliseconds are unread: an RST on io_uring, an EOF on epoll).
• A conn a dispatch goroutine owns (asyncPromoted) with a request in the kernel is still left to the end of the wait on epoll (follow-up F2 of epoll: the pool wait stops serving HTTP/1 and detached writes and closes idle keep-alive conns, queued driver rounds are not idle rounds (v1.6.0, Refs #885) #1116's review); io_uring looks inside it.
• No epoll arm on an async loop (a held conn can be promoted inside the pass): the code path is the one drainRead already has, but it is not tested here.

For the reviewer to attack

1. The first-pass-only rule: a request that arrives between the first pass and a later one is refused and its conn closed (RST), as in epoll: the pool wait stops serving HTTP/1 and detached writes and closes idle keep-alive conns, queued driver rounds are not idle rounds (v1.6.0, Refs #885) #1116; is "the kernel holds it at the first pass" the right place to cut "before the gate" from "after"?
2. holdUnreadAtGate runs drainRead inline inside the walk of liveConns (loop thread, handler latency added to the pass; the walk goes from the top down so a close of the current conn is safe). Reentrancy: Loop.readYielded is reset before each event's own drainRead, so a yield inside the pass leaves nothing behind; confirm. An async-mode loop can promote the conn inside the pass.
3. An h2c upgrade request held and read past the gate: the switched conn should get GOAWAY as the late-preface case of epoll: the pool wait stops serving HTTP/1 and detached writes and closes idle keep-alive conns, queued driver rounds are not idle rounds (v1.6.0, Refs #885) #1116's r3 did; not tested.
4. CloseAfter = refuseRecv in initProtocol also reaches an adopted or transplanted conn on a loop past the gate.
5. The test TestPoolWaitGateDoesNotCloseAConnWithAnUnreadRequest1104 emulates an accept inside the pass (acceptAll); production accepts nothing at the gate (stopAccepting runs at the cancel). The keep-alive, silent and budget arms are the production-reachable ones, and Protocol: Auto in them is what makes a conn undetected (with HTTP1 the state exists at accept).
6. io_uring: holdUnreadAtGate returns "held" on an ioctl error; a conn held whose recv is not armed (needsRecv, recvPaused) waits, with its bytes in the socket, until its recv is re-armed or the end of the wait (a later pass no longer closes it while the bytes are unread). The COOP_TASKRUN order (sockets, then the queue) is by reading only.
7. sweepUnreadAtGate calls gateCloseable a second time per conn on the first pass (idempotent: it only marks close-after and takes a TryLock).

[coderabbitai]

Caution

Review failed

The pull request is closed.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 95.38462% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/engine/iouring/shutdown_gate_884.go	95.34%	2 Missing ⚠️
internal/engine/epoll/poolwait_gate_885.go	94.44%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!