[FumingPower3925]

Lane L19 H2-TIMEOUTS, stage 1 of the per-stream deadlines (the std / net/http model) for HTTP/2 on epoll, io_uring and adaptive: D1 + D0 + D3, with the M2 and M3 changes of the design review. Refs #889.

Draft. Head after review r5. Nothing in the trackers is ticked or edited. Design: design-per-stream-deadlines.md v2 (approved by the user after the Opus design review, APPROVE-WITH-CHANGES, M1 to M4); the write deadline (D2, the #1119 minimum-rate rule) is stage 2 and is not in this PR. #1134 (A, A1, A2, A3) is the base this builds on.

What it does

	rule	where
D1 request deadline	a stream whose request is not whole ReadTimeout after its HEADERS is answered 408 and reset with RST_STREAM(NO_ERROR) (RFC 9113 8.1); the conn and its other streams go on. Absolute from the HEADERS (std's rule and the documented meaning of Config.ReadTimeout): DATA and PING do not move it. A request whole at its HEADERS (every GET) has no deadline	Manager.ArmRequest / DisarmRequest / ExpiredRequests, Processor.ExpireRequest, H2State.Sweep
M3	the expired stream is remembered until the client ends it (markResetOpen, resetIDs, and a tombstone map: allocated on the first expiry, at most 1024 entries, each for at most 2 x ReadTimeout): the slow uploader's DATA and trailers in flight are ignored and credited to the conn's window whatever the age of the stream. Without it a stream older than the 2048-id reset window, after a newer reset, turned the uploader's next DATA into GOAWAY(PROTOCOL_ERROR, "DATA on idle stream"): every stream of the conn gone	deadline.go, handleData, admitHeaders
M2	ServerOwes no longer consults requestOpen when there is a request deadline (A1 retired for ReadTimeout > 0): a quiet client got the whole conn closed, SSE siblings included, where D1 resets one stream. The D pass runs before the conn scan in both sweeps; after an expiry the A3 stamp is cleared	Manager.ServerOwes, checkTimeouts
D3 header block	a header block (HEADERS without END_HEADERS) not complete ReadHeaderTimeout (default 10 s) after its first frame ends the conn: GOAWAY(ENHANCE_YOUR_CALM) and close. A fragmented block that completes in time is served	Processor.armHeaderBlock, ExpireHeaderBlock
D0 idle clock	for an h2 conn with no stream open, IdleTimeout runs from the last stream activity the sweep saw (a new client stream ID, a stream open, the DATA or trailers of an expired request still remembered, at most 2 x ReadTimeout); PING, SETTINGS, conn WINDOW_UPDATE and frames for a stream the server reset for another cause do not move it (net/http's idle timer is touched by no frame). IdleTimeout alone, as net/http: with it off a PING-only conn is left alone (pinned). The close is a proposal: the engine closes at once a conn that holds no output (write buffers, SEND in flight, and the kernel's send queue by TIOCOUTQ); a conn that holds output gets one more idle period (ExtendIdle(now, acked, ok)) only if the peer acknowledged more bytes since the last extension than the control frames the server wrote in the meantime (tcp_info bytes_acked, the source #1119's rule uses); otherwise (a client that stopped reading, or that pings faster than its round trip) it is closed within a few periods: GOAWAY(NO_ERROR) where it can still be sent, then the close (the tail may be lost). Without an acknowledged-bytes count the period is extended (fail-open, below). Silent conns are still closed by the conn's own clock at min(ReadTimeout, IdleTimeout)	H2State.Sweep, ExtendIdle

The native path runs a handler only at END_STREAM (the body is buffered up to MaxRequestBodySize; no request streaming, bidi or extended CONNECT), so an expired request has no handler to race and the 408 is always possible.

std is not touched. celeris std answers 400, not 408, when a request body read times out (bridge.go 131-135: io.ReadAll fails; net/http then resets with RST_STREAM(NO_ERROR)); a 3-line errors.Is(err, os.ErrDeadlineExceeded) to 408 is possible, but it is std's behaviour to test and review on its own, so it is a follow-up.

Commits (base 3f516f1; tests first, each rule a red commit then its code)

	tests (red on the previous head)	code
D1	13f5a2d 21 red cells, eff6322 (uploader trickles)	921ca9f
D3	502a60a 3 red cells	3838939
D0	213986f 9 red cells	6954ccc
unit, bench, godoc	8a2e5c3 H2State.Sweep unit tests; 98e90d1 benchmarks	27ef121 godoc (comment only)
later cells and fixes	56c9fba, a73dfd4, e1fa329 (the silent conn goes with the reset; io_uring's clock; Idle below Read), d7ae9d5, a0b7dea (found by -race: the test client's request() writes under the framer's lock), 4d1e0f9, f3de4f9 (lint)	469070d the sweep's look at an h2 conn is one cache line (10.7 ns to 4.3 ns per conn, below)

All new cells against origin/main 3f516f1 (logs/d1-red-main-final.log): 39 red, 27 green (the guards). Red: the flipped ping cell, a DATA dribbler (absolute), the slow uploader after the deadline, the reset-window scenario (DATA and trailers, 2100 newer streams and a newer reset), the mixed cell (quiet SSE + unfinished request) and Idle below Read, the silent client's conn, the header-block dribble, PING-only conns (after a request, from the start, after an expiry), the uploader after the reset. Green on main and still green (guards): an upload that completes before the deadline, a 1 MiB flow-controlled upload, a flowing 8 MiB download taking 2 s (longer than ReadTimeout), the pinned response cell (a pinging client that never takes a 1 MiB response: still pinned until stage 2, the test comment names it), a fragmented header block completed at once and after 0.5 s, ReadHeaderTimeout off, a busy conn at IdleTimeout 1 s, IdleTimeout off.

Cells per engine (epoll, adaptive, io_uring; std unchanged): e.g. ping cell status="408" reset=true rst=NO_ERROR at 1.0 s and the next request on the same conn served; header block GOAWAY ENHANCE_YOUR_CALM at 1.00 to 1.02 s; PING-only conn closed at 1.00 to 1.14 s with IdleTimeout 1 s (logs/d1-green-*.log).

Review r3 (Opus, REQUEST-CHANGES: one MAJOR, no blocker) and what changed

• F1b/F1c, the regression (fixed): D0 closed the conn while the tail of a finished response was still on its way to a slow client: the stream is gone from the manager once its last DATA is in the conn's buffers, and a close with the client's frames unread in the receive queue is a TCP reset that discards the send queue. Measured by the reviewer on the first head: a client reading 8 MiB at about 1 MB/s with normal WINDOW_UPDATEs (4 MiB stream window, IdleTimeout 1 s): origin/main whole, head cut at 5046272 bytes on epoll and adaptive; the same client pinging instead: main whole, head 49152 bytes. Two red-first cells (h2_idle_tail_889_linux_test.go, from the reviewer's probes): on the first head 4 red (epoll and adaptive, both cells; io_uring green) and, with the burst cell, 7 red and 2 green (logs/d1-r3-red.log), after the fix 9 of 9 green (logs/d1-r3-green.log: epoll and adaptive whole at 2.5 s and 8.7 s). A mutant that skips the held-output check: 4 red again (logs/mutant-r3-no-holds-output.log); the PING-only cells (reaped at IdleTimeout) stay green. The fix chosen, and why: Sweep's CloseNow is now a proposal; the engine closes only if the conn holds no output: write buffer, dirty list, EPOLLOUT, writeSince, and the kernel's send queue (TIOCOUTQ, unsent plus unacknowledged: the notion the pool-wait gate already uses before it closes a conn; an ioctl error counts as "holds"); otherwise H2State.ExtendIdle restarts the idle period. This is the smaller safe change: GOAWAY plus closeWhenFlushed would re-implement the closing drain for a conn that is not closing, while the check does nothing while a response is parked, so D0 did not fight epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119's minimum-rate rule (merged, 46753f9: it governs parked responses and the conn scan's exemption); r4 then capped the extension (below), which is a different, weaker test than the rate rule, and a client that finishes taking its download is not closed on the instant (IdleTimeout runs again from the drain). One TIOCOUTQ syscall, and, when output is held, one getsockopt(TCP_INFO) (r5), only when a conn is already about to be closed as idle.
• F6: the idle close D0 makes now sends GOAWAY(NO_ERROR) (best effort: TryLock, flushed inline on epoll; on io_uring staged and sent through the deferred close, whose drain gives the SEND its time: r5 F1 found that the r4 rework lost it, fixed). The closes of the conn scan (a silent client after the 408) stay bare closes, as on main: the scan reads h2State under the detachMu logic and has many close sites; follow-up (Findings 6).
• F4: expiries are answered in batches of 32 under a 2 ms budget until none is due (were 8 per sweep): the last of 100 simultaneous expiries was 0.3 s late on epoll and 1.27 s on io_uring; red-first cell (TestH2BurstOfExpiriesIsAnsweredInOneSweep889, 3 red with 8 per sweep, logs/d1-r3-burst-red.log), now all 100 within one sweep (1.014, 1.024 and 1.172 s).
• N1 reqArmed (written, never read in production) removed: one atomic add and one CAS fewer per POST. N3 a client RST_STREAM on an expired stream ends its memory. N4 ExpireRequest calls MarkStreamClosed on the conn writer as sendRSTStreamAndMarkClosed does; r4 found that both writer adapters' MarkStreamClosed are empty (internal/conn/response.go ~685 and ~1047), so the call is a no-op today: it stays for a writer that records closed streams, the comment now says so, and what keeps a late write from following the reset is the stream's removal from the manager. N2 (a cached clock instead of time.Now() per POST, wall-clock steps) and N5 to N7 are noted, not changed (bounded, harmless: the reviewer's grading).
• The dangerous leak direction is pinned: with a request deadline every stream whose request is open carries it and the hint is due by the earliest (TestEveryRequestOpenStreamCarriesADeadline889; a mutant that skips the arm for one opener is red, logs/mutant-r3-opener-without-arm.log): a request-open stream without the flag would make ServerOwes true for it and nothing would ever reset it.
• Verified by the reviewer (kept here as evidence): x/net http2.Transport returns the 408 to the caller and keeps the conn (all three engines, 1.001 to 1.004 s); curl 8.14.1 (nghttp2) reports 408 and treats the RST_STREAM(NO_ERROR) as success; the h2c upgrade arms its deadline in the frames trailing the upgrade (408 at 1.0 s, stream 5 served); late frames on an expired stream (DATA, WINDOW_UPDATE, PRIORITY, empty trailers) keep the conn; 100 simultaneous expiries keep the conn with no GOAWAY; the conn window is credited for the ignored DATA (1 MiB upload completes after an expiry).

Review r4 (Opus, APPROVE-WITH-FIXES) and what changed

Tests first where a test applies; merged main (a merge commit; nothing rebased).

• F-ZW, D0 was not capped (fixed, the main change): ExtendIdle restarted the idle period for as long as the conn held output, so a client that stops reading and keeps pinging held a stream-less conn for ever (the reviewer: 60 s with IdleTimeout 1 s, epoll, adaptive, io_uring; origin/main holds it as long, so not a regression, but D0 is where it is bounded), and a client that reads everything but pings faster than its round trip held it too, because the PING ACKs are always unacknowledged bytes in the send queue (reviewer: 10 s at 300 ms RTT, 200 ms pings, netem). (The measure below is r4's; r5 replaced it, see "Review r5" below, because on io_uring it double counted.) The rule was: ExtendIdle(now, outstanding) remembered the bytes the conn holds (write buffers plus TIOCOUTQ, unsent and unacknowledged; never the unsent-only count) at each extension, and at the next expiry refuses unless the count is lower; the engine then sends GOAWAY(NO_ERROR) best effort and closes (a reset that drops the tail is accepted: the client is not taking it). Stream activity clears the baseline. Bounds, as tested: a reader that does not read, within two idle periods (+ one 100 ms look); a reader at 1 MB/s with pings, the whole 8 MiB (the count goes down at every period); pings faster than the RTT: the count of unacknowledged ACKs goes up and down, never below the last extension for long, so it ends after as many extensions as there are ACKs in flight at most (2 here) and is closed within 4 periods, measured 2.5 s on epoll and adaptive, 3.7 s on io_uring with IdleTimeout 1 s (held 10 s on the previous head). On io_uring the close of a conn with a SEND in flight is deferred behind the drain (closingDrainBound, up to WriteTimeout, celeris#761): closeIdle (now closeRefused) sets that clock back as closeStalled does (taking the acknowledged-bytes baseline first, or the sweep's first sample would count as progress and give the bound again), so the conn is reaped at once (in the sweep that refused it; r5 narrowed that to the refused conn).
  • Red first (beef41f; the cap is ddeada2): zero-window pinger, 3 red on the previous head (open and unread after 10 s on epoll, adaptive, io_uring); ACK lag (-tags netem889: root and tc with NET_ADMIN, not built in CI: the CI skip rule allows no skip, and it would run nothing there): 3 red (still open after 6 s) on the previous head; both green with the cap. The existing tail cells stay green (the PING one now waits 1.5 s instead of 2.5 s: one idle period, which the cap rightly ends at the second; the 1 MB/s reader whole at 8.7 s).
  • Mutants (logs/mutant-mc*.log): always extend: 3 e2e + unit red; equal count extends (> for >=): unit red (the e2e cell is not, because the PING ACKs make the count of the zero-window conn go up); baseline never recorded: 3 e2e + unit; activity does not clear the baseline: unit; io_uring close held a drain bound: 1 e2e (io_uring).
  • How this relates to epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119 and to stage 2: epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119's rule (epoll, WriteTimeout) is a minimum rate on a response parked on the socket: at least min(64 KiB, what is held) acknowledged per WriteTimeout window, else the conn is reset. The D0 cap is not a rate: since r5 it asks only that the peer acknowledge more bytes than the control frames the server sent it over one IdleTimeout (r4's version asked that the bytes held go down), for a conn with no stream open that the idle clock already proposed to close. It is weaker than epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119's rule on the amount (a client that takes a few bytes per period, more than the control frames the server wrote in it, passes it, and is the rate rule's to bound) and stricter on the time (IdleTimeout is usually shorter than WriteTimeout), and it also covers a conn with no parked write at all (the tail is in the kernel's queue, which epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119 does not govern) and io_uring, which has no such rule yet. The two do not fight: D0 only closes where it would otherwise have extended, and an extension was never a promise to keep a conn the rate rule would reset. Stage 2's D2 (a per-stream minimum-rate write deadline for a response waiting on the client's flow-control window, epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119's rule per stream) does not replace this cap: D2 is per stream, and this is the conn-level, byte-count cap for a response whose stream is already gone from the manager.
• F-BURST (fixed): TestH2BurstOfExpiriesIsAnsweredInOneSweep889/io_uring was 3 of 10 red under 6 busy loops (spreads 206 to 308 ms against 150 ms). The bound is now 800 ms (what a loaded machine adds, the client's reads included; the old 8-per-sweep behaviour was 1.2 s on io_uring): 30 of 30 green under 6 busy loops (3 engines x 10; the largest spread 0.30 s); the one-sweep semantics are pinned by the unit test.
• NIT-b: the ExtendIdle comment said the period restarts "once the output has gone"; the code restarts it at once, the comment says so. NIT-a (the io_uring byte count on a fixed-file conn: the ioctl would read a ring index): the conn is not asked (development-only mode, CELERIS_IOURING_FIXED_FILES=1, off by default). r5 turned outstanding() back into holdsOutput(), a yes/no. N4 (above). NIT-c (the hot path section below).
• A CI flake in a stage 1 cell (test only, 41421ba): TestH2ExpiredRequestDataAfterTheResetWindow889/adaptive/trailers failed once under -race in CI (the 408 at "900 ms"): its lower bound counted from the last frame the client sent, and 2100 requests on a loaded runner take over 100 ms; the bound now counts from the HEADERS of stream 1. 5 of 5 under 6 busy loops with -race.
• Q4, not changed: the 2 ms expiry budget is per conn, so C conns each with N simultaneous expiries cost one sweep C x N x (a few us) on the loop thread; the work is what an attacker had to send first. A worker-wide budget is a follow-up.
• Q3 (GOAWAY(NO_ERROR) on the idle close): confirmed by the reviewer at r4 on the r4 head; r5 found it lost on io_uring on the r5 head (F1) and it is back, pinned on every engine (below).

Review r5 (Opus, REQUEST-CHANGES: one BLOCKER, one MAJOR, both in the new D0 cap on io_uring) and what changed

Tests first (61db56e), the fix after (b6d5d6e), unit tests with it.

• F0 BLOCKER, the cap cut a reader (io_uring): outstanding() added the in-flight SEND's sendBuf to TIOCOUTQ, and the two hold the same bytes (the reviewer's trace: sendBuf constant at 2,296,102 for the whole SEND, TIOCOUTQ rising 245 KB when the kernel pulled the next chunk of it), so the count went up while the client read, ExtendIdle refused, and the close was a reset: a plain client (default buffers, no PING, a WINDOW_UPDATE per frame) at 200 to 500 KB/s was cut at 1 MiB of 4. Fix: the measure is the bytes the peer acknowledged (tcp_info bytes_acked, monotonic, counted once, the source epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119's rule reads), on both engines (one rule). ExtendIdle(now, acked, ackedOK) extends when the peer acknowledged more since the last extension than the control frames the server wrote to it in that time (ctlOut, counted where the state flushes its own buffer: PING and SETTINGS acknowledgements, WINDOW_UPDATEs): without that floor the PING answers, acknowledged one by one at the pace of the RTT, would count as the client taking the response (the ACK-lag case). TIOCOUTQ remains the "does it hold output at all" test, a yes/no, not a sum. A flat acknowledged count refuses (zero window), a growing one extends (any reader).
  • When the count is unavailable (getsockopt failed, a fixed-file conn, or a 0 on a host whose stack does not fill bytes_acked; see r6 F7 below: a 0 on a Linux stack that fills it is a count): fail-open, the period is extended, as before the cap. Why: closing a conn whose progress cannot be seen is the r3 failure (a reader cut), and the stacks that cannot answer are exactly the ones where that would happen (gVisor); such a conn is bounded by the conn's own clock only, as epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119's rule falls back to the flush clock. A disagreement with the review (F3 NIT): it called fail-closed after two periods the right policy for a failing ioctl; with the acknowledged count as the measure, an ioctl that fails is the "holds output" answer (yes), and a socket on which TIOCOUTQ fails is one on which the next read or write fails too and closes the conn; closing it earlier for a measurement error trades a bounded hold for a possible cut of a live reader.
  • Cells (every engine; the shapes of the reviewer's probes): a plain client reading 2 MiB and 4 MiB at about 500 KB/s (TestH2IdleClockDoesNotCutAReaderThatReads889): red on ade1035: 1 of 6 (io_uring 4 MiB, cut at 1 MiB), green after; a reader taking 64 KiB every 500 ms with a 4 KiB receive buffer set before connect and a PING every 300 ms, 2 MiB (...BurstyReader889): red on io_uring (cut at 786432), epoll and adaptive green, green after. (My first version of the bursty cell set the buffer after connect and was red on epoll too, for the wrong reason: zero-window probe back-off, no progress for more than a period.) The existing cells stay green: zero-window pinger 3 of 3 (closed at 3.5 s), the 8 MiB PING tail, the 8 MiB reader at 1 MB/s (8.8 to 9.3 s), and the netem ACK-lag cell (-tags netem889) 6 of 6 green (closed after 2.5 to 3.5 s), 6 of 6 red without the control-frame floor.
• F1 MAJOR, io_uring lost the idle close's GOAWAY: closeIdle set the drain clock back for every idle close, including a conn with nothing to send but its GOAWAY, and the same sweep's liveConns pass then reaped it and shut the socket before the GOAWAY's SEND was submitted (the reviewer: 0 of 5 on ade1035, 5 of 5 on the r4 head). Now only a conn whose extension was refused (output not going out; a GOAWAY behind a SEND the peer does not take is lost anyway) is set back and reaped at once (closeRefused); every other idle close is closeConn, which defers behind the GOAWAY's SEND and gives it its drain. Cells: TestH2IdleCloseSendsGoAway889 (3 conns per engine) and the three PING-only cells (after-a-request, from-the-start, expired-then-ping), which accepted a bare EOF, now require GOAWAY(NO_ERROR) before the close: red on ade1035: 4 cells (io_uring only), green on all engines after.
• F2 MINOR, a frame on a reset stream restamped the clock for ever: after any server reset, a client could send one byte of DATA on that stream each period, which was "ignored" but counted as stream activity (idle clock and the cap's baseline), indefinitely and free. Now only the DATA and trailers of an expired request (a tombstone, at most 2 x ReadTimeout) are activity; a frame for a stream reset for another cause, or in the window of reset ids only, is ignored and is not. Cells: TestH2FramesOnAResetStreamAreNotActivity889 (no :path, no END_STREAM, DATA every 300 ms): red on ade1035 on 3 of 3 engines (open at 6 s), closed at 1.03 to 1.11 s after; units for both sides (TestSweepFramesOnAResetStreamAreNotActivity889). The uploader cell now watches 1.7 s after the reset instead of 3 s (it saw a close at 3.9 s under -race, when the tombstone's 2 s ended; the new rule's consequence, not a regression).
• F4 MINOR, burst bound: the 800 ms e2e bound caught the eight-per-sweep regression on io_uring only, and "one batch of 32 per sweep" survived everything. The expiry budget is now a variable and the unit test counts sweeps: with the budget off, 100 expiries take one Sweep (a mutant with one batch per sweep needs 4, eight per sweep 13); with the budget spent at once, 32 per sweep and 4 sweeps. The e2e bound stays 800 ms (flake-proof; it is the loaded-machine fallback, not the measure).
• F6 mutants, re-run after the rewrite (unit = internal/conn, e2e = the cells of the table):

mutant	killed by
always extend	unit, zero-window cell (3 engines), netem cell (3)
an equal acknowledged growth extends (< for <=)	unit only
baseline never recorded	unit, zero-window cell (3 engines)
activity does not clear the baseline	unit only
no control-frame floor	unit, netem cell (6 of 6)
io_uring: the refused close held a drain bound	zero-window cell (io_uring)
io_uring: every idle close reaped at once	GOAWAY cell (io_uring)
any ignored frame is activity	unit, reset-stream cell (3 engines)
one batch of 32 per sweep	unit
eight per sweep	unit

• NITs: the outstanding() comments are gone with it (the engines' holdsOutput is a yes/no; its comment says what a failing ioctl means); the release note's "about two periods" is "a few periods" (zero window: two; ACK lag: up to four, measured 2.5 to 3.5 s with IdleTimeout 1 s); the stale Q3, "staged before closeConn" and "next sweep" lines above are corrected. The netem cell is not run in CI (nothing references -tags netem889: it needs root, tc and NET_ADMIN, and no existing job has them; adding a privileged job is not done here), so it is not CI evidence: its evidence is the manual root-container runs listed in Evidence. The hot words of the sweep's look are the first 40 bytes of H2State (the size pin is corrected in r6 F8).

Review r6 (Opus, APPROVE-WITH-FIXES: F0 and F1 confirmed fixed with the reviewer's own probes; three MINOR and nits)

• F9, io_uring lost the GOAWAY of a refused conn that is still reading: a client that reads everything but pings faster than its round trip is refused by the cap; closeRefused set the drain clock back past its bound, so the same sweep reaped the conn before the GOAWAY's SQE was submitted (bare EOF, 0 of 3; epoll and adaptive 3 of 3). Now the bound runs out at the next sweep (lastActivity = now - bound + 1): the loop submits the SQE in between, and a zero-window peer is still reaped one sweep later, not after the drain bound. Submitting the ring before the reap was not used (on an SQPOLL tier a submit only wakes the poller). The closeRefused comment ("lost anyway") was true only for a zero-window peer and is rewritten. Cell: the netem ACK-lag cell now requires GOAWAY(NO_ERROR) before the close on every engine (-tags netem889, manual): red on ebf390e: io_uring 3 of 3 (bare EOF at 2.50 to 2.52 s; epoll and adaptive green), green after: 15 of 15 (5 rounds x 3 engines, GOAWAY seen every time, closed after 2.4 to 3.5 s). Not CI evidence.
• F7, a count of 0 was treated as "no count": fail-open on bytes_acked == 0 let a peer with its own TCP stack that never acknowledges data (or advertises a zero window from the start) hold a conn whose output fits in the kernel's send queue for ever; on Linux since 4.1 a 0 is a real count (a fresh socket reads 0: the passive side does not count the SYN-ACK). Now 0 is a count, unless the host's stack is known not to fill the field: internal/tcpack answers that once per process, latched one-way (any socket that has reported a count above 0, or a loopback probe: one byte, the writer's tcp_info read for up to 100 ms), both engines. The probe runs in a goroutine and is never waited for (tcpack.Start, called when an engine is built; Filled and Usable answer from the latch at once, so an event-loop or worker thread never blocks on a broken loopback; until the probe says yes, a 0 reads as "no count", the behaviour before the cap). The first version ran the probe inside sync.Once.Do on the first caller, a loop thread: found by the coordinator's read of the diff. The baseline is an explicit d0Has flag, not the max(a, 1) marker (a second 0 wrapped to a 4 GiB "progress" and extended). Fail-open stays for a failed getsockopt, a fixed-file conn and a host that does not fill the field; the godoc and the ExtendIdle comment say so. Tests: unit (a-zero-count-is-a-count: the first extension is given, the second at 0 refused) and tcpack (the latch is one-way, the probe runs once, an error is never usable, the probe sees this host's stack, a probe that blocks on a channel does not block Usable/Filled callers and the latch sets when it completes, a probe that never completes leaves a 0 as "no count"; red against the synchronous version: 2 tests). No e2e cell (a never-ACKing peer needs a user-space TCP stack; not built), so this is pinned by the units, said plainly. The unit cell cannot be red on ebf390e as an assertion (it references the new flag: a compile failure there); the old behaviour is demonstrated by the marker mutants below.
• F8, the size pin's premise: Go 1.27 adds an 8-byte malloc header to an object with pointers over 512 bytes, so a 640-byte H2State costs 704 (main's 560 costs 576), and the edge is 696, not 640. So this PR moves every h2 conn's H2State from 576 to 704 bytes (+128 B), small next to a conn's buffers but not "staying in its class" (the earlier body lines and test comment said so; corrected). The test now pins sizeof + 8 <= 704. Stage 2 has 56 bytes of room in H2State. The same arithmetic for Stream (1024 costs 1152, 120 bytes free) is in the design note (the existing Stream pin is h2: end or reset a stream whose handler returns without finishing its response; log and count a Goexit #1030's and is not touched here).
• NITs: the stale lines (r3's TIOCOUTQ sentence, the "go down at all" sentence, the NIT-a line) now say what r5 changed; the netem cell's comment follows the new rule; the godoc says a peer that acknowledged nothing is refused; "two periods" is "a few periods" everywhere (the zero-window cell bounds it at 3.5 s; ACK lag 2.4 to 3.6 s).
• Mutants (r6 additions): reap the refused conn in the same sweep (the r5 code): netem cell, io_uring 3 of 3; a 0 treated as unavailable for ever (tcpack.Usable): tcpack unit; the latch never set: tcpack unit; the old max(a, 1) marker (two forms): unit; acked == 0 is no count in ExtendIdle: unit.

Evidence

CI failure at 7392da9 and what it was (TestH2IdleClockClosesAConnWhoseClientStopsReadingButPings889/io_uring, -race, a GitHub runner; the zero-window cell, conn still open 9.5 s after): not the async probe and not a hole. Reproduced under load (6 busy loops, -race): before, io_uring 15 of 20 red at 2 cpus and 20 of 20 at 1, also on ebf390e (5 of 10), so older than the probe. A trace of the judgements (holds true, the count readable every time, so the latch and the probe were not involved) showed the client's acknowledged bytes growing by 33 to 65 KB per period while it read nothing: the cell set its 4 KiB receive buffer after the connect, the window the handshake had offered stayed, and under load the kernel went on accepting bytes; by the cap's rule (more acknowledged than the control frames) that is a slow reader, and the cap extended it as designed. The cell now sets the buffer before the connect (as the reviewer's probe does): 60 of 60 green on all three engines at 2 and at 1 cpu under the same load. Also from this: closeRefused no longer lets the next sweep take its own sample of the peer's acknowledged bytes (closeSampled set past it): the ACKs of bytes in flight at the decision arrive after it and the sample gave the conn another drain bound; a unit test with a real socket pair (TestRefusedCloseIsNotRescuedByAStragglerAck889) is red 10 of 10 before, green 10 of 10 after; this was a real race, found while looking, but not the CI failure (the loaded rate did not move with it: 13 of 20 red after). A unit subtest pins that a count appearing late (probe answered mid-life) opens no hole.

After the probe change (tcpack in the background; head below): internal/conn, internal/tcpack and internal/protocol/... -race -count=5 ok; the changed cells -race -count=5 -cpu 2 ok (601 s); ./internal/engine/epoll/... ok (372 s) and ./internal/engine/iouring/... ok (486 s) under -race; lint v2.14.0 0 issues (also -tags netem889); GOOS=windows and darwin vet ok; apidump ok; size tests pass (logs/r7-verify.log). The r6 numbers below are for the head before it; no other code changed.

After r6 (head below): the new and changed cells (the r5 pattern plus the GOAWAY cells) -race -count=5 -cpu 2 ok (606 s: the zero-window cell, the reader cells, the GOAWAY cells and the PING-only cells 5 times each); ./internal/conn, ./internal/tcpack and ./internal/protocol/... -race -count=5 ok; ./internal/engine/epoll/... ok (400 s) and ./internal/engine/iouring/... ok (491 s) under -race once; the whole test/server H2|889 set non-race: 1281 PASS, 0 FAIL, 0 SKIP (logs/r6-verify.log); the netem cell (manual, root container, not CI) 15 of 15 green without -race and 9 of 9 with it (GOAWAY seen every time, closed after 2.4 to 3.7 s); golangci-lint v2.14.0 0 issues (also -tags netem889); GOOS=windows and darwin go vet ok; apidump ok (53 packages); size tests pass (connState 448, Stream 1024, H2State sizeof + 8 <= 704); BenchmarkSweepH2Deadlines889 (epoll, 4096 conns, nothing due), interleaved with the r5 head, 3 rounds: 28.8 to 36.9 us against 30.1 to 35.5 us (the machine was shared; no difference).

After r5 (head below): the new and changed cells (TestH2IdleClock*, TestH2IdleClose*, TestH2FramesOnAReset*, TestH2Burst*, TestH2PingOnly*, TestH2ExpiredRequest*, TestH2Uploader*) -race -count=5 -cpu 2 ok (604 s; the first such run found the uploader cell's bound, above); ./internal/conn and ./internal/protocol/... -race -count=5 ok; ./internal/engine/epoll/... ok (396 s) and ./internal/engine/iouring/... ok (488 s) under -race once; the whole test/server H2|889 set non-race: 1281 PASS, 0 FAIL, 0 SKIP (logs/r5-verify.log, logs/r5-full.log); the netem cell (-tags netem889, root container with NET_ADMIN, manual, not CI) 5 of 5 rounds x 3 engines green (closed after 2.35 to 3.56 s, IdleTimeout 1 s); golangci-lint v2.14.0 0 issues (also -tags netem889); GOOS=windows and darwin go vet ok; apidump ok (53 packages); size tests pass: connState 448, Stream 1024, and a new one pins H2State at 640 (a malloc size class). BenchmarkSweepH2Deadlines889 (epoll, 4096 conns, nothing due): 28.7 to 29.6 us, 256 conns 1.33 to 1.43 us, where r4 recorded 31.2 us and +2.9 ns per conn: the sweep's look is unchanged by the rework.

After r4 (head below): the new and changed cells (TestH2IdleClock*, TestH2Burst*) -race -count=5 -cpu 2 ok (152 s); ./internal/conn and ./internal/protocol/... -race -count=5 ok; ./internal/engine/epoll/... ok (396 s) and ./internal/engine/iouring/... ok (487 s) under -race once (logs/r4-verify.log); the whole test/server H2|889 set non-race: 1262 PASS, 0 FAIL (and 1 SKIP, the netem cell, which then skipped without its env; it is now behind -tags netem889 and not built in CI; run separately in a root container with NET_ADMIN: red 3 of 3 on the previous head, green 3 of 3 and then 12 of 12 more, closed after 2.4 to 3.6 s; one earlier run failed in its setup under netem without the cell's own message; logs/r4-full.log); the burst cell 30 of 30 under 6 busy loops; golangci-lint v2.14.0 0 issues (internal/conn, protocol, engine, test/server, root); GOOS=windows and darwin go vet ok; apidump (what mage apiCheck runs) ok, 53 packages; TestConnStateStaysInItsSizeClass (448) and TestStreamStaysInItsSizeClass889 (1024) pass.

• New test/server cells 75 of 75 green on the branch (all three native engines; 66 before r3, 9 added): the first 66 -race -count=10 -cpu 2 ok (757 s, logs/d1-race10.log; its first run found a race in the test client, fixed in a0b7dea); after r3 the new and changed cells -race -count=5 -cpu 2 ok (396 s, logs/d1-r3-race.log). Unit tests (internal/protocol/h2/stream, internal/conn): -race -count=5 ok.
• Packages under -race once, after r3 (logs/d1-r3-race.log): ./internal/protocol/..., ./internal/conn/... ok; ./internal/engine/epoll/... ok (405 s); ./internal/engine/iouring/... ok (492 s). Main (with epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119 and cache: the Store.Set default bound is 1 second (v1.6.0, Refs #888) #1151) is merged in. Whole test/server non-race: ok (817 s) before the last commits. TestConnStateStaysInItsSizeClass (448 bytes) and the Stream size test (1024 bytes) pass: nothing grew.
• golangci-lint v2.14.0 (the CI pin, installed in docker): 0 issues on the touched packages and .; GOOS=windows and darwin go vet of ./internal/conn/... ./internal/protocol/... . ok; mage apiCheck ok (the Config change is comment only).

Mutants (each applied, the named tests run, file restored; logs/mutant-md*.log)

mutant	red
expiry never fires	24 cells (the D1 set)
every request armed (GET included)	6 cells (a long SSE stream reset) + 3 unit
a DATA frame moves the deadline	12 cells
no tombstone (M3)	6 cells (DATA and trailers after the window) + unit
ServerOwes consults requestOpen (M2)	3 cells (mixed, Idle below Read)
A3 stamp not cleared at an expiry	2 cells (epoll, adaptive)
header-block deadline never armed	3 cells
header block ends with PROTOCOL_ERROR	3 cells
the idle clock counts every frame	9 cells + unit
ignored frames not counted as activity	3 cells
the progress cap: always extend (r4)	3 cells + unit
the progress cap: an equal count extends (r4)	unit
the progress cap: no baseline recorded (r4)	3 cells + unit
the progress cap: activity keeps the baseline (r4)	unit
io_uring: the refused close held a drain bound (r4)	1 cell (io_uring)

Hot path and cost

A GET pays one test (the endStream test admitHeaders makes anyway); nothing else on its path. TestRequestDeadlineArmedOnlyWhenTheRequestIsNotWhole889, TestRequestDeadlineNeverAllocatesOnTheRequestPath889 (0 allocs for arm and disarm, one flag load for a stream that never armed). A POST with a body pays, at its HEADERS, one clock read, a plain store, a flag Or and the earliest-deadline hint (a CAS loop), and at END_STREAM a flag And; StartTimeNs is zeroed at the disarm so the handler's latency does not include the upload. No lock, no table, no allocation. (The armed-request counter, with its atomic add and CAS, was removed by N1.)

Measured by the r4 reviewer, natively on arm64 (Mac, go1.27.3, 15 rounds interleaved over 5 binaries, medians vs main; an A/A copy of main as the control):

benchmark	A/A copy of main	head	head with deadlines off (H2Config{})
ProcessH2Headers889/single (GET, main 394 ns)	+3.0%	+1.3%	+1.4%
ProcessH2Headers889/pipelined8 (main 3132 ns)	+1.0%	+2.3%	-0.4%
ProcessH2Post889/single (POST with a body, main 619 ns)	+0.3%	+7.2%	-1.0%
ProcessH2Post889/pipelined8 (main 4994 ns)	-0.5%	+7.8%	-1.5%

An earlier 2 x 5-round run: POST +5.7% and +5.4% (A/A +1.0% and -0.5%). So the cost of the feature is about 30 to 45 ns per POST that carries a body (+5 to 8% of the ProcessH2 work on this host), from the arm and the disarm; a GET pays within noise (+1 to 2%); with the deadlines off the head equals main. Allocations unchanged (POST 4 and 32, GET 2 and 16). The N1 removal is not resolvable at this noise. darwin clock reads are slower than Linux's vDSO, so amd64 is likely smaller (the docker run below: +1.0% and +0.2%), but amd64 is unmeasured (docker amd64 emulation is not a timing source): the cluster P-run covers it. The builder's earlier docker run (arm64 in a VM, 8 rounds with a twin that itself differed from main by up to 10%) is kept for the sweep numbers:

Docker --cpus 2, arm64, 8 rounds interleaved main / branch / A/A twin of main (the Mac was in use: the twin differs from main by up to 10% in medians and 15% in minimums), medians and minimums (logs/bench7-table.md):

benchmark	main med / min	A/A twin vs main	branch vs main	allocs
ProcessH2Headers889/single (a GET)	461 / 404	+0.0%, +0.4%	-6.2%, -1.6%	2 -> 2
ProcessH2Headers889/pipelined8	3497 / 3111	+1.5%, +1.3%	-3.7%, +0.9%	16 -> 16
ProcessH2Post889/single (POST + body)	710 / 619	-6.4%, +3.6%	+1.0%, +6.0%	4 -> 4
ProcessH2Post889/pipelined8	5574 / 4960	-2.7%, -0.1%	+0.2%, +7.0%	32 -> 32
InlineDispatch893	94 / 78	+1.6%, +3.6%	-0.2%, +5.1%	0 -> 0
AsyncRequest951	969 / 878	+7.1%, +1.2%	+0.6%, +0.3%	same
PoolDispatch	792 / 660	-10.8%, -5.0%	+0.3%, -0.2%	1 -> 1
StreamedResponse889/closed	604 / 572	-4.6%, -4.4%	+0.3%, +1.7%	2 -> 2

The POST minimums there (+6 to +7%) were the one place a cost showed; the reviewer's run above resolves it. The sweep gains a pass over the h2 conns (BenchmarkSweepH2Deadlines889: n h2 conns on h2Conns, nothing due, main vs branch): epoll 4096 conns 13.4 us to 31.2 us (+4.3 ns per h2 conn per sweep), 256 conns +2.9 ns; io_uring 4096: 30.7 us to 47.9 us (+4.2 ns), 256: +3.6 ns, the same whether the conn has no stream, an exempt stream or an armed request (CheckTimeouts865/888, the conns not on h2Conns, move within the twin's spread). The first version of the pass cost 10.7 ns per conn (three heap objects, three cache lines); commit 469070d puts the hints in one struct at the head of H2State and reads the idle clock's counters every 100 ms. At 10 000 h2 conns on a loop and 40 sweeps a second: 43 us per sweep, 0.17% of a core. amd64 is unmeasured (docker amd64 emulation is not a timing source): the cluster P-run covers it.

Release notes (draft)

• HTTP/2 on epoll, io_uring and adaptive: ReadTimeout now bounds each request, as net/http's does: a stream whose request (body or trailers) is not complete ReadTimeout after its HEADERS is answered 408 and reset with RST_STREAM(NO_ERROR); the connection and its other streams continue. The deadline is absolute: DATA and PING do not move it, so an HTTP/2 upload must fit in ReadTimeout (default 60 s: a 4 MiB body needs about 70 KB/s); there is no per-route override. A request without a body is unaffected. Native HTTP/1.1 keeps "no progress for ReadTimeout".
• ReadHeaderTimeout also bounds an HTTP/2 header block (a HEADERS frame without END_HEADERS): a block not complete within ReadHeaderTimeout ends the connection with GOAWAY(ENHANCE_YOUR_CALM).
• IdleTimeout for an HTTP/2 connection with no stream open runs from the last stream activity; PING, SETTINGS and connection WINDOW_UPDATE frames do not move it, so a client that only pings no longer holds a bare connection for ever: PING-only connections are reaped at IdleTimeout when IdleTimeout > 0 (with IdleTimeout off they are left alone, as before). It is IdleTimeout alone (ReadTimeout is a request's deadline). A connection that still has output pending is held only while the output makes progress: a connection with bytes to send or unacknowledged (in its buffers or in the kernel's send queue) gets another idle period for as long as its client acknowledges more bytes in each period than the control frames the server sent it; a client that stops reading, or that pings faster than its round trip, is closed within a few periods (the zero-window cell bounds it at 3.5 s with IdleTimeout 1 s; ACK lag: measured 2.4 to 3.6 s with 300 ms RTT and 200 ms pings, at most about four periods; GOAWAY(NO_ERROR) where it can still be sent, then the close; the tail may be lost). The measure is the bytes the peer acknowledged beyond the control frames the server sent it: a client that reads, however slowly, is not cut. A peer that has acknowledged nothing (a count of 0: a client with its own TCP stack that never acknowledges data) is refused like any other; where the cap cannot judge (the host's stack does not fill bytes_acked, as gVisor's is believed not to, a failed getsockopt, a development-only fixed-file conn) the period is extended, as without the cap.
• Still not bounded per stream: a response the client does not take, from a client that keeps the connection alive with frames (planned: the write deadline). Bounded only for a client that sends nothing (closed at ReadTimeout, engines: ReadTimeout and IdleTimeout skip an HTTP/2 conn that has a stream open, proposal with the policy decision pending (v1.6.0, Refs #889) #1134).
• Clients see a 408 followed by RST_STREAM(NO_ERROR) (a complete response, then the reset of the stream; the upload must fit ReadTimeout): Go's http2.Transport returns the 408 to the caller (measured), curl reports 408, and clients that treat a reset with NO_ERROR after a complete response as success do likewise. Keep-alive PINGs from clients and proxies (gRPC keepalive, a Transport's ReadIdleTimeout) no longer hold a connection with no open stream past IdleTimeout (default 600 s); with IdleTimeout: -1 such connections are left open as before.
• std is unchanged (it answers 400 where the native engines answer 408 for a request body that timed out).

Findings, not fixed here

1. D2 (the per-stream write deadline, the epoll: the write timeout of a parked response is a minimum rate on acknowledged bytes, no ceiling, inbound bytes are not progress (v1.6.0, Refs #885) #1119 minimum rate) is stage 2; the pinned ping cell for an untaken 1 MiB response stays pinned until then.
2. celeris std answers 400, not 408, for a body read that timed out (above).
3. io_uring: a completed send restamps the conn's clock, so a silent conn whose request was reset closes ReadTimeout after the 408's send, where epoll and adaptive close it with the reset (cells bound it per engine).
4. Native HTTP/1.1 keeps its own body-dribbler hole (one byte per ReadTimeout restarts the clock): out of scope.
5. H2State.Sweep takes H2State.mu with TryLock when something is due; a held lock (the dispatch goroutine of an h2c upgrade, for the instant it runs the first frames) means "next sweep". Never blocks the loop.
6. epoll's conn scan truncates a slow reader that sends nothing (r3 F1, not this PR, same on main): a client with wide windows that reads nothing for 2.5 s and then reads an 8 MiB body gets 2572288 of 8388608 bytes: unexpected EOF on epoll and adaptive on origin/main and on the head alike (io_uring whole): the scan closes with the tail queued once IdleTimeout passes (its clock is the last read). The fix is the same check as D0's (outstanding, or writeSince) in the scan; a follow-up issue. D0 does not add to it (it no longer closes a conn that holds output and is getting it out).
7. Per-conn expiry budget (r4 Q4): a worker-wide budget would spread a burst over many conns; not done.
8. A client that stops reading and sends nothing is closed by the conn scan, as on main (r4: the scan never looks at the send queue); D0's cap is for the one that sends frames.

Unverified

• amd64 (above); adaptive on its io_uring side (every adaptive cell ran on epoll, active=epoll); an end-to-end h2c upgrade (a request that arms its deadline in the frames trailing the upgrade, on the dispatch goroutine; covered by reading and the lock logic: it is under H2State.mu, and the conn joins h2Conns only afterwards); a real client other than the raw framer (x/net/http2.Transport and curl reading the 408, planned for the Opus round).
• Timing margins of the cells under a loaded CI runner (1 s timeouts, holds of 3 to 5 s).

For the reviewer to attack

• M3 completeness: every frame type that can arrive for an expired stream (WINDOW_UPDATE, PRIORITY, RST_STREAM, DATA with padding, trailers with END_STREAM) is ignored without ending the conn; the tombstone is dropped at END_STREAM or 2 x ReadTimeout, then resetIDs (2048 ids) is the fallback; is the 1024-entry cap reachable by an attacker without a rate that the other budgets would stop first?
• D0: d0Check is stamped by the sweep, not by a frame: a stream that opens and ends between two 100 ms looks shows only through the last client stream ID; is there a path (a stream ID that does not move: a refused stream, PRIORITY) where real activity is invisible and a busy conn is closed at IdleTimeout? The conn's own clock still needs to be due for the conn scan to close it; D0 closes on its own (CloseNow).
• Ordering: the D pass runs before the conn scan; epoll flushes the 408 inline (flushH2Conn) so a close in the same sweep does not truncate it; io_uring relies on closeConn sending what was staged. A writeRefused conn?
• Locks: Sweep reads the hints without H2State.mu (atomics), the stream's StartTimeNs under Manager.mu read lock in ExpiredRequests while the arm is on the loop thread (or the upgrade goroutine) under H2State.mu; a -race run is clean.
• A stale hint after a disarm costs one scan per ReadTimeout per conn and no reset (unit tested).
• The 408 goes through the inline writer into the conn's buffer: Content-Length: 0, no date/server header; compare with what a handler's 408 looks like.
• Godoc: ReadTimeout, ReadHeaderTimeout, IdleTimeout (comment only).

[coderabbitai]

Caution

Review failed

The pull request is closed.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 76.63818% with 82 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/engine/iouring/worker.go	32.60%	31 Missing ⚠️
internal/engine/epoll/loop.go	57.50%	17 Missing ⚠️
internal/tcpack/tcpack_linux.go	83.58%	11 Missing ⚠️
internal/protocol/h2/stream/deadline.go	83.60%	10 Missing ⚠️
internal/protocol/h2/stream/manager.go	85.41%	7 Missing ⚠️
internal/conn/h2.go	93.93%	4 Missing ⚠️
internal/engine/epoll/conn.go	0.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!