[FumingPower3925]

Lane L19 H2-TIMEOUTS, stage 2 of the per-stream deadlines: D2, the write deadline of an HTTP/2 stream on epoll, io_uring and adaptive (std unchanged). Refs #889. Built on the merged stage 1 (#1150) per design v3.1 (lane evidence: design-per-stream-deadlines.md). Draft.

What it does

A response whose bytes wait for the client's flow-control window (the tail of an inline handler's body, or a pool handler's, buffered in the stream; Manager.outboundHeld) had no bound at all (#1134 Finding 1): a client that granted no window and kept pinging held the stream, and its buffer, for ever (measured: 6 s, 60 s, no reset). Now WriteTimeout bounds it as a minimum rate, #1119's rule at the h2 layer, refined by review r7 so that it does not cut a reader that is slow but moving (final rule and its bounds: design v3.3):

rule
window	a stream is stalled from the moment its buffer goes from empty to non-empty; its window is WriteTimeout long
floor	at the end of a window the stream must have released (DATA the windows let out, FlushOutbound) at least F = min(64 KiB, what it held when the window began); if so the next window begins from that instant; if not it is reset with RST_STREAM(INTERNAL_ERROR) (net/http's code at a write deadline, and #906's). 64 KiB per window is #1119's parkedMinAcked: about 64 KiB/s at a 1 s WriteTimeout, about 1 KiB/s at the 60 s default
carry-over (r7 F1)	what a stream released beyond F carries to the next windows, up to 4 floors (256 KiB), and a stream's first window is covered by the bytes it had in flight when it stalled (the peer's initial window less what is left of it), capped the same way. A client that sends few WINDOW_UPDATEs (nghttp2/curl: at half the window) releases in big steps with empty windows between: it survives while it releases on average F per window and its updates are no further than about five windows apart
no ceiling	a stream that keeps that average is never cut, however long the response (std's deadline is absolute)
bytes, not frames	a PING, a SETTINGS, a WINDOW_UPDATE that lets no byte out, a WINDOW_UPDATE of one byte are not progress (a dribbler on a 64 KiB window is reset within two windows: the seed covers the first)
the connection's rate (r7 F6)	when the connection as a whole released at least its floor in the window, a stream below F is held if it took half its fair share (F / 2N, N the streams stalled), and for 3 consecutive windows if it took less (paused, dribbled, or queued behind its siblings' bytes in the same socket); the fourth is the verdict. When the connection itself released less than its floor, the first window is the verdict, as for a single stream. So N downloads sharing a slow link are held, and a stream the client does not read pins its buffer for at most 4 x WriteTimeout
the connection window	a stream whose own send window has room but whose bytes wait for the connection window is credited with what the whole conn released in the window (held while the conn moves, with no idle bound: waiting for the conn is not its fault). A stream whose own window is shut is judged by its own releases and the rules above, even when the conn window is shut too
A2 retired	for WriteTimeout > 0 a stalled stream no longer takes the conn out of the conn scan's exemption (A2, #1134: the conn of a silent client was closed at ReadTimeout without a word): the stream is bounded by WriteTimeout, reset, and then the conn is stream-less and its clocks apply (as M2 did for A1). A silent client's unread response, and the buffer it pins, therefore lives for WriteTimeout, not ReadTimeout (the conn used to be closed at ReadTimeout and take the buffer with it). A pool handler blocked in AwaitSendWindow stays counted (its bound is #906's)
#906 unchanged	a pool handler blocked in AwaitSendWindow keeps its absolute WriteTimeout from the start of the wait; a stream both would reset gets one RST_STREAM (the second finds it cancelled or gone)

State: a small per-conn map on the Manager (stalls, guarded by stallMu, made on the first stall), a record {window start, held, pending, released, conn counters} per stream that holds buffered bytes, and the fourth sweep hint DeadlineHints.WrDL. Nothing in Stream (it has 120 free bytes: that is not the reason; the reason is that only a stalled stream, a rare case, needs the state, and the conn credit needs conn-wide state anyway; the per-stream-field alternative is in the design note). Every path that ends a stalled stream ends its record (7 sites, and the sweep's net for a stream gone from the map); the leak test pins each. The common send costs one atomic load of stallN in ReserveSendWindow (to count the conn's releases while any stream is stalled).

The RST_STREAM is queued behind the frames the stream already queued (through the conn writer's Resetter, as resetHandlerStream does): written straight to the connection it could overtake DATA still in the write queue. The bytes the stream held are dropped and refunded to the outbound budget; the stream leaves the conn as a reset from the peer does (a pool handler still running on it retires it); its identifier goes in the reset window, so the client's late WINDOW_UPDATE, RST_STREAM and PRIORITY are ignored (cell).

Review r7 (Opus, REQUEST-CHANGES: two MAJOR, three MINOR) and what changed

Tests first (h2_write_rate_889_linux_test.go), the rule after.

• F6, parallel downloads on a slow link (new against main at any ReadTimeout): unless the conn window was shut each stream was judged on its own bytes, so N streams sharing a slow socket (link/N each, below the floor, the conn far above it) were reset: 2 streams on 100 KB/s both at 2.3 s, 4 on 200 KB/s all four; main completes all. Red on 46947d4: 6 of 6 (3 engines x 2 shapes); green now: all complete at 5.25 s. Complete rule: the connection's rate and the share/idle bound above. Whether the scheduler can starve a stream of a healthy client: a stalled stream is flushed only by its own WINDOW_UPDATE (or the conn window opening, for the starved), so who gets bytes is the client's choice and the order the socket delivers them; the server has nothing to make fairer. A stream queued behind its siblings' bytes sees no release for N x W / link (4 streams of 64 KiB on 200 KB/s: 1.3 s), which the 3 idle windows absorb. I chose the bounded grace (K = 3) over credit for bytes in flight (cannot tell a paused stream from a queued one: unbounded shelter) and over a per-connection judgement for all (same shelter). A paused or dribbled stream among healthy siblings: red on the previous head 6 of 6 (reset at 1.2 s), now reset at 4.3 s (paused) and 5.3 s (dribbled; the seed covers its first window).
• F1, few WINDOW_UPDATEs: progress counted only when an update arrived and nothing carried over. Real curl 8.14.1 (nghttp2 1.64.0) at 300 KB/s, 16 MiB, WriteTimeout 1 s: reset after 7.9 s at 2.5 MB before (curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR), complete in 54.5 s now (the review measured main at 54.5 s too); 8 MiB at 1m: complete both. Raw-framer cells (CI-able): a client that updates at half a 1 MiB window reading at 300 KB/s (4.6x the floor): reset at 1.0 s before, complete now; half of 6 MiB at 1 MB/s (gap 3 s): reset at 1.0 s, complete; Go-Transport-like (4 KiB updates): green both. Red on 46947d4: 6 of 9 (the 3 guards green). The fix is the bounded carry-over (4 floors) and the seed; the exact guarantee is in the table. When ReadTimeout <= WriteTimeout (including the 60 s defaults) main closed the whole conn at the same gap, so the new cut was where WriteTimeout < ReadTimeout or ReadTimeout is off.
• F2: the verdict is for the record phase 1 found due (a record ended and re-made while the windows were read was expired ~2 us into its window); nothing is reset whose buffer went out whole between the verdict and the reset. Unit tests (the reviewer's interleaving).
• F3: WriteTimeout near math.MaxInt64 overflowed armedAt + wt: every stalled stream had to release 64 KiB per sweep (Go's Transport at 500 KB/s got INTERNAL_ERROR after 8.4 s). Timeouts are clamped to 100 years in SetWriteTimeout, SetRequestTimeout and SetHeaderBlockTimeout (stage 1's request deadline had the same overflow); tests for all three.
• F4: "the renewal keeps the first window's held" turned out to be an equivalent mutant at the verdict level (the connection's floor, connHeld, carries the same bytes; a unit test pins the intended behaviour), and the own-window-shut shelter, killed only by a unit test before, is now also a cell: a paused stream on a healthy conn is reset in the fourth window.
• F5: the window clock is the wall clock (UnixNano, the engines' sweep now, as D1's): a forward step judges every stalled stream at once. Not changed: the sweep's now is a wall-clock value; documented.
• Docs: the WriteTimeout godoc says the guarantee exactly and drops "never cut" without its conditions; the detached-stream paragraph no longer says "no minimum rate" for an h2 stream; the release note says a silent client's buffer lives for WriteTimeout.

Commits (tests first)

3883c56 tests, red on main	33 cells on 3 engines: 24 red, 9 green guards
a4d068b the change	write_deadline.go, Manager, FlushOutbound, ReserveSendWindow, H2State.Sweep, 7 removal sites; unit tests
the pinned cell	#1134's "a pinging client holds an unread 1 MiB response" is flipped: red on main 3 of 3
r7 tests (red on 46947d4)	18 cells: sparse updates 6 red (3 guards green), shared link 6 red, paused/dribbled stream 6 red
r7 fix	carry-over and seed, connection rate, share and idle bound, re-check, clamp, docs

Evidence

• Unit (internal/protocol/h2/stream, internal/conn): the floor and windows, the carry-over and its cap, the average-rate guarantee (floor on average held for ever, half of it reset once the bank is spent), the seed, the dribbler within two windows, the idle stream on a healthy conn (3 held, 4th reset; its share holds it; a slow conn is the verdict at once), the conn credit, a record ends with every removal path (9 paths), one RST, the re-check, nothing held means no reset, the three clamps, the common path free, a direct send makes no record, frames-to-progress, the sweep end to end.
• -race -count=5: internal/conn, internal/tcpack, internal/protocol/... ok; the new and changed cells ok (752 s); internal/engine/epoll (381 s) and internal/engine/iouring (492 s) under -race ok.
• The whole test/server H2|889|Shutdown|SSE|Stream set, non-race: 1674 PASS, 0 FAIL, and again on the head merged with main after epoll, io_uring: every HTTP/1 response after Shutdown carries Connection: close and ends its conn (v1.6.0, Refs #885 #884) #1169 (Connection: close after Shutdown) with the 1165|884 cells added: 1704 PASS, 0 FAIL.
• CI: the first push of the r7 cells made the root package's rest step (-race, -timeout=540s) time out; the 9 slow cells are listed in .github/root-test-shards.txt (shards a and b, root-shards.sh check ok) so that the rest stays inside its timeout. CI at the merged head 56ca76a: 29 pass, 2 skipping.
• golangci-lint v2.14.0: 0 issues (also -tags netem889); GOOS=windows and darwin go vet ok; apidump ok; size tests pass (connState 448, H2State 648 bytes = the 704 malloc class, Stream unchanged).
• CI evidence: every cell runs in the ordinary test/server package, no privileges, so CI runs them all. The curl numbers above are a one-off measurement (docker, curl 8.14.1/nghttp2 1.64.0), not a cell: a CI cell that skips without curl would trip the skip rule.

Mutants (each applied, the named tests run, file restored; logs/mutant-d2-*.log)

mutant	killed by
no expiry	unit, 12 cells
any byte is the floor	unit, 9 cells
floor off by one	unit x3
the window never restarts	unit x3, 6 cells
a shut own window gets the conn's credit	unit
no conn credit for a starved stream	unit, 2 cells
any release counts as a full floor	unit, 3 cells
a direct send arms a record	unit (PoolDirect)
A2 not retired	unit, 3 cells
CANCEL for INTERNAL_ERROR	unit x2, 6 cells
no RST queued	unit x3
a second reset; the identifier not remembered	unit each
the hint never lowered	unit x3, 6 cells
no conn count in ReserveSendWindow	unit, 3 cells
no carry-over	unit x3, 5 cells (sparse)
the bank uncapped	unit
no seed	unit x3, 6 cells
idle windows unbounded (a paused stream sheltered for ever)	unit, 6 cells
no idle grace (a queued stream reset at once)	unit, 3 cells (shared link)
any release is a share (a dribbler on a healthy conn)	unit, 3 cells
a slow conn is held (the verdict not at once)	unit
the record not re-checked	unit
reset with nothing held	unit
timeouts not clamped	unit x3
a record outlives its stream, 7 ways	the leak test, one path each
equivalent: the renewal keeps the first window's held	survives: the connection's floor (connHeld) carries the same bytes, so the verdict is the same; the unit test pins the intended behaviour

Hot path and cost

Docker arm64 --cpus 2 (a shared machine; amd64 unmeasured), main (3f8d83f) vs head, interleaved, 5 rounds x 2 runs per engine, medians (minimums): the sweep is where D2 adds a per-conn cost (one more atomic, hints 8 bytes longer):

benchmark (nothing due)	main	head
sweep, epoll, 4096 h2 conns	32.81 us (30.61)	34.81 us (33.82)	+6.1% (+10.5%): about +0.5 ns per conn
sweep, epoll, 256	1520 ns (1404)	1646 (1449)	+8.3% (+3.2%)
sweep, io_uring, 4096	45.2 us (43.3)	47.6 us (46.2)	+5.3% (+6.9%)
sweep, io_uring, 256	2402 ns (2310)	2486 (2375)	+3.5% (+2.8%)

(The review measured +6.6 to +7.9% at 4096 on epoll: the same order.) At 10 000 h2 conns and 40 sweeps a second the addition is about 0.2 ms of a second. The conn and stream benchmarks (ProcessH2Post889, StreamedResponse889, OneByteGrant911) are within the noise of the shared machine, allocations unchanged (previous round: ProcessH2Post889/single 673 vs 668 ns, StreamedResponse889/closed 559 vs 548 ns; the review: +2 to 3% medians, minimums equal or lower). The common send pays one atomic load of stallN in ReserveSendWindow. A stalled stream's flush takes the books' lock once: BenchmarkOneByteGrantWriteDeadline889 on 61 to 73 ns, off 53 to 122 ns (noise-bound). The window verdict allocates when more than 32 streams of a conn are due at once.

Release notes (draft)

• HTTP/2 on epoll, io_uring and adaptive: WriteTimeout now bounds a response that waits for the client's flow-control window, as a minimum rate, the rule it already applies to a response parked on the socket: in each WriteTimeout window the client must let at least 64 KiB of the stream's waiting bytes out (all of them, if fewer are waiting), or the stream is reset with RST_STREAM(INTERNAL_ERROR), as net/http resets a stream at its write deadline. The guarantee: a stream that releases on average at least that per window, lagging that rate by no more than four windows' worth (256 KiB; the bytes in flight when it stalls count for its first window), is never cut, however long its response takes (unlike std, whose deadline is absolute); so a client that updates in big steps (curl/nghttp2: half the window) survives while its updates are no further than about five windows apart. Parallel downloads that share a slow socket are held while the connection moves at least that much and each takes half its fair share; a stream that takes less is reset after four windows. A client that PINGs, or sends WINDOW_UPDATEs of a few bytes, no longer holds a stream (and the memory of its response) for ever.
• A connection with such a stream is no longer closed by ReadTimeout or IdleTimeout while the stream lives: the stream's WriteTimeout bounds it, and the connection's clocks apply once it is gone. A silent client's unread response, and the buffer it pins (up to the 4 MiB outbound budget plus one body per connection), used to be dropped with the whole connection at ReadTimeout and now lives for WriteTimeout (the stream gets a reset, the connection is closed after it). With WriteTimeout: -1 nothing changes.
• A WriteTimeout or ReadTimeout of math.MaxInt64 no longer makes every deadline due at once (timeouts are clamped to 100 years).
• A handler that is blocked waiting for window keeps its own absolute bound of WriteTimeout from the start of the wait (unchanged).
• std is unchanged.

Findings, not fixed here

1. A pool handler blocked in AwaitSendWindow is bounded by fix(h2): bound the response bytes a connection holds for the peer's window; over the budget, handlers run on the pool and send as the windows open (celeris#893) #906's absolute WriteTimeout, a buffered tail by this rate: the two policies differ. Aligning them is a follow-up (design v3.7 Q1).
2. The windows run on the wall clock (the engines' sweep now): a forward clock step judges every stalled stream at once, as D1's does (r7 F5).
3. ExpiredWrites allocates when more than 32 streams of one conn are due at once. The per-expiry budget (2 ms per conn) is the stage 1 one.
4. The share/idle rule can hold a dribbler for 4 windows on a healthy conn, and a stream that releases its share every window for as long as the conn moves; an attacker pays the floor in bytes the socket carries for each conn it uses, and the buffer stays under the outbound budget.

Unverified

• amd64 (every number is arm64); adaptive on its io_uring side (its cells ran on epoll); the review ran its F1/F6 probes on epoll only (the rule is in internal/protocol/h2/stream, engine-independent), and the new cells ran here on all three engines.
• Browsers' update policies (Chromium flushes small updates on a timer, Firefox acks in large steps: from memory, not measured); the cells' timing on a loaded CI runner (1 s windows; the paused-stream cell expects a reset in [3.5 s, 7 s]).

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 95.38462% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/protocol/h2/stream/write_deadline.go	95.15%	8 Missing ⚠️
internal/protocol/h2/stream/manager.go	87.50%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!