Repository navigation
redis: dial waits out the io_uring number refusal, sentinel moves on from a lost sentinel, pubsub backoff, portable unregister, wake source guard (v1.6.0, Refs #887 #890) - #1129
Merged
Conversation
…bsub backoff, portable unregister during open, wake source guard (Refs #887 #890) Tests only, plus two declared and unused knobs the tests set (sentinelDownGrace, pubsubStableAfter). They compile on main and fail there as assertions, except the wake source guard, which pins code that is already correct and is checked by mutants.
…ves on from a lost sentinel, pubsub backoff survives a hang-up after the replay; eventloop: portable UnregisterConn ignores a registration still opening, wake source guard wording (Refs #887 #890) 887 item 28: eventloop.RegisterConnWait retries RegisterConn on the io_uring engine's 'fd already registered' (bounded 2 s and by ctx); the redis, memcached and postgres dials use it. The retry in the PubSub test rig that hid the failure is removed. 887 item 29: watchFailover gives up a sentinel whose PubSub has been down (or flapping) for sentinelDownGrace; subscribeLoop moves to the next address and closes the replaced sentinel's PubSub. 887 item 30: the reconnect backoff runs over the whole outage and starts over only when a replayed conn has stayed up for pubsubStableAfter. 890 item 89: portable UnregisterConn treats register's placeholder as an unknown number. 890 item 50: comments that said the WAKEATOMIC shape is covered only by the probabilistic oracle now point at the source guard.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
… guard bypass, the wait's error (Refs #887 #890) Tests only, plus one declared and unused error (errRegisterWaitBudget) the test checks. Client.Close from two goroutines and SentinelClient.Close against the give-up race on the shared standalone loop; Close during subscribe leaves a sentinel conn open; the wake guard did not see a copy of the eventfd number that enqueueFlush writes; RegisterConnWait's error did not wrap the ctx or budget cause.
…idempotent, Close during subscribe leaks nothing; eventloop: RegisterConnWait wraps why it stopped, wake guard sees a copy of the eventfd number (Refs #887 #890) Review round 1 of the PR. redisPool.Close runs its body once. The give-up of watchFailover takes the sentinel out of sentinelConns under sentinelMu and closes it only if it was still listed, so SentinelClient.Close and the give-up never close the same Client; subscribeLoop closes a sentinel it connected after Close instead of listing it. RegisterConnWait returns the refusal together with ctx.Err() or the budget, and its godoc says it must not run on the worker it waits for. The wake guard also pins where the eventfd number goes (only wakefd.New, unix.EpollCtl, unix.Close) and that the only raw write is flushLocked's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane L15 DRV-REDIS of v1.6.0. Refs #887, Refs #890. Draft.
Method: the premise of each item was checked on main first (0897add, then 8936e89 after #1127 merged; none of the five changed). Commit 1 holds the tests and compiles on main; commit 2 is the change. Every test below fails on main as an assertion, except the wake source guard (item 890.50), which pins code that is already correct and is checked by mutants.
Item numbers are the position in the issue body counting EVERY checkbox, ticked or not (#887 has 31, #890 has 98). #1099 and #1050 count unchecked boxes only, which gives other numbers (#1050 calls the wake test "item 49"); each row quotes the item's opening words.
celeris/iouring: fd already registered"SentinelClient.watchFailoverwaits onsc.subPS.Channel()"conn.closedcheck"ClusterClient.SubscribeandClusterClient.SSubscribebind their PubSub to the client of one node"TestWriteRacingCloseNeverTouchesTheClosedEventfd862... catches its defect only through the race detector"UnregisterConn(fd)that runs whileRegisterConn(fd, ...)is still opening the net.Conn is lost"Files:
driver/internal/eventloop/{register_wait.go,portable.go}(new helper, portable fix),driver/{redis,memcached,postgres}/conn_unix.go(one line each),driver/redis/{pubsub.go,sentinel.go}, tests. No exported API of a public package changes (driver/internal/...is internal):mage apiChecksaysapi/ matches the exported API of 53 packages;api/driver/redis.txtis untouched.Commits
0e27c41tests (and two declared knobs the tests set, see below).502a4f4the change.5152237review round 1, tests first (one more declared, unused name:errRegisterWaitBudget).4340ecdreview round 1, the change.4a8f934merge of main (f4ae3ad), no conflicts.Review round 1 (Opus, head 502a4f4) found one major and four minors; all are in "Review round 1" below.
Round-0 tested sha: 502a4f4 (rebased on 8936e89). The head after review round 1 is
4a8f934; its runs are listed under "Review round 1". The mutant and-count=20runs below were made at 3680b82, the same two commits before the rebase onto #1127 (which changedinternal/engine/iouring/worker.go; the driver files and the engine's refusal text atinternal/engine/iouring/driver.go:242are unchanged by it). The full driver suites and the one-worker runs were repeated at 502a4f4.887.28: the dial
Reproduction first, on main, the real io_uring engine (a
celeris.ServerwithEngine: IOUring, the PubSub rig ofpubsub_rig_engines_887_linux_test.go; docker--cpus 2 --security-opt seccomp=unconfined, since docker's default seccomp profile blocks io_uring and the rig skips):TestDialAfterTheCloseOfAConnOnTheSameNumberSucceeds887has 4 goroutines close a conn and dial the next at once, 2000 dials per run, over a fake redis (ForceRESP2, so the dial is all that is measured).--ulimit memlock=8388608, a runner's 8 MiB)-race -count=20(2 workers) and in a one-worker runThe standalone loop and the epoll engine were clean on both trees (0 of 2000 in every run; the epoll engine runs
onClosebeforeUnregisterConnreturns). Log:l15drvredis/logs/c1-dial-reuse-redis-count10.log.The change:
eventloop.RegisterConnWait(ctx, loop, fd, onRecv, onClose)isloop.RegisterConnthat, on exactly the io_uring engine's refusal, retries with a pause that grows from 100 us to 5 ms until the number is finalized, ctx is done, or 2 s have passed, and then returns the refusal. The threeconn_unix.godials call it. The 2 s bound is for a close that lingers (SO_LINGER delays that conn'sonClose, celeris#735); it is a guess, not a measurement.eventloop.IsNumberBusymatches the text (celeris/iouring:prefix,fd already registeredsuffix): the engine has no sentinel and I did not touch engine files.TestTheIOUringRefusalIsWhatTheDialWaitsOut887registers one number twice on a real io_uring worker and requires the match, so a reworded engine message turns that test red instead of silently turning the wait off. The Adaptive engine'sdriverWorkerLoop.RegisterConnreturns the inner error unwrapped (internal/adaptive/provider.go:73-76), so the prefix survives there. Any other error (... is already an HTTP connection, a shut-down worker, the epollfd N is already registered, the standalone loop'sErrAlreadyRegistered, which does not clear by itself) is returned at once.WorkerLoopwrapper that refuses the first 3RegisterConncalls with the io_uring text (TestDialWaitsOutAnUnfinalizedNumber887in redis, memcached, postgres; fail on main withRegisterConn was called 1 times), plus two that pass on main and guard the fix against being too wide or unbounded: another error is not retried, a refusal that never clears gives up with the ctx (300 ms ctx: 302 ms, 44 calls).newPubSubAfterReuse, from 887 item 10) is removed, as its comment asked. Honest limit:TestPubSubControlWritesDropsAndCloseInterleave929passes 20 of 20 on all three engines at one io_uring worker both on commit 1 with the retry removed and on commit 2, so it does not discriminate; the regression test for this item is the dial test above.IsNumberBusy(nil)returns at once). No command path is touched; I did not run a benchmark.887.29: the sentinel that went away
On main,
watchFailoverwaits on the sentinel PubSub's channel, which a conn drop does not close, and the PubSub reconnects to the same address for ever. Three tests, each on two sentinels A and B (A listed first), each fails on main withtimed out after 8s waiting for the client to subscribe to the next sentinel after the first went away: A gone for good (listener closed); A accepts every reconnect and hangs up 5 ms later; A hangs up every 200 ms (a conn that is up as often as it is down). After the move B announces a failover and the client must follow it.The change:
PubSub.downFor()(unexported, built on the outage state of item 30 below): zero while the conn is up and has been since its replay stayed up forpubsubStableAfter; during an outage the time since it began while no conn is up or a replay is running; once a conn is up again, the time that outage lasted until that conn came up, so a conn that keeps dying soon after its replay shows a figure that grows and a blip that healed does not.watchFailoverchecks it on a ticker and, pastsentinelDownGrace(5 s), closes that sentinel's PubSub and client and returns "gave up";subscribeLoopthen callsconnectSentinelFrom(index of that sentinel + 1), so a sentinel that accepts and hangs up does not take the subscription back from the next address for ever (mutant SENT-NO-ROTATE fails the two flapping tests).subscribeLoopalso closes the replaced sentinel's PubSub: until now only its client was closed, and a PubSub's conn is checked out of the pool, so its reconnect loop outlived it. The tests count the(*PubSub).reconnectLoopgoroutines of the process before and after (the two changes together; the mutant that drops both fails the gone-for-good and the 5 ms tests).887.30: the reconnect backoff
On main the backoff is reset when the replay returns, so a server that hangs up just after the replay is retried at the base delay for ever.
TestReconnectBacksOffWhenTheServerHangsUpAfterEveryReplay887(a fake that closes each conn 10 ms after SUBSCRIBE): main sees 39 SUBSCRIBEs in 2.5 s, gaps 53 to 80 ms; the change sees 6 (gaps 56, 115, 246, 484, 725 ms).TestReconnectBackoffStartsOverAfterTheConnWasStable887: conns 1 to 5 are cut after 10 ms, conn 6 lives 800 ms (stable window 400 ms in the test): on main the gap before SUBSCRIBE 5 is 62 ms; with the change 357 ms, and the reconnect after the stable conn waits about 47 ms (the base delay).The change: an outage starts at the first drop (or at a drop of a conn that stayed up for
pubsubStableAfter, 5 s = the backoff cap) and the backoff attempt continues across reconnect loops within it;Resetruns when an outage starts, not when a replay returns.TestReconnectLoopCountsAConnThatDiesDuringTheReplayAsFailed887(item 15) still passes unchanged.890.89: the portable worker
TestUnregisterConnWhileRegisterConnIsOpening890is written as linearizability: eitherUnregisterConnreturnedErrUnknownFDand the registration stands, fully live (and its ownUnregisterConnfiresonCloseonce), or it returned nil and the conn is fully torn down (onClose(nil), the net.Conn closed, the table clean). On main it returns nil and the conn stays registered:onClose never fired, 2 s. The fix:UnregisterConntreatsregister's placeholder (conn == nil) as an unknown number and leaves it, which is what the Linux workers answer (theirRegisterConninserts in one step). The alternative named by the issue,registerre-checking that its placeholder is still in the table and closing the opened conn otherwise, would keep a nil return but thenonClosehas to fire for a conn the driver never saw registered. I took the first because it needs no new callback contract; redis and memcachedCloseignoreUnregisterConn's result and postgres'ClosetoleratesErrUnknownFD. TheshutdownConncomment that documented the bug is rewritten.890.50: the source guard, and why it is not the suggested hook test
The issue asks for a hook in
shutdownbetween the eventfd's close and the return ofw.mu, with a wake that "loaded the number before the close" let through.teardown_hooks_890_linux_test.goalready says why that cannot order the WAKEATOMIC shape:FD()returns -1 once the close has run, so a hook after the close never sees a stale number, the race detector reports nothing (an atomic and a syscall), and the window of the mutant is between two statements of its ownwake, where the production code has no hook and, by design, no window.TestWakeGoesThroughSignalOnly890(wake_source_guard_890_test.go, no build tag, parses the package's non-test files, so it runs on darwin and windows too) requires that every use of the worker'swakeFDfield is a method call allowed to its function (newWorkerbuilds it,runcallsFD,wakecallsSignal,shutdowncallsClose), that nothing reads the field bare (a copy or method value escapesWakeFD's lock), that(*worker).wakeis exactly the statementw.wakeFD.Signal(), and that none of the four allowed uses has disappeared (so the scan cannot go empty).Mutants of
loop_linux.go(logs/mutants-wake-guard-c1.log; each restored,RESTORED_OK):wake=unix.Write(w.wakeFD.FD(), ...)wake uses wakeFD.FDefdCachefield written innewWorker,wakewrites itkeep := w.wakeFDinrun)And WAKEATOMIC against the existing tests, 10 runs
-race(logs/wakeatomic-existing-tests.log):TestAWakeAfterShutdownClosedTheEventfdWritesNothing89010 of 10 PASS (does not catch it),TestWriteRacingCloseNeverTouchesTheClosedEventfd8621 of 10 FAIL, the guard 10 of 10 FAIL. The two comments that said the shape was covered only by the probabilistic oracle now point at the guard. The guard is a source pin: whoever changes the wake design must update its allow-list, and its messages say so.Failing on main (commit 1,
l15drvredis/logs/c1-*.log,-race)TestDialAfterTheCloseOfAConnOnTheSameNumberSucceeds887/iouringTestDialWaitsOutAnUnfinalizedNumber887(redis, memcached, postgres)RegisterConn was called 1 timesTestSentinelMovesOn...887(3 tests)TestReconnectBacksOff...887,TestReconnectBackoffStartsOver...887TestUnregisterConnWhileRegisterConnIsOpening890TestWakeGoesThroughSignalOnly890TestDialGivesUp...,TestDialDoesNotRetryOtherRegisterErrors887,TestIsNumberBusy887/TestRegisterConnWait887(eventloop, commit 2)Commit 1 declares two package variables with no production reader,
sentinelDownGraceandpubsubStableAfter, so the tests can shorten them and still compile on main (the precedent is #1050's commit d07fd64, which declared hooks nothing called).Mutants of the change (
logs/mutants-fix.log; each red test named; restored after)TestDialDoesNotRetryOtherRegisterErrors887(redis, memcached),TestIsNumberBusy887TestDialGivesUpWhenTheNumberStaysRegistered887, helper testTestTheIOUringRefusalIsWhatTheDialWaitsOut887,TestIsNumberBusy887...StartsOverAfterTheConnWasStable887TestUnregisterConnWhileRegisterConnIsOpening890Local results (docker golang:1.27.3,
--cpus 2, one Go process at a time; a redis:7-alpine, postgres:17-alpine, memcached:1.6-alpine container each,--cpus 1, on a private network, removed after)go test -race -count=1 ./driver/...andgo test -count=1 ./driver/...(without-race, so the!racealloc guards run): all five packages ok.TESTING_STRICT_ALLOC_BUDGETS=1 go test -run TestAllocBudgets ./driver/redis ./driver/memcached: ok.-race(tip before the rebase):-tags redis ./test/conformance/redis(CI's-skipfor the three password tests) 48 PASS 0 SKIP;-tags postgres48 PASS 0 SKIP;-tags memcached24 PASS 0 SKIP. Not run: the other server versions, valkey, sentinel and cluster jobs against real servers (there is no such test in the tree for item 29; the fakes are the fixtures), Windows and darwin runtime (GOOS=darwin|windows go vet ./driver/...clean).-race -count=20: the backoff, dial and pin tests (redis package, 116 s), the portable test (eventloop), and the three sentinel tests (151 s): ok.--ulimit memlock=8388608, the rig logs1 workers): dial tests and pin test pass; 929 test 20 of 20 on all three engines.golangci-lint v2.14 run ./driver/...: 0 issues.mage apiCheck: ok.driver/redispackage under-race(18.7 s to 24.8 s). The new io_uring subtests andTestTheIOUringRefusalIsWhatTheDialWaitsOut887skip on a host without io_uring like the existing rig tests (the CI Unit job's SKIP tally and its alone-rerun apply as they do to those); the other new tests need no external service.Release notes
celeris/iouring: fd already registeredwhen the new socket takes the number of a conn that was just closed; the dial waits (up to 2 s, or its ctx) for the old conn to be finalized.SentinelClientgives up a sentinel whose subscription has been down or flapping for 5 s and subscribes to the next address ofSentinelAddrs; the replaced sentinel's PubSub is closed (its reconnect goroutine used to outlive it). After a give-up the next sentinel is tried first, so the preference for the first address returns only when the next one fails.UnregisterConnanswersErrUnknownFDfor a number whoseRegisterConnis still opening the connection, instead of losing the registration.Overlap with the follow-ups of #1117 (L11, not fixed here)
None of the four touches a file this PR changes: F1 (
releaseOnExit/pollSyncMulti) is inloop_linux.goandconn.go, F2 in theexecDirecthelpers, F3 in the memcached/redis bridge users, F4 (releaseCmdnot deferred) inclient.go,commands.go,pool.go. This PR addsregister_wait.go, one line in eachconn_unix.go, and editsportable.go,pubsub.go,sentinel.go; it edits two comments (not code) in eventloop tests. Item 887.28 is the issue's own sibling of F4 in spirit (pool churn) but shares no code.Findings, not fixed here
+switch-mastermessages; it does not askSENTINEL get-master-addr-by-nameagain, and the master is swapped only if it differs. Not a regression: on main the client stayed on the dead sentinel for ever. Fix shape (reviewer): after a reconnect that follows a give-up, runSENTINEL get-master-addr-by-nameand swap the master client if the address differs.errors.Isonce an engine lane addsengine.ErrFDBusy(or similar) tointernal/engine/iouring/driver.go:242. Not done: engine files.workerHint = fd % NumWorkersputs the new socket on the stale worker); the wait is needed for one worker anyway, so this is not done.F_DUPFD_CLOEXECfromfd+1,os.NewFile, close the original, register the duplicate). It works on one worker and on any thread, including the engine worker (see the review-round item 2 below), but changes which descriptor the driver owns and needs its own review. Not done. The engine-side option the issue names (the io_uringRegisterConnre-keys a closing conn) is an engine change.driver/postgresctxhandling ofdoStartupwaited about 30 s after a 1.5 s ctx expired while the server stayed silent (seen while writing the postgres dial test, which now uses a server that hangs up). Not investigated.Decision pending: 887.31 (cluster PubSub, maintainer decision D3). Nothing is implemented.
The issue text offers: (a) have the cluster constructors re-route the PubSub on repeated reconnect failure (re-resolve the owner of the slot) and/or (b) expose a health signal on
PubSub(aDown()channel or a counter of failed reconnects).ClusterClient.Subscribe/SSubscribegive the PubSub a private re-route hook; when it fires, the PubSub re-resolves (any live node forSubscribe, the slot owner forSSubscribe), swapsps.clientunderctlMu, and replays.api/driver/redis.txt: no change. Messages in the swap gap are lost (already documented as at-most-once).*PubSub, for examplemethod (*PubSub) Down() <-chan struct{}ormethod (*PubSub) Reconnects() uint64, as one added line inapi/driver/redis.txt. Permanent API for v1.x, and it leaves the re-routing to every caller.What A can and cannot detect (checked, review round 1 m4). This PR's unexported
PubSub.downForis a trigger for a node that is gone: a dropped conn, or a conn that keeps dying. It cannot see a slot move, because a slot move does not drop the conn. I ran it on redis:7-alpine, two cluster nodes,--cpus 1, on a private network, removed afterwards: anSSUBSCRIBE fooclient (slot 12182) on node A, then the slot moved to B (CLUSTER SETSLOT ... IMPORTINGon B,MIGRATINGon A,NODEon both). The client receivedsunsubscribe / foo / 0and stayed connected to A (CLIENT LISTshows it,ssub=0). The PubSub's push handling takessunsubscribeas the acknowledgement of its ownSUnsubscribeand ignores it (state.go:656,676), whileps.shardSubsstill lists the channel, so today the subscription is silently dead after a slot move and a reconnect would replay theSSUBSCRIBEto the same node. So A needs two triggers:downForfor a gone node, and a server-initiatedsunsubscribe(one the PubSub did not ask for) of a channel still inshardSubsfor a slot move, after which it re-resolves the owner and re-sendsSSUBSCRIBEthere. A is therefore not just a re-routing of the existing reconnect: the second trigger needs the push path to tell its own unsubscribe from the server's. The plainSubscribe(SUBSCRIBE is cluster-wide) only needs the first trigger.Recommendation, unchanged but now stated with its cost: A, B only if a user asks. A needs no exported surface and removes the silent dead subscription for both a gone node and a slot move; B would freeze a signal shape before anyone used the unexported one. The residual case, the whole cluster down, retries with backoff and nothing a caller could do differently is lost. The risk of A is in
SSubscribe: the slot-map refresh and the unsolicited-sunsubscribedetection.Review round 1 (Opus, head 502a4f4: REQUEST-CHANGES, one major)
Verified good by the reviewer: 887.28 off the worker (main 122, 209, 212 of 2000 refused, head 0 of 2000), 887.30 (at most 1 reconnect loop across 645 forced reconnects), 890.89 (no new leak), the 11 mutants,
-race ./driver/..., windows/darwin vet, the 11 required checks.M1 (major), fixed. A new data race (887.29):
watchFailoverclosed the given-up sentinel's client outsidesentinelMuwhile the sentinel was still listed, so a user'sSentinelClient.Closein that window closed the same*Client;redisPool.Closereads and writesp.providerwith no lock and callseventloop.Releaseon the shared standalone loop, so two closers could drop a reference other clients hold. Three changes:redisPool.Closeruns its body once (sync.Once); a second concurrent caller returns at once.sentinelConnsundersentinelMu(takeSentinel) and closes it only if it was still listed; whoever takes a sentinel out of the list, or declines to list it, closes it, soCloseand the give-up never close the same client.subscribeLooplooked at no closed flag before it listed the sentinel it had just connected; undersentinelMuit now closes that sentinel instead of listing it whenClosehas run.Tests (commit 3,
driver/redis/sentinel_close_race_887_linux_test.go), each in separate-raceprocesses, 8 runs per test (logs/r1-red-on-502a4f4.log,logs/r1-green-on-fix.log):TestClientCloseTwiceConcurrently887(two goroutinesClient.Close, a bystander PubSub on the shared loop)TestSentinelCloseVsGiveUp887(the reviewer's probe: both sentinels hang up every 2 ms, grace 20 ms, 40 x New/Close)TestSentinelCloseDuringSubscribeLeaksNothing887(Close at a random 0-400 us, 300 iterations)Mutants (
logs/mutants-r1.log):Closenot idempotent:TestClientCloseTwiceConcurrently887red 3 of 3. The closed check before the list append removed:...LeaksNothing887red 3 of 3. The give-up closing without owning (takeSentinelbypassed) with theOncekept:TestSentinelCloseVsGiveUp887stays green 6 of 6, because theOncemakes the second close harmless: the two layers overlap, and the test is red only on the original 502a4f4 where neither exists.Item 2 (minor, 887.28): the dial wait on the worker it waits for. A dial that runs on the io_uring worker that must finalize the number sleeps on itself (the reviewer: 2.0 to 6.0 s frozen, then the same refusal; main failed in milliseconds). Documented in the godoc of
RegisterConnWait; not guarded, because the engine gives no cheap, reliable way to know the caller is on its worker. Reachability: withWithEngineon io_uring, epoll or adaptive all three drivers dial their command connections as direct net.Conns and never callRegisterConn(redisclient.go:62-67, postgrespool.go:226-244, memcachedclient.go:52,67). The only public path left is redisClient.Subscribecalled from a handler (the PubSub pool registers with the engine), and only withForceRESP2and no password or DB; with the default config that call already blocks on main (the HELLO handshake waits on the worker it occupies).F_DUPFDis the alternative, listed under Findings. The same commit wraps the returned error (nit n1): the refusal together withctx.Err()or the budget (errors.Issees both; tests-first in commit 3).Item 3 (minor, 890.50): guard bypass, fixed. The reviewer's mutant (a copy of the eventfd number kept in a new field by
newWorkerand written byenqueueFlush) leftwakeFDuntouched and passed. The guard now also requires thatefdinnewWorkerappears only as an argument ofwakefd.New,unix.EpollCtl(including theint32(efd)of its event) orunix.Close, and that the only rawunix|syscall .Write*/.Pwrite*call in the package isflushLocked's onc.fd. Mutants (logs/mutants-wake-guard-r1.log, 6 of 6 red): WAKEATOMIC, EXTRA-STATEMENT-IN-WAKE, COPY-OF-THE-FIELD, WAKE-STALE-CACHED-NUMBER, the reviewer's BYPASS (red withnewWorker uses the eventfd number efd outside ...andenqueueFlush calls unix.Write(w.efdNum, ...)), and NUMBER-STORED-IN-ANOTHER-FIELD. Known limit, in the test's comment: it pins syntactic shapes; a number reaching a producer by another route (a global, an integer returned by a helper) is not seen, and the probabilistic oracle of the 862 test is the backstop.Item 4 (minor): the 887.31 paragraph, corrected above after running Redis 7 (a slot move sends
sunsubscribeand keeps the conn).Item 5: a failover missed between sentinels, listed under Findings with the reviewer's fix shape.
Runs on the new head (
4a8f934for the full run, the tests at4340ecd; main merged is f4ae3ad): the new and changed tests-race -count=10ok (redis 63 s, eventloop 21 s, the three sentinel move-on tests 76 s); the timing tests (TestSentinelMovesOn,TestReconnectBack,TestSentinelClose*,TestClientCloseTwice,TestDialAfter) at--cpus 1-race -count=3ok (59 s);go test -race -count=1 ./driver/...ok on4340ecdand on4a8f934;GOOS=darwinandGOOS=windows go vet ./driver/...clean; golangci-lint v2.14 0 issues;mage apiCheckok.Release-note addition (reviewer nit n2): a PubSub blip within 5 s of recovering from a long outage now waits up to the 5 s cap before reconnecting instead of 50 ms, because the backoff runs over the whole outage.
For the reviewer to attack
IsNumberBusyis a string match on an engine message; the pin test is the only protection. Does the Adaptive engine ever wrap (I readprovider.go:73-76: no)?RegisterConnWaitholds the dialing goroutine up to 2 s underPool.Acquire; is 2 s right, and should it be aConfigknob (not added: public API)?downFor's blip-versus-flap semantics (is "time the outage lasted until the conn came up" the right figure for a conn that is up but unstable?); the sentinel's own PubSub is now closed on replacement.ps.backoff.Attempts()is no longer reset on success; tests that replaceps.backoffstill pass. CanonConnDrop's episode decision and the loop'sdeferre-check ever start two loops, or start a loop whoseattemptwas written by a loop that has not yet returned?UnregisterConnon a placeholder returnsErrUnknownFD: a caller that unregisters its own number while its ownRegisterConnof it is still opening gets "unknown" and the registration stands. Is that the contract you want, or the deferred-teardown alternative?loop_linux.gothan the property (a legitimate refactor ofrunthat moves theFD()call to a helper must update the allow-list).-raceruns at--cpus 2; I did not run them at--cpus 1or under a loaded CI runner.