Repository navigation
Conversation
A browser that stops reading its socket (hung, or stopped) blocked the sending thread forever in IO#write, before the command's timeout even started, while holding the lock every other command needs to be sent. The whole client froze: no error, no timeout, threads asleep. Write with write_nonblock and wait up to the command's timeout for the socket to take more data. If none is accepted, close the connection, since the frame was cut off midway, and raise DeadBrowserError; the commands waiting behind it fail the same way.
guigs
force-pushed
the
fix/bound-websocket-write
branch
from
October 12, 2026 13:39
6fd81f8 to
f7043fe
Compare
guigs
marked this pull request as ready for review
October 12, 2026 13:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A browser that stops reading its websocket freezes every thread using the client, with no error and no timeout.
WebSocket#send_messageholds@driver_mutexacross a blocking@sock.write, and the command timeout (pending.value!(timeout)) only starts once that write returns. When the browser stops reading and a frame is bigger than the kernel socket buffers can hold, the sending thread blocks inIO#writeforever. Every other command then blocks on the mutex before it is even sent, so its timeout never starts either.Browser#quithangs the same way, becauseClient#close→WebSocket#closeneeds the same mutex.We hit this in production: a Sidekiq process whose 6 threads share one browser to render PDFs froze for hours. Its threads sat idle, it logged nothing, and its health checks stayed green. It's reproducible by sending a
SIGSTOPto Chrome and then sending a multi-MB command. On my machine the threshold was between 1.3MB and 2.5MB, and it depends on the kernel's socket buffer settings.The write now uses
write_nonblockand waits up to the command's timeout for the socket to take more data. The wait restarts whenever some bytes are accepted, so a slow link that is still moving isn't affected. If the browser takes nothing for that long, the connection is closed, because the frame was cut off midway, andDeadBrowserErroris raised. Commands queued behind it fail the same way through the existing dead-connection path. The handshake, close and pong frames aren't tied to a command and keep the old unbounded write; they are a few bytes each.WebSocket#send_messagetakes the timeout as an optional positional argument, so existing calls likesend_message(id: 1)keep working.