Repository navigation
relay: every RPC pays a TCP dial, an HTTP Upgrade and (behind an edge) a TLS handshake #195
Description
Problem
The data plane opens one relayed connection per RPC: the SDK dials the owner node, sends GET /v1/sandboxes/{id}/agent with Upgrade: silkd, waits for the 101, and sandboxd dials the guest's vsock port for that one request. silkd serves one RPC per connection and closes after the terminal frame, so nothing can be reused.
Per RPC that is two round trips of setup before the first frame (TCP connect, Upgrade request/response) plus the guest-side vsock connect. Behind an HTTPS edge (#194) a TLS handshake is added on top. Measured on loopback, Apple M2 Max, Go SDK:
| per relay dial | |
|---|---|
| plain TCP dial | 0.12 ms |
| TCP + fresh TLS 1.3 handshake | 1.58 ms, 121 KB, 775 allocs |
| TCP + resumed TLS handshake (client session cache) | 0.67 ms |
On a real network the two setup round trips dominate: a 0.2 ms RTT LAN puts 0.4-0.6 ms of fixed cost in front of every exec, read_file, stat and friends, an order of magnitude above the per-exec savings the silkd fan-out work (#175) chased. Agents doing many small file operations pay it on every call.
What would remove it
A persistent, multiplexed relay: one upgraded connection per sandbox (or per client/node pair) carrying many RPCs as independent streams. That is a wire change on every hop:
protocol/wire: a stream id on request and response frames, or a framing layer beneath the newline-JSON frames; new fixtures inprotocol/wire/fixtures/v1.- silkd (Rust): demultiplex streams on one connection instead of one RPC per connection; per-stream cancellation when a stream closes.
- sandboxd relay: keep the client and guest legs open across RPCs; idle accounting must keep counting an open relay as data-plane activity, or the idle-hibernate policy needs a stream-level notion of "in flight".
- both SDKs: a per-sandbox connection with stream allocation, plus the long-lived streams (
dial_port,watch,open_pty, LSP) riding on it or keeping their own connection.
What does not remove it
An SDK-only pre-dial (open the next upgraded connection in the background while the current RPC runs) hides the latency without changing the wire, but an idle pre-dialed relay is an open data-plane connection, so idle_hibernate_seconds would never fire for a client that keeps one warm. It would need a bounded policy (drop the spare after a short idle window) and still costs a vsock connect per RPC on the node.
The client-side TLS session cache and a larger control-plane idle pool land separately; they take the HTTPS handshake from 1.6 ms to 0.7 ms and stop a single client from re-dialing on every claim burst, but the two setup round trips per RPC stay.
Scope
Cross-repo: silkd, sandboxd, protocol/wire, sdk/go, sdk/python. Not a follow-up to #194; a design issue in its own right.
Activity
CMGS commentedon Sep 16, 2026
Step one, without a wire change: the guest keeps the connection and the SDKs reuse it.
inforeports proto 2.wire.IsContinuation; the relay docs say an open relay holds the idle clock.WithKeepAlive),MSG_PEEKliveness probe before reuse, pipelinedinfoproto probe with fallback to one dial per call, rpcbench mode C.ConnPool,Client(keep_alive=...),selectprobe, oneinforound trip per handle).Kept connections hold the sandbox's idle-hibernate clock, which is why the SDK closes them after the idle window; the numbers of record come from
rpcbenchA vs C on the testbed once the image is rebaked from #196. #34 stays open for the mux and binary bulk frames.silkd: serve RPCs back to back on one connection (#196)
sdk/go: keep a handle's relay connection across calls (#198)
sdk/python: keep a handle's relay connection across calls (#199)