feat: gsoc grained api - #5614
Merged
Merged
Conversation
Add `Swarm-Soc-Fields` header to allow clients to request specific SOC fields (address, recoveredpubkey, identifier, signature, wrappedaddress, span, payload) in GSOC WebSocket messages. Add `Swarm-Cache-Wrapped-Chunk` header to enable caching of wrapped chunks on the node. Update GSOC handler to pass full SOC object instead of just payload, enabling access to all chunk properties. Adjust WebSocket buffer sizes to accommodate maximum SOC fields message size.
Add tests for `Swarm-Soc-Fields` header to verify requesting specific SOC fields (identifier, wrappedAddress, payload) and full wrapped chunk data (span + payload). Add test for `Swarm-Cache-Wrapped-Chunk` header to verify wrapped chunks are cached and retrievable. Update test helpers to support custom headers and return storer instance. Update gsoc handler signature to accept full SOC object instead of payload bytes.
gsoc.Handle spawned a goroutine per subscriber handler, so message delivery order to a subscriber was not guaranteed. Call handlers synchronously in registration order instead.
nugaon
force-pushed
the
feat/gsoc-grained-api
branch
from
September 15, 2026 08:31
f8f40ac to
8c0ba37
Compare
nugaon
marked this pull request as ready for review
September 15, 2026 14:11
nugaon
requested review from
acud,
aloknerurkar,
martinconic and
sbackend123
September 15, 2026 14:11
martinconic
requested review from
akrem-chabchoub,
gacevicljubisa and
janos
September 17, 2026 10:03
aloknerurkar
previously requested changes
Sep 17, 2026
martinconic
requested changes
Sep 18, 2026
…al parameters - Updated multiple test files to modify the `newTestServer` function calls, adding an extra parameter to accommodate changes in the test server options. - Ensured consistency across various test cases in readiness, redistribution, router, settlements, SOC, staking, status, stewardship, subdomain, tag, topology, tracing, transaction, wallet, and welcome message tests.
martinconic
reviewed
Sep 22, 2026
martinconic
left a comment
martinconic
left a comment
Contributor
There was a problem hiding this comment.
Changes look good, checked them manually also, but discovered with AI some things, please take a look
| go func(hh Handler) { | ||
| hh(c.WrappedChunk().Data()[swarm.SpanSize:]) | ||
| }(*hh) | ||
| (*hh)(c) |
Contributor
There was a problem hiding this comment.
Not sure but should we add this instead, wdyt @janos ?
safe.Run(l.logger, "gsoc-handler", func() {
(*hh)(c)
})
aloknerurkar
left a comment
aloknerurkar
left a comment
Contributor
There was a problem hiding this comment.
Thanks for addressing all the comments. Changes look good from my end. Only additional thing is the comment from @martinconic around the loop and ping starvation. Once that is fixed I will approve it!
martinconic
approved these changes
Sep 23, 2026
aloknerurkar
approved these changes
Sep 23, 2026
| // connection, so it must not be aborted just because a websocket | ||
| // closes mid-write; it is bound to the node's lifetime instead. | ||
| wrapped := c.WrappedChunk() | ||
| safe.Go(s.logger, "gsoc-cache-wrapped-chunk", func() { |
Contributor
There was a problem hiding this comment.
If I'm reading this right, it starts a new goroutine for every incoming message, and I don't see a limit. What would happen if someone floods this address or the disk is slow?
Would it make sense to cap it, maybe like gsocQueue ?
Also, it looks like these goroutines aren't tracked by wsWg. Could they still be writing after the storer is closed, or am I missing something?
Member
Author
There was a problem hiding this comment.
yeah, it is handled by the queue mechanism in the code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
merging #5497 and #5593 PRs into one.
I took out the slow client handling part because the incoming updates frequency are unpredictable.
The feature is still notifying the user in case of piled up messages with warning log.
after the 1st master merge you can see the new code.
The following changes were made by Claude Sonnet 5 while chasing down the CI failures on this branch:
pkg/api/gsoc.go: replaced the fixed-size buffered channel + non-blockingdefault-drop ("slow consumer") logic with an unbounded FIFO queue (gsocQueue). The olddefaultbranch could silently drop a GSOC update on any goroutine-scheduling jitter (not just a genuinely slow client) and, after an earlier refactor, could also permanently hangHandle()— called synchronously by push/pull-sync — since nothing ever unblocked it for a client that never sends a close frame. The queue guarantees in-order delivery with no cap and no dropped messages, while a truly dead connection still times out via the existing per-message write deadline.pkg/api/gsoc.go: movedSubscribe()to run synchronously ingsocWsHandler, before the connection is handed off to its own goroutine, closing a window where a GSOC update could arrive before the handler was registered and be silently missed.pkg/api/gsoc_test.go: updatedTestGsocWebsocketSlowConsumerto assert the new unbounded-queue behavior (the full backlog is delivered, in order, once the consumer catches up, instead of being disconnected), and added a synchronization fix specific to this test's in-memorynet.Pipetransport —Dial()returning is not a happens-before guarantee that the server has reachedSubscribe(), so the test now waits before sending and confirms with a throwaway round-trip before building the unread backlog it asserts on.pkg/api/gsoc_test.go: removed the now-unnecessary fixed sleep between updates inTestGsocWebsocketMessageOrdering— ordering is guaranteed by the fix above rather than by pacing.🤖 Generated with Claude Code