Repository navigation
Conversation
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
October 26, 2026 00:14
b298e44 to
90966d6
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
2 times, most recently
from
October 26, 2026 02:08
326073b to
d018d29
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
2 times, most recently
from
October 26, 2026 08:44
1c5e48f to
84f1587
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
2 times, most recently
from
October 27, 2026 09:07
b5bb616 to
f5a8d77
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
2 times, most recently
from
October 27, 2026 09:17
c3f30a4 to
b9572b1
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
October 27, 2026 09:17
f5a8d77 to
6d49da8
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
2 times, most recently
from
October 28, 2026 07:54
925cf34 to
cc8e51c
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
October 28, 2026 07:54
026e0a3 to
11cd676
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
October 28, 2026 08:05
cc8e51c to
e7710c0
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
October 28, 2026 08:05
11cd676 to
c68482a
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
November 5, 2026 17:19
effd2e0 to
b445a38
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
November 5, 2026 17:19
eb66e79 to
a912bfb
Compare
This was referenced Nov 5, 2026
Open
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
November 5, 2026 23:10
a912bfb to
1c9ed91
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
November 5, 2026 23:10
b445a38 to
27b6d1c
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
November 6, 2026 09:06
1c9ed91 to
d57a2ae
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
November 6, 2026 09:06
27b6d1c to
06f4615
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
November 6, 2026 09:22
d57a2ae to
aef29b9
Compare
moetemp
force-pushed
the
moe/brook/review/s6-redis-memory
branch
from
November 6, 2026 09:22
06f4615 to
84bc88f
Compare
moetemp
force-pushed
the
moe/brook/review/s7-workflow-streams
branch
from
November 6, 2026 10:11
aef29b9 to
ee2e261
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds Workflow Streams (
workflow.create_stream,create_or_get_stream, their_nowaitforms,to_streamandWorkflowStream, which makes the Stream calls in Workflow code). They write through Local Activities.workflow.create_stream[Turn]("session-7-turns"). A Workflow creates only standalone Streams, under an id of its choice, so the ref has no owner and the Stream doesn't end with the run.create_streamfails withStreamAlreadyExistsErrorif the id exists, andcreate_or_get_streamgets it. The request id is<first execution run id>:<stream id>, so it's the same in each run of the chain. A Workflow Task retry, a retry run or a cron run that creates the Stream again gets the Stream that the first create made. A client sends a new request id with each create call. In a Workflow therequest_id()source of [Brook S2] Generated the Stream service bindings with nexgen, and added the writer client. #372 now returns an empty id, so a create draws nothing from the user'sworkflow.random().workflow.to_stream. There's noworkflow.get_stream.start_system_nexus_operationand runs each one as a write Local Activity. So History keeps a marker per write and has no Nexus events.local_activitycase with the Workflow Task that dispatched it ([Brook G3] Addedactivity.Info.workflow_taskfor Local Activities. #369). So a reader withdraws the writes of a failed task once its retry writes. A retry run's first publish to a Stream withdraws the run that it retries, and a create never does, since the Store refuses that on a Stream with no owner. Theworkflow_writer()andclient_writer()sources of [Brook S2] Generated the Stream service bindings with nexgen, and added the writer client. #372 give the run's writer and the Activity attempt's writer._support.start_nowruns the generated call up to its first wait. Sopublish_nowait,close_nowaitand the_nowaitcreates work in sync code. The error of a_nowaitwrite fails the run once no other write is in flight, since no Local Activity marker may follow the fail command. A write queued behind a create that fails sends nothing and fails with the create's error, since the Stream can be another owner's.publishreturns a future and doesn't block the Workflow. A publish error that no caller retrieved fails the run at its end, and the Workflow's logger logs it when the write fails.async withcloses at the block end, also when the run continues as new in the block. It defers the close to the run's end when a retry run may follow. ABaseException, such as an interrupt, follows the same rule as an exception. On a Stream bound to another run chain it raisesStreamNotOwnerErrorat__aenter__.temporalio.workflow.ReadOnlyContextError.Why?
This is the main use of Streams. An agent Workflow writes its turns to a Stream, and a client or a UI reads them as the Workflow writes them.
This PR is over the size target, since the Workflow calls, the Local Activities and the interceptors only test together.
Open question. Workflow Streams use private parts of the Workflow runtime, such as the run's outbound interceptor chain, the context-free converters and the error that a
_nowaitcall raises in the run. Should the SDK add a hook API inworkflow.unsafefor them?Part of the Streams review series.
Checklist
poe lint,poe gen-docsandpytest tests/contrib/streamson both backends.