feat(runtime): ADR 0012 burst + preemption (staged on the #44/#50 design track) - #53
Draft
ThomasK33 wants to merge 9 commits into
Draft
feat(runtime): ADR 0012 burst + preemption (staged on the #44/#50 design track)#53ThomasK33 wants to merge 9 commits into
ThomasK33 wants to merge 9 commits into
Conversation
Implement the remaining ADR 0012 surface: ConcurrencyDebounce, ConcurrencyBurst, ConcurrencyConcurrent, a configurable LockScope (thread default, channel opt-in), and handler preemption/steerability via RuntimeOptions.OnLockConflict with force-release-lock coordination through the new chat.LockForcer optional State capability (ForceReleaseLock on memory/redis/postgres/nats states + conformance). Drop stays the default; existing drop/queue behavior is unchanged and the public API is additive. Closes #7
- preempt: verify the waiter is still pending before destroying in-flight work; wait (ctx-bounded) for the local victim to finish before force releasing so drop/queue serialization holds locally; bound the force release by the Detached Work Context instead of WithoutCancel - outcome telemetry: record OutcomePreempted by cancellation cause even when the handler returns nil, and close acquire-failure waits (queue/ debounce/preempt/burst) as OutcomeError instead of ignored - validation: reject DetachTimeout <= DebounceInterval under debounce and burst, which would abandon every event before dispatch - lock scope: length-prefix channel key fields so the mapping is injective across adapter/tenant/channel combinations - debounce: superseded waiters exit promptly on a displacement signal instead of parking through the full interval holding their payloads
- preempt: re-validate pending ownership after waiting for the local victim, so a displaced preemptor cannot force-release a lease a newer waiter already holds; residual instruction-scale races stay fenced by lease refresh cancellation - lock waits: re-check pending ownership after AcquireLock returns held, releasing the fresh lease when the waiter was displaced mid-acquire (debounce/queue newest-waiter-only guarantee) - lock scope: namespace channel-scope fallback thread keys (thread-scope/<len>:<id>) so an opaque Thread ID can never collide with a synthesized channel key - burst: document join-order (not arrival-order) batch semantics
A local preemption victim releases its own lease after cancellation, so the preemptor now simply waits for it and acquires normally - a lease acquired by a fresh unrelated handler after the victim's release can no longer be destroyed. Force release through LockForcer is reserved for leases with no locally registered holder (remote instance or orphaned), where lease-refresh cancellation remains the documented fence. The preemption log/observation now carries forced=true/false.
- burst: the batch dispatch now runs under a fresh DetachTimeout that starts when the collection window closes, so collection time can no longer consume accepted batch members' execution budget - nats: gate ForceReleaseLock's delete by the observed revision so only the lease seen in-call is invalidated (parity with the atomic Redis DEL / Postgres DELETE), treating a mid-call handover as no-op - docs: state explicitly that queue/debounce supersession and burst windows coalesce per runtime instance (cross-instance coalescing is the ADR 0012 wait/coalesce State extension, tracked in #50)
…newal retry - preempt: local lease ownership is now reserved the moment a lease is acquired (prelude and lock-wait acquisitions), not when the handler starts, so a preemptor arriving while the owner is still routing or deduping can never mistake the fresh local lease for a remote one and force-release it; a reservation preempted before the handler starts prevents the handler from running at all (span outcome preempted, started=false) - nats: a revision mismatch during ForceReleaseLock re-reads the entry and retries while the token is unchanged, so an ordinary same-token ExtendLock renewal is no longer mistaken for a lease handover
…er budgets, universal lease-loss cancel - preempt: ownership reservations are registered BEFORE AcquireLock (a lease can never exist locally without a visible reservation) and the registry is multi-entry per scope, so concurrent acquirers and the holder coexist; the preemptor cancels and awaits every reservation, treats a formerly-held one as a clean local handoff, and loops on non-holding acquirers before concluding the lease is remote - runtime: every deferred lock holder is now cancelled (ErrPreempted) when its lease is lost, regardless of the local hook configuration, so a force release from another instance actually stops the victim - burst: each accepted batch member runs under its own DetachTimeout budget; earlier members and collection time never consume it, and lease loss cancels the remaining members
…burst cause telemetry - preempt: pending-slot validation and local cancellation are now one atomic operation under the pending registry lock, so a preemptor displaced by a newer registration can never cancel the active holder or poison the newer waiter's reservation on behalf of a dead event - burst: a member stopped by lease loss is classified by its cancellation cause (OutcomePreempted + preemption log), matching runLockedTail, instead of leaking handled/error outcomes
An ExtendLock error exits the refresh loop, after which the lease expires at TTL while the handler (or remaining burst members) would keep running unserialized. Treat a refresh error like a lost lease: mark it lost (benign release) and cancel the handler with ErrPreempted so no holder outlives a lease the runtime can no longer maintain.
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Draft — do not merge. Staged pending the deferred-dispatch admission-bound design (#44) and the fenced cross-instance coordination design (#50), per the maintainer's convergence decision on #38.
Carries the remaining ADR 0012 surface split out of #38:
ConcurrencyBurst, theOnLockConflictforce/steerability preemption hook, and theLockForcerState capability (ForceReleaseLockon memory/redis/postgres/NATS + conformance coverage), with all fixes from nine codex review rounds (pre-acquire ownership reservations, atomic pending-validation+preemption, per-member burst budgets, NATS revision-gated force with renewal retry, victim-await handoff, burst cause telemetry).Why staged
Review kept producing legitimate P1 races in exactly these two areas — the latest being non-atomic remote-force validation vs. supersession, a stallable burst refresh call, and unfenced sync-mode holders. Root cause: key-only
ForceReleaseLockcannot bind the force to the lease observed at conflict time (the State contract hides holder identity), and burst/preemption lifecycles need the admission-bound model to be settled first. Both get designed once on the #44/#50 track; the outstanding round-9/10 findings on #38 apply to this branch and should be folded into that design pass:ExtendLockcall).This branch is the full pre-split state of #38 (
0892821) and will need a rebase over the reduced #38 once it merges.Refs #7.
Generated with
mux• Model:anthropic:claude-fable-5• Thinking:xhigh