Skip to content

Fix test server time-skipping locks after workflow closure - #3167

Closed
thejens wants to merge 1 commit into
temporalio:mainfrom
thejens:fix/test-server-clock-lock-leaks
Closed

thejens wants to merge 1 commit into
temporalio:mainfrom
thejens:fix/test-server-clock-lock-leaks

Conversation

@thejens

@thejens thejens commented Oct 27, 2026 •

Copy link
Copy Markdown

What was changed

The test server's time-skipping lock was not released on two paths:

  • A workflow that is force-closed (terminated, or run timeout) while a workflow task is scheduled or started never releases the lock taken in scheduleWorkflowTask. Later task responses are rejected by the terminal-state check in update before reaching their normal release. update now releases that lock when it force-closes the workflow.
  • Child notifications (childWorkflowStarted/Completed/Failed/Canceled/TimedOut, failStartChildWorkflow) release their lock inside the parent's update callback. If the parent has already closed, update throws NOT_FOUND before the callback runs and the lock is never released. The release now happens in a finally (updateWithTimerUnlock).

Related: #3168 fixes the two paths listed below (pending activities and external signal callbacks) plus cancel-before-start; the two PRs are independent.

Known limits: Two further unbalanced paths remain and are not changed here: pending activities when their workflow closes first (the lock from processScheduleActivityTask), and external-signal callbacks when the source workflow has closed. They need separate tests, so I kept this change to the two paths above.

Why?

With a locked clock the server stops skipping time, so later timers in the same server never fire. We hit this running a large Python test suite on one shared time-skipping server: after tests that terminated workflows, a later test waiting on a virtual timer hung until the test timeout.

Checklist

  1. Closes: no issue filed. It is a small bug fix in the test server and I found no existing issue for it.

  2. How was this tested:

New TestWorkflowMutableStateTimeSkippingTest (10 tests): termination with a scheduled task, with a started task, with a late task completion, idle termination (guards against a double unlock), run timeout, and each of the six child callbacks on a closed parent. Each requires a future timer to fire through automatic time skipping afterwards. Against the unpatched server 9 of 10 fail (the idle-termination control passes); with the patch all 10 pass. ChildLivesLongerThanParentTest.testAbandonChild now also requires a following virtual sleep to complete. :temporal-test-server:test plus the ChildLivesLongerThanParentTest, TestEnvironment, TimeSkipping, TerminatedWorkflowTest and TerminateTest selections in temporal-sdk pass (114 tests).

  1. Any docs updates needed? No. No public API or documented behavior changes. Breaking changes: none. Server PR: none (Java test server only).

This change was developed with AI assistance (Claude) and reviewed and tested locally.

Release outstanding workflow task locks on forced closure. Release child
notification locks even when the parent rejects the update.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@GregoryTravis

Copy link
Copy Markdown
Contributor

We're not accepting fixes for this sort of thing, since we are planning to begin deprecation for this server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants