Limit Edge query threads and remove unused binary allocator - #18587
Open
JackieTien97 wants to merge 1 commit into
Open
Limit Edge query threads and remove unused binary allocator#18587JackieTien97 wants to merge 1 commit into
JackieTien97 wants to merge 1 commit into
Conversation
JackieTien97
force-pushed
the
ty-edge-query-thread-limits
branch
from
September 6, 2026 00:54
e4415a0 to
4f2eb0c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Edge still creates query scheduling notifications through an unbounded cached pool, while fragment dispatch uses a hard-coded minimum of 20 workers. Add restart-only settings and set both Edge defaults to 2:
driver_task_scheduler_notification_thread_countfragment_instance_dispatch_thread_countPositive notification sizes queue callbacks on a limited worker pool; workers can exit after 60 seconds of inactivity. Callbacks do not fall back to the threads completing driver futures. Fragment dispatch retains its existing caller-runs behavior when the pool is busy. Negative configuration values are rejected.
Remove the unused BinaryAllocator implementation, its configuration, metrics and lifecycle hooks. Repository-wide call-site inspection found allocation/deallocation calls only in its unit tests; production references merely started it through GC monitoring or configuration reload. The old auto-releaser could remain blocked in
ReferenceQueue.remove()after shutdown, accumulating threads across restarts. Removing the unused subsystem prevents those workers from being created. Normal BLOB handling remains covered by the real Edge read/write test.ConfigNode and SchemaRegion continue to use Ratis; DataRegion continues to use IoTConsensus.
Validation
Validated after rebasing onto master (
4f2eb0c790):IoTDBEdgeBasicITtests passed, including tree/table reads and writes, concurrent table queries, 40 cross-database join queries across four concurrent connections, consensus assertions, and a 64 KiB BLOB verified before/after flush and configuration reload.enable_binary_allocator=truekey. The thread dumps contain no BinaryAllocator workers. The cross-database join dump contains two fragment-dispatch workers and two driver-notification workers.