Skip to content

[client] Resolve dynamic partition bucket counts before sending - #4309

Open
wuchong wants to merge 3 commits into
apache:mainfrom
wuchong:client-dynamic-bucket-count
Open

[client] Resolve dynamic partition bucket counts before sending#4309
wuchong wants to merge 3 commits into
apache:mainfrom
wuchong:client-dynamic-bucket-count

Conversation

@wuchong

@wuchong wuchong commented Sep 12, 2026

Copy link
Copy Markdown
Member

Purpose

Refs #4287.

A writer may buffer records before a dynamically created partition's bucket count is known. Resolve the partition ID and bucket count together before sending, so a stale table default cannot silently route records to the wrong buckets.

This PR covers tentative routing resolution and server-reported routing failures. The existing synchronous partition-existence check and shared asynchronous metadata refresh work remain follow-ups.

Brief change log

  • Let RecordAccumulator own each physical partition's assigner, temporary bucket count, and resolved layout. Pass the current count to assignBucket and onNewBatch.
  • When the actual count differs, retain buffered batches only for non-hash assigners with every occupied bucket ID below the new count. Otherwise, fail all tentative batches for that context with BucketRescaleException; callers can resubmit the failed records with the same writer.
  • Keep memory allocation and failure callbacks outside routing locks. Synchronize tentative routing resolution and historical target switches while preserving independent writes to resolved buckets.
  • Let the server validate resolved batches using the count carried in each request; remove the redundant metadata comparison and invalidBucketRoutingTables result from the send path. Treat server-reported INVALID_BUCKET_ROUTING and partition-creation failures as fatal writer errors, including queued/in-flight batch cleanup and propagation of the original failure to later writes.
  • Simplify historical routing using the existing restriction that historical partitions and bucket rescaling cannot be enabled together.

Tests

  • Passed 154 focused client tests across BucketRoutingTest, RecordAccumulatorTest, SenderTest, HashBucketAssignerTest, StickyStaticBucketAssignerTest, PartitionBucketCountRescaleITCase, PartitionedTableITCase, and AutoPartitionedTableITCase, including sending the original resolved count after metadata changes and handling server rejection for both count growth and shrinkage.
  • Coverage includes growth/shrinkage, no-key retention and out-of-range rejection, retries with the same writer, consistent metadata snapshots, buffer exhaustion, independent bucket appends, historical routing, and close/fatal-error races.
  • Passed ./mvnw -B spotless:check validate.
  • Full ./mvnw -B verify stopped in fluss-server: ReplicaTest.testBucketPhysicalStorageLocalLogSizeIncludesFollower and testPhysicalStorageLocalLogSizeIsScopedPerBucket both reported expected 450 but actual 7220. No server files are changed by this PR. An isolated rerun of all 16 ReplicaTest cases passed; the full-suite failure remains unresolved, and downstream modules were not verified by that run.

API and Format

Adds the public evolving BucketRescaleException for buffered records rejected before sending. Its recovery contract permits resubmission with the same writer. Bucket assigner signature changes are internal; RPC and storage formats are unchanged.

Documentation

The retry contract is documented in BucketRescaleException Javadoc. No standalone documentation changes.

Generative AI assistance: OpenAI Codex (GPT-6 Astra).

wuchong and others added 3 commits September 13, 2026 00:35
Track temporary and resolved bucket counts in each write context, and
resolve the partition ID and count together before draining buffered
batches. Retain batches only when their assignment remains valid;
otherwise fail them with BucketRescaleException so callers can retry
using the resolved layout with the same writer.

Pass the current bucket count to bucket assigners and synchronize only
while resolving tentative routing or switching historical targets.
Preserve concurrent writes across resolved buckets, keep memory
allocation outside routing locks, and let the server validate resolved
batch counts. Stop the writer on server-reported invalid routing or
partition-creation failures.

Cover bucket-count growth and shrinkage, safe no-key batch retention,
same-writer retries, concurrent routing resolution, and writer cleanup.

Refs apache#4287

Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6-astra
AI-Contributed/Feature: 41/41
AI-Contributed/UT: 33/33
AI-Contributed/Feature: 0/0
AI-Contributed/UT: 0/3
Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6-astra
AI-Contributed/Feature: 10/10
AI-Contributed/UT: 102/179
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant