perf(cpp): share a writer-owned schema cache across tree and table writes - #945
Closed
ColinLeeo wants to merge 1 commit into
Closed
perf(cpp): share a writer-owned schema cache across tree and table writes#945ColinLeeo wants to merge 1 commit into
ColinLeeo wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repeated writes currently resolve each measurement name through a per-device schema map. This PR reuses those lookups through one bounded cache owned by
TsFileWriter, covering tree records, tree tablets, and table FIELD columns.Related to #885 and the earlier tree-only implementation in #934. This is an alternative implementation based on the current
developbranch; the initial cache regression cases build on #934.Changes
E_NOT_SUPPORTrather than producing incomplete historical alignment.The entry-count bound is 64 devices, not a byte limit: each entry retains vectors sized for its device's schema. This does not add support for concurrent public calls on one writer; internal parallel encoding continues to use the existing thread pool.
Validation
detect_leaks=0). UBSan reported existing null-pointer arithmetic atcommon/allocator/page_arena.h:92; a standalonePageArena::alloc(2048)followed byalloc(8)with a 1024-byte page reproduces it without any writer or cache. The allocator is unchanged from the base, and this is not a clean UBSan run.git diff --check.The regression suite checks row values and timestamps across repeated writes, changed names/order/width, late registration, multiple devices and writers, destroy/re-init, eviction beyond 64 devices, table TAG/FIELD layouts, sparse aligned fields, serial/parallel page boundaries, and recovery preserving an empty-page backfill and the following valid chunks.