Skip to content

tipb: backport NDV rate and sampled NDV summaries to release-8.5 (#410, #430) - #438

Merged
0xPoe merged 3 commits into
pingcap:release-8.5from
0xPoe:cherry-pick-410-430-to-release-8.5
Nov 16, 2026
Merged

0xPoe merged 3 commits into
pingcap:release-8.5from
0xPoe:cherry-pick-410-430-to-release-8.5

Conversation

@0xPoe

@0xPoe 0xPoe commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: ref pingcap/tidb#67449

Problem Summary: Cherry-pick #410 and #430 to release-8.5, so that TiKV release-8.5 can sample rows for NDV in full-sampling Analyze.

What is changed and how it works?

What's Changed:

How it Works: Each cherry-pick is one commit. Fields 6 and 7 (from #407) never existed on release-8.5; they are reserved so that the field numbers stay the same as on master. scripts/proto.lock is updated for the analyze changes only, and go-tipb/analyze.pb.go is regenerated.

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Summary by CodeRabbit

  • New Features
    • Added optional configuration for controlling row retention during distinct-value estimation, with validation against the existing sample rate.
    • Added support for recording hashes observed multiple times and tracking the number of rows used for distinct-value estimation.

0xPoe and others added 2 commits November 9, 2026 14:09
Signed-off-by: 0xPoe <poe.liu@pm.me>
Signed-off-by: 0xPoe <poe.liu@pm.me>
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Explain Change →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Files ignored due to path filters: 2

  • go-tipb/analyze.pb.go is excluded by !**/*.pb.go
  • scripts/proto.lock is excluded by !**/*.lock
Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 17819787-5da2-4b4f-8cd9-60c182c35ed0
Commits

Reviewing files that changed from the base of the PR and between 3af26b3 and e754349.

Files ignored due to path filters: 2

  • go-tipb/analyze.pb.go is excluded by !**/*.pb.go
  • scripts/proto.lock is excluded by !**/*.lock

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The analyze protocol adds an optional NDV sampling rate, repeated-hash storage for FMSketch, and an NDV sample count for RowSampleCollector. It also reserves two RowSampleCollector field numbers and documents sampling and count behavior.

Changes: NDV sampling protocol

Layer / File(s) Summary
NDV sampling data contract
proto/analyze.proto
AnalyzeColumnsReq adds ndv_rate and its sampling rules. FMSketch adds multi_hashset for repeated hashes. RowSampleCollector reserves fields 6 and 7 and adds ndv_sample_count, with comments defining count and scaling behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Merge Risk: Minimal · previous review 3af26

The change adds NDV sampling fields; no merge-blocking risk is established.

Pre-merge checks: Passed 4

Check name Status Explanation
Description check Passed The description follows the repository template. It fills in the problem summary, the cherry-pick scope, the list of changes (#410 and #430), and how the changes work, including the reserved fields an…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title names the backport of two changes (#410 and #430) to release-8.5 in the tipb protocol: an NDV rate field on AnalyzeColumnsReq and sampled NDV summaries in RowSampleCollector and FMSketch. Th…
Full details: Description check

Explanation

The description follows the repository template. It fills in the problem summary, the cherry-pick scope, the list of changes (#410 and #430), and how the changes work, including the reserved fields and the proto.lock and generated Go file updates. The 'Issue Number' line references issue #67449 with 'ref' rather than 'close', which is acceptable for a cherry-pick. The 'Proposal' line is omitted, which is allowed because it is marked not applicable. The 'Related changes' section keeps its template placeholders, with the docs PR line left blank and no checkbox state for the release branch note. These are minor gaps, so the description is mostly complete.

Full details: Title check

Explanation

The title names the backport of two changes (#410 and #430) to release-8.5 in the tipb protocol: an NDV rate field on AnalyzeColumnsReq and sampled NDV summaries in RowSampleCollector and FMSketch. The raw summary confirms these additions to proto/analyze.proto, and the objectives describe the same cherry-pick, so the title matches the main change. It is specific and descriptive, though it uses terms like 'backport' and PR numbers that a reader outside the team may need to look up.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit reads the fields with care
New sampling rates are written there
Hashes twice now find a home
Counts describe the sampled roam
Then hops away through protobuf air

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6099a382-2b84-452f-b8ad-a005f166fe49
Commits

Reviewing files that changed from the base of the PR and between 5817d66 and 3af26b3.

Files ignored due to path filters: 2

  • go-tipb/analyze.pb.go is excluded by !**/*.pb.go
  • scripts/proto.lock is excluded by !**/*.lock
Files selected for processing: 1

  • proto/analyze.proto

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@0xPoe 0xPoe changed the title [cherry-pick] analyze: add NDV rate and sampled NDV summaries (#410, #430) proto: add NDV rate to analyze request (#410), analyze: carry sampled NDV summaries (#430) [release-8.5] Nov 9, 2026
@0xPoe 0xPoe changed the title proto: add NDV rate to analyze request (#410), analyze: carry sampled NDV summaries (#430) [release-8.5] [cherry-pick] proto: add NDV rate to analyze request (#410), analyze: carry sampled NDV summaries (#430) Nov 9, 2026
@0xPoe 0xPoe changed the title [cherry-pick] proto: add NDV rate to analyze request (#410), analyze: carry sampled NDV summaries (#430) tipb: backport NDV rate and sampled NDV summaries to release-8.5 (#410, #430) Nov 16, 2026
@0xPoe
0xPoe merged commit a173c6f into pingcap:release-8.5 Nov 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants