Skip to content

executor: add SkipScan to TableScan and IndexScan - #441

Merged
coocood merged 1 commit into
pingcap:masterfrom
coocood:skip-scan
Nov 19, 2026
Merged

coocood merged 1 commit into
pingcap:masterfrom
coocood:skip-scan

Conversation

@coocood

@coocood coocood commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

What problem does this PR solve?

Problem Summary:

SELECT a, MAX(b) FROM t GROUP BY a on an index (a, b) reads every row of the index today, although only one row per distinct a is needed. This PR adds the protocol for TiDB to ask TiKV for a loose index scan: return the first row of each group, then skip the rest of the group.

What is changed and how it works?

What's Changed:

  • Add SkipScan with prefix_column_count to executor.proto.
  • Add skip_scan to IndexScan (field 9) and TableScan (field 15). Both are only used by TiKV.
  • Regenerate go-tipb/executor.pb.go with scripts/generate-go.sh, and update scripts/proto.lock.

How it Works:

  • Rows are grouped by the first prefix_column_count key columns, plus whether the next key column is NULL. TiKV returns the first row of each group in scan order and skips the rest of the group.
  • MAX(b) uses a backward scan: NULL sorts lowest, so the first row of a group holds the largest b. MIN(b) uses a forward scan: rows whose b is NULL form their own group, so the smallest non-NULL b is still returned.
  • Key columns are the index columns for IndexScan, and the common handle columns for TableScan.
  • A group can span regions, so TiDB still aggregates the returned rows.
  • An absent skip_scan keeps the current behavior. A TiKV that does not know the field ignores it and returns every row, so TiDB must check that TiKV supports it before sending it.

Checks:

  • scripts/generate-go.sh; the other generated Go files are unchanged
  • go build ./go-tipb/... and go vet ./go-tipb/
  • protolock status -lockdir=scripts -protoroot=proto
  • Rust generation: left to CI

Related changes

  • TiKV implementation: follow-up PR
  • PR to update pingcap/docs/pingcap/docs-cn: not needed
  • Need to cherry-pick to the release branch: TBD

Summary by CodeRabbit

  • New Features
    • Added optional skip-scan support for table and index scans on TiKV. Scans can group rows by leading key-column values and the next key column’s null status, with null rows grouped separately. Table scans require a common-handle table for this feature.

SkipScan lets TiKV return only the first row, in scan order, of each
group of rows sharing the same leading key columns, and skip the rest
of the group. It serves loose index scans such as
`SELECT a, MAX(b) FROM t GROUP BY a` on an index (a, b).

Signed-off-by: Evan Zhou <coocood@gmail.com>
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The executor schema adds a SkipScan message and optional skip_scan fields to TableScan and IndexScan. The comments specify TiKV constraints and describe grouping by leading key values and the next key’s null status.

Changes: SkipScan schema

Layer / File(s) Summary
Scan message fields
proto/executor.proto
TableScan and IndexScan gain optional SkipScan fields. The new message defines a non-nullable prefix column count and documents grouping behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Suggested reviewers: changrui-ryan

Merge Risk: Minimal · up to 70f20

No demonstrated issue blocks merging this schema change after normal checks.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: adding SkipScan to TableScan and IndexScan.
Description check Passed The description covers the problem, protocol changes, behavior, compatibility, validation checks, and related follow-up work. The optional issue number and proposal sections are omitted appropriately.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

Buf (1.74.0)

Lint configuration: default MINIMAL rules with the repository root as the protobuf import root; no repository Buf configuration was found. These results do not establish the project's lint policy or build configuration. For unresolved imports or symbols, check the project's declared import roots and dependencies before concluding that its build fails.

proto/executor.proto

fatal: unable to access 'https://github.com/pingcap/tipb.git/': Failed to connect to github.com:443 over proxy 127.0.0.1 after 0 ms: Could not connect to server
fatal: could not fetch e40e19ee471f54c817f6c492df6775a734e072a8 from promisor remote

CODERABBIT_BUF_COMPATIBILITY_PHASE=materialize-base

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit reads the scan fields with care,
SkipScan joins the schema there.
Prefix counts mark the keys in line,
Nulls form a separate sign.
Two scan messages share the name,
The protobuf grows a frame.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bd3d6d2f-59e6-4da1-b7d1-4190b7a8b4c7
Commits

Reviewing files that changed from the base of the PR and between 7c87d58 and 70f20be.

Files ignored due to path filters: 2

  • go-tipb/executor.pb.go is excluded by !**/*.pb.go
  • scripts/proto.lock is excluded by !**/*.lock
Files selected for processing: 1

  • proto/executor.proto

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@pingcap-cla-assistant

pingcap-cla-assistant Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coocood
coocood marked this pull request as draft November 11, 2026 02:54
@coocood
coocood marked this pull request as ready for review November 19, 2026 09:25

@elsa0520 elsa0520 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coocood
coocood merged commit 50c2ce5 into pingcap:master Nov 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants