From f5797b632b2aa74d5c929472a5f1e920839c1239 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Manciot?= Date: Tue, 27 Oct 2026 17:50:15 +0100 Subject: [PATCH] docs(sql): the 65,536 subquery bound is fixed, and how it fails The bound on an uncorrelated subquery's values is a fixed 65,536: raising index.max_terms_count does not lift it. Past it, a default cluster fails with Elasticsearch's own too-many-buckets error (measured on 8.18.3 with 0.24.0), loudly and never truncated. The advice is to narrow the inner query or rewrite the statement as a JOIN. Co-Authored-By: Claude Opus 5.5 --- documentation/sql/dql_statements.md | 7 ++++--- documentation/sql/known_limitations.md | 6 ++++-- documentation/sql/operators.md | 2 +- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/documentation/sql/dql_statements.md b/documentation/sql/dql_statements.md index 9cca484f..a1b93709 100644 --- a/documentation/sql/dql_statements.md +++ b/documentation/sql/dql_statements.md @@ -1117,9 +1117,10 @@ plain REPL: 2. Those values are injected into the outer query as a literal `terms` filter, and the outer query runs as an ordinary search. -The inner result set is therefore **bounded**: past `index.max_terms_count` (65,536 by default) the -statement fails loudly rather than truncating silently. Narrow the inner query, or raise the index -setting. +The inner result set is therefore **bounded**: past **65,536** distinct values the +statement fails loudly rather than truncating silently. The bound is fixed: it is the default of +Elasticsearch's `index.max_terms_count`, and raising that setting does not lift it. Narrow the inner +query, or rewrite the statement as a JOIN. Elasticsearch's own `terms` lookup is not what this uses and could not be: it reads **one document by id**, and cannot express a query. diff --git a/documentation/sql/known_limitations.md b/documentation/sql/known_limitations.md index 684f1f8f..615e03d5 100644 --- a/documentation/sql/known_limitations.md +++ b/documentation/sql/known_limitations.md @@ -149,8 +149,10 @@ the engine** — the JDBC driver, the ADBC driver and the Arrow Flight SQL sidec ### The bound on an uncorrelated subquery The two-phase path resolves the inner statement into a set of values, so it is bounded by what an -Elasticsearch `terms` query accepts — **65,536 distinct values** (`index.max_terms_count`). Past that the -statement fails loudly, naming the limit and suggesting the JOIN rewrite; it is never silently truncated. A +Elasticsearch `terms` query accepts by default — **65,536 distinct values**, a fixed bound that raising +`index.max_terms_count` does not lift. Past that the statement fails loudly, on a default cluster with +Elasticsearch's own *too many buckets* error, which names the 65,536 limit; it is never silently truncated. +Rewrite such a statement as a JOIN. A plain `SELECT FROM …` body is resolved with a single bounded `terms` aggregation, so the values are already distinct and `DISTINCT` buys nothing. diff --git a/documentation/sql/operators.md b/documentation/sql/operators.md index 18e411c2..c69f580e 100644 --- a/documentation/sql/operators.md +++ b/documentation/sql/operators.md @@ -883,7 +883,7 @@ WHERE category_id IN ( Subqueries are accepted **since engine `0.24.0`**. The subquery must project **exactly one column** (`IN (SELECT * FROM …)` is refused), must read a table (`IN (SELECT 1)` is refused) and may not be a set operation. An uncorrelated body is executed first and its -values are collected as a distinct set — bounded at **65,536** values (`index.max_terms_count`), past which +values are collected as a distinct set — bounded at **65,536** values (a fixed bound, the default of `index.max_terms_count`), past which the statement fails loudly rather than truncating. A body over a plain column is resolved with a single `terms` aggregation, so `DISTINCT` inside it buys nothing. A **correlated** body (one that reads the outer row) runs on the relational engine — see