Repository navigation
feat(core): computed-only projections without _source, DuckDB numeric widths (SCHEMA-5) - #455
Merged
Merged
Conversation
… widths (SCHEMA-5) Story SCHEMA-5. Reviewed: r0:must-fix, r1:clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…res BIGINT (SCHEMA-5) Story SCHEMA-5, fix round 3 (lead ruling R-67): the rounded value always fits a BIGINT, so it no longer overflows; a BIGINT operand keeps the overflow failure. Reviewed: r0:must-fix, r1:clean, r2:clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…SCHEMA-5) ROUND takes its operand's type, so ROUND(r, 1) over a REAL is a REAL column read as java.lang.Float: the same float DuckDB answers, where the test compared the text of the float widened to a Double. The test now asserts the Float class apart (== accepts the widened Double) and the float values; the WHERE assertion keeps proving the float rounding. Story SCHEMA-5, CI fix. Reviewed: r3:clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fupelaqu
marked this pull request as ready for review
November 4, 2026 17:27
This was referenced Nov 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Computed-only projections and numeric widths in core (SCHEMA-5, after #453 and #454). A SELECT whose items are all computed (
SELECT 2 AS k FROM t,SELECT n + 1 AS v FROM t) no longer fetches_source("_source": false), so its rows hold only the selected columns on every path. The computed columns core still declared as an unsized number (DATEDIFF family, ROUND, EXP / LOG / LOG10 / SQRT / POW) declare DuckDB 1.5.5's type, and DATE_TRUNC over a date declares TIMESTAMP. A quoted number beside a whole type reads as DuckDB casts it.Closes #418
#418 asked for one plain value per SELECT item and for
_source: falsewhen no item is a plain column (SELECT 2 AS k FROM treturned every source field besidek). The one-plain-value half shipped with #453 and #454; this PR ships the_sourcehalf and the widths of the columns such a statement declares.What changes
*,_id, nested or UNNEST item, aggregate, GROUP BY, window, or DELETE / UPDATE by query. Both bridges (ES 6 and ES 7/8/9) send_source: falsefrom it; every other request keeps its bytes. The_msearchlegs and the per-legUNION ALLroute reuse the conversion._source.includesfor its PARTITION BY columns, in the same parse that adds the instants'docvalue_fields. A base with a plain column keeps its includes.BIGINT. EXP / LOG / LOG10 / SQRT / POW declareDOUBLE(values unchanged). ROUND declares its operand's type (REALforREAL,DOUBLEforDOUBLE, the integer type for an integer); a negative scale overTINYINT/SMALLINT/INTdeclaresBIGINT, so the result never overflows. DATE_TRUNC over a date operand declaresTIMESTAMP, in a query only. Every consumer of the declared type follows:describe, set-operation slots, GROUP BY key levels, MIN / MAX / SUM, per-group calculations, window values, CTAS / materialized-view mappings, date arithmetic, GREATEST / LEAST, conditional arms.signum(x) * round(|x| / 10^-s) * 10^-s. Over aBIGINTwhose rounded value leaves the BIGINT range the query fails (Overflow in ROUND of integer).TINYINT/SMALLINT/INT/BIGINT, DATEDIFF, ROUND over an integer or a date part, a decimal text reads as its integer rounded half away from zero ('1.5'= 2,'-2.5'= -3,'1e3'= 1000), in comparisons, IN, BETWEEN, NULLIF, CASE, HAVING key filters and arithmetic, on either side of a scripted comparison. Past the BIGINT range after rounding,DOUBLE/REALoperands, whole-number texts and non-numeric texts ('abc','NaN') are unchanged.functions_math.mdand the type pages describe the new widths and the BIGINT overflow rule.Release notes
_source: their rows hold only their columns, schema-less included. INSERT ... SELECT writes only those columns. An entity's unselectedOptionfield reads None (a defaulted one its default);searchAsUnchecked/scrollAsUncheckedfail on an unselected required field. A schema-less window row carries only its partition columns as extras.BIGINT(Long): its GROUP BY key isLong(wasDouble3.0); MIN / MAX over it maplongin CTAS / MV (was refused); SUM over it isBIGINT(wasDOUBLE), so its CTAS / MV column islong(wasdouble) and its window value and JDBC / Arrow type move the same way.d + DATEDIFF(a, b)over aDATEdis aDATEin whole days (was aTIMESTAMP).DOUBLEit answersDouble(wasLongat scale 0), overREALFloat. GROUP BY ROUND(n) keys areLong(were 3.0). CTAS / MV and MIN / MAX mapinteger/long/float/double(was refused); SUM(ROUND(whole)) isBIGINT,long(wasdouble).d + ROUND(n)over aDATEis aDATE. A negative scale overTINYINT/SMALLINT/INTanswers aLongand DuckDB's value:ROUND(n, -5)forn= 423789293 is 423800000 (main: 423799999.99999994), andROUND(CAST(2147483647 AS INT), -1)is 2147483650 (main: the double 2.14748365E9; DuckDB wraps to -2147483646 because it keeps INTEGER). An untyped whole literal inside a function isBIGINT, soROUND(2147483647, -1)answers 2147483650 as aLongwhere DuckDB labels it INTEGER.DOUBLE: CTAS / MV maps themdouble(was refused). Values unchanged.TIMESTAMP.n = '1.5'matched no row,HAVING k = '1.5'no group), and rendered a literal on the left of a scripted comparison as text.n + '1.5'(now rounded); a wrapped DATEDIFF or whole-day ROUND term (d + ABS(n)-style wrappers: onlyMath.round(x * 86400000.0)becomesx * 86400000L); GREATEST / LEAST over DATEDIFF or an integer ROUND, bare or wrapped (now exact beside aBIGINTpast 2^53, where main went throughMath.max(double, double)).Answered before, refused now (accepted)
INSERT ... SELECTof computed columns only that projects none of the target's PRIMARY KEY / PARTITION BY columnssearchAsUnchecked/scrollAsUncheckedof a SELECT that does not project a required field_sourceOptionfield reads None)ABS(n) + '1.5'over an integer columnABS(n) + '1'already isROUNDover aBIGINTwhose rounded value leaves the BIGINT rangeOverflow in ROUND of integer)Acceptance evidence
_source: false, every other request keeps its bytesSQLQuerySpecin both bridges (moved pins are all computed-only row statements); ES 8.18.3 and 6.8.23 area legs_msearchand per-leg routesComputedProjectionSpecon ES 8.18.3 and 6.8.23UNION ALLleg, GROUP BY key with HAVING, MIN / MAX / SUM, per-group calculation, window value, CTAS mapping)ComputedProjectionSpecandNumericWidthReportedTypeSpec(21/0)ROUND(n, -5)=Long423800000,ROUND(CAST(127 AS TINYINT), -1)= 130,ROUND(CAST(2147483647 AS INT), -1)= 2147483650, overflow refused on aBIGINTindex; ES 8.18.3 70/0 and ES 6.8.23 58/0 on the round's specssbt "+ compile"OKheaderCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck)DuckDB 1.5.5 is the type oracle. It keeps the integer type for ROUND and wraps silently on overflow; core declares
BIGINTfor a narrower operand at a negative scale instead, and fails aBIGINTthat overflows.Cost (warm JIT, setup split from the row loop, main and branch runs alternated on the same machine, load 3.8 to 7.5):
The window base-request rewrite adds 2.4 us per statement. The planning figures predate the last two fix rounds, which add render-time memo objects only (nothing per row).
Recorded, not fixed
Measured the same on main and on this branch:
GREATEST/COALESCEover DATEDIFF, ROUND, SQRT and ABS of an integer fail to compile in Elasticsearch (the arm's NULL-guarded ternary is inlined without parentheses);ABS(n) + 1andABS(n) + '1'fail the same way.COALESCE(f, c)over those functions answers NULL wheref's operand is NULL; DuckDB answersc.DOUBLEorBIGINTkeeps the integral type:SQRT(n) * n,POW(n, 2) + n,DATEDIFF(d, d2) * n,ROUND(n, -1) + nare declaredINTand can hold aDoubleor aLong.BIGINTnear 2^63 rounded to a negative scale is computed in double:ROUND(9223372036854775000, -3)answers 9223372036854774784 (DuckDB: 9223372036854775000).LocalDatereceiver (CURRENT_DATE,CAST(ts AS DATE), DATE_PARSE, LAST_DAY), QUARTER over CURRENT_TIMESTAMP / NOW and over DATE_PARSE, and DATE_ADD at DAY.obj.a) or a multi-field sub-path (name.keyword) reads NULL on every row.SELECT UPPER(id) AS u, COUNT(*) OVER ()) answers one row withuNULL.k: [4]), and a computed key gives_id{0=r1}.'1.5' BETWEEN n AND bigfails the shard; the IN form is a parse error).Test plan
sql/test(2196 passed; the one failing row replaced and green,NumericWidthReportedTypeSpec21/0),core/test1420/0, bridges es8 354/0, es6 361/0, es9 354/0,sbt "+ compile"(2.12 + 2.13)headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck) on the committed bytes🤖 Generated with Claude Code