[fupelaqu]

Summary

Slice 1 of 2 of a schema-aware result path in core (SCHEMA-2). Every statement core executes now gets a declared result schema before any row. The schema rides on the result, and GatewayApi.describe(sql) returns it without running the statement. Every value read from a hit (_source, script fields, inner hits) is read by its column's declared type, where it is first read. The same holds on every hit path: one-shot search, searchAsync, _msearch legs, the UNION ALL legs, scroll / search_after / PIT pages for the five client page parsers, and inner hits.

Aggregation values (bucket keys, metrics, top_hits) and window-enriched statements are slice 2 (SCHEMA-2b). They keep today's reading in this PR.

Refs #418, #421 (both stay open: #418's second part and #421's aggregation side land with the next slices).

What changes

• Public API: ResultSchema / ResultColumn, GatewayApi.describe(sql): ElasticResult[Option[ResultSchema]], schema: Option[ResultSchema] on QueryRows and QueryStream, def schema on QueryStructured.
• Hit reading (HitReading): one reader per column, built once per statement. It covers numbers and booleans as Elasticsearch coerces them, dates by mapping format (one format build per mapping format per statement), geo_point, binary, STRUCT by declared field and declared ARRAY. A value the reader cannot read is returned as stored.
• Integer columns follow Elasticsearch's own rules per type and per JSON node kind: truncation, range checks and whitespace handling are read from the 6.8.23, 7.17.29 and 8.18.3 sources. A value Elasticsearch refused under ignore_malformed is returned as stored.
• Projection follows the schema: SELECT * expands to the cached mapping's columns. An explicitly selected field the mapping lacks triggers one mapping reload, then is returned as stored.

Release notes

• Downstream rebuild (binary and source break): QueryRows, QueryStream and ElasticResponse gain a field, so positional patterns need one more _. QueryStructured patterns compile unchanged. Recommended consumer shape: case q: QueryRows => q.rows.
• New: describe(sql) and schema on results (nullable is always true).
• Values in canonical classes:
  • BIGINT Long, DOUBLE Double, REAL Float (was Double).
  • SMALLINT / TINYINT Integer, in core UNION ALL too (was Short / Byte).
  • Keyword numbers stay text.
  • Dates are read by their mapping format into java.time at UTC. LocalDateTime is gone: an undeclared date holding date-only values answers a ZonedDateTime at midnight UTC (was LocalDate).
  • Binary answers its decoded bytes; geo_point answers {lat, lon}; one-element arrays are unwrapped.
  • Computed BIGINT cells (GREATEST / LEAST over integers) answer Long (was Integer).
• Integer columns answer the value Elasticsearch indexed (e.g. a TINYINT 127.9 answers 127). A value Elasticsearch refused stays as stored.
• SELECT * returns the cached mapping's columns, in mapping order. A key stored but not mapped is not returned; a field mapped since appears after the schema-cache TTL. Rows hold no leaked parent object.
• Dates Elasticsearch did not index (_ignored) answer NULL.
• Core UNION ALL of a date with a zone-name or region-id format answers VARCHAR, where 0.24.0 failed.
• A date column asks the Elasticsearch version once per statement (cached per client).

Acceptance evidence

Check	Evidence
describe = the result's schema, no search issued	DescribeSpec (unit, counting double = 0 searches); SchemaReadingSpec on ES 8.18.3 and 6.8.23
One class per column on every hit path	SchemaReadingSpec (ES 8.18.3, ES 6.8.23); canned pages per client page parser (es6 rest, es6 jest, es7 rest, es8 / es9 java)
No statement answered on 0.24.0 is refused	Full client legs on the branch: ES 8.18.3 *JavaClient* *ReplGateway* 832 passed / 0 failed; ES 6.8.23 *RestHighLevelClient* 727 / 0
SELECT * and unmapped fields	SchemaReadingSpec (ES 8.18.3)
Downstream	JavaClientSchemaConsumerSpec (consumer forms). Throwaway local publish, deleted afterwards: extensions, JDBC and arrow compile with the consumer shape only and each run one real query on ES 8.18.3

Cost (warm JIT, setup split from the row loop, branch runs interleaved with main runs on the same machine):

Measure	0.24.0	This PR	Bound
Per row, full page (12 columns incl. 3 dates)	11,653 ns	1,366 ns	≤ main × 1.03
Per row, agreeing page (keyword, numbers, boolean)	722 ns	414 ns	≤ main × 1.03
Per statement (reader build), median of 5	0.55 µs	21.8 µs (14.7 / 20.6 / 21.8 / 26.0 / 30.4)	≤ 25 µs

The per-statement runs were taken under a machine load of 5 to 8. Two of the five runs exceed 25 µs; the median is the bound's measure.

Recorded, not fixed

• Set-operation slot integer narrowing still trims and wraps out-of-range values, as in 0.24.0.
• Each distinct unmapped, explicitly selected field costs one mapping read per schema-cache TTL. That reload also empties the index's alias, pattern and shard-count caches.
• BIGINT texts on which ES 6.8 and 8.x index different values (e.g. "5.000000000000000000001", "0e20") are returned as stored on both majors, as in 0.24.0.
• The ES 8 / 9 Java client's deserializer alters JSON integers beyond the long range (and 1e400) before core reads them, as in 0.24.0.
• The REPL twins of the gateway row tests assert no rows on a streamed result (a testkit gap that predates this PR).
• Window-enriched statements feeding a set operation keep one date pass-through case; they move whole to slice 2.
• Downstream smoke limits: ES 8 modules only. JDBC's arrow test artifacts built on 0.24.0 were not re-proven against this core.

Test plan

• core/test (1,388 passed), sql/testOnly *SetOperation* *Select* *ValueCoercion*, sbt "+ compile" (2.12 + 2.13)
• Docker-free client unit specs: es6 rest, es7 rest, es6 jest, es8 java, es9 java
• ES 8.18.3: full client leg, plus the date / schema area specs after the last fixes
• ES 6.8.23: full client leg, plus the date / schema area specs after the last fixes
• CI lint line (headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck) on the committed bytes
• ES 7.17 and ES 9 legs, es6 jest leg (CI)

🤖 Generated with Claude Code