Skip to content

feat(columnar): extend cold consumers through STRING and INLINE JOINs - #2292

Open
justinjoy wants to merge 9 commits into
mainfrom
codex/1372-cold-join-group
Open

justinjoy wants to merge 9 commits into
mainfrom
codex/1372-cold-join-group

Conversation

@justinjoy

@justinjoy justinjoy commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Sealed inputs previously required numeric lanes and a narrow JOIN position. This group extends wirelog_session_seal_input to standalone direct/filtered projections, multiple independently sealed inputs, and bounded JOIN consumers in either position, including both-cold, self-JOIN and chains through resident intermediates.

The added reference-consumer commits support STRING keys/payloads and scalar child fields of declared flat INLINE compounds. Plans own declaration metadata and original-use provenance; ordinary source initialization establishes the real layout before capture. Sealing and reading validate that layout and originating intern identity, retain reference/functor ownership, and produce scalarized results. Unsupported layouts fail before source retirement. Truthful STRING typing also preserves builtin interpreter result schemas.

This PR keeps the work in separate commits for projections, multiple bindings, numeric JOIN directions, STRING projections, STRING JOINs, declared INLINE initialization and INLINE cold consumers. Follow-up test commits strengthen formatter stability and output-layout assertions. The nine feature/test commits are rebased onto main 3d2dd8f0, retaining main’s UNION/frontier, inherited JOIN_BATCH environment tests and CI fixes. Already squash-merged #2289 commits are excluded from this series.

Validation on the exact rebased tree: production broad suite 476 passed, 14 skipped, 0 failed; tidy 4/4; rebuilt ASan/UBSan 17/17 and pthread TSan 17/17 affected targets, including all three inherited JOIN_BATCH environment variants. TSan used the documented --timeout-multiplier 4 from the outset. State/thread attribution, formatting and strict production-profile checks passed. Three overlap resolutions retain every feature test, initialize the inherited environment before any fixture, and combine the generic/compatibility sealing contract with the creation-time knob restriction. Tests cover exact typed results, both sealing orders and JOIN directions, self/chained consumers, malformed provenance, independent reference lifetime, capture/read/output failures and retry, and fixed-budget source reclamation. Hosted native-platform results are tracked by this PR's checks.

The supported path remains serial, immutable and nonrecursive, with a single matching INT64 or STRING equality key and the documented restricted projection/filter forms. Intern/declaration metadata, outputs and intermediates remain resident. Whole compounds, SIDE/nested layouts and mutable/differential/control/worker consumers remain follow-up work.

Refs #1372; this group does not close it.

Base automatically changed from codex/1372-cold-readers to main October 29, 2026 08:34
@justinjoy justinjoy changed the title feat(columnar): extend sealed inputs through bounded JOIN consumers feat(columnar): extend cold consumers through STRING and INLINE JOINs Oct 29, 2026
Expose generic seal_input with a compatible seal_join_input spelling. Admit complete direct and narrowly filtered standalone readers without a JOIN prerequisite, preserving mutation freeze, output floors and typed errors. Refs #1372.
Pin and audit originating intern provenance for standalone cold projections. Preserve declared STRING input types and fixed builtin MAP result schemas so ordinary JOIN pipelines retain their key metadata. Cover public consumers, malformed bytecode, ownership, failures and resident memory floors.

Refs #1372
@justinjoy
justinjoy force-pushed the codex/1372-cold-join-group branch from 61b2d1a to d9e3816 Compare October 30, 2026 06:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant