Conversation
Signed-off-by: Andrew Stein <steinlink@gmail.com>
Signed-off-by: Andrew Stein <steinlink@gmail.com> # Conflicts: # rust/perspective-client/src/rust/virtual_server/generic_sql_model/tests.rs
Signed-off-by: Andrew Stein <steinlink@gmail.com> # Conflicts: # packages/viewer-datagrid/src/ts/custom_elements/datagrid.ts
texodus
force-pushed
the
zero-cost-table-view
branch
5 times, most recently
from
October 1, 2026 04:40
a5c1c38 to
8bdba05
Compare
Signed-off-by: Andrew Stein <steinlink@gmail.com>
texodus
force-pushed
the
zero-cost-table-view
branch
from
October 1, 2026 14:16
8bdba05 to
b0159df
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes
client.table(view)an engine-supported API and optimizes it to be effectively zero-cost, sharing memory with the underlyingViewit wraps. In addition, two related breaking API changes have been introduced which cover functional gaps in makingtable(view)symmetric (in preparation for adding it to the UI).client.table(view)optimizationIt has always been possible to flatten a
Viewinto aTableby passing theViewto theclient.table()function. Internally however, this mechanism had no true special treatment - theViewwas serialized and de-serialized via Arrow to populate another copy. This was slow (relatively) and memory wasteful.Now,
client.table(view)(when called with the sameClientthat owned theView) will create a special derivedTable, allocating only metadata but sharing its columns with the inputView, with a few changes:update(),remove(),clear()andreplace()raise, and so do theindexandlimitoptions. These could technically fall back to the cost of a copy, but I'm not sure its work it.view.delete()raises while a derived table still reads the view; delete the derived table first. Previously the copy became a frozen snapshot.sortis ignored. Rows follow source insertion order. This is similarly a consequence of sharing the underlying (which is not sorted in-place). It is easy enough to manually inherit the sort but this is up for debate.filterare removed, where the copy kept them with stale values.rollup; setgroup_rollup_mode: "flat"for leaves only. The old copy appended its changed rows on every tick.split_byvalue that appears later is dropped, and one that vanishes goes all-null. The newschemaoption onclient.table(view, { schema })pre-declares columns.table(view)now errors unless the handler reports the newview_derivationsfeature and implementsview_make_table. DuckDB and Postgres implement this (Vieware alreadyTEMPORARY TABLEon these engines so this is free but static).view.column_paths()returns structured pathsPreviously a
string[]of"a|b|Sales"; it is nowarea[level][column]of typed scalars, in JS, Python and Rust (Vec<Vec<Scalar>>). Because column names in flattenedTables may now have separator values routinely, splicing by the separator character no longer works for separating group levels from column names.datetimesplit value is epoch milliseconds, floats are full precision (not%gwith 6 significant digits), and null isnull, not"null"(in headers).|:to_columns/to_jsonkeys, CSV headers, and derived-table column names.split_byandgroup_byheaders use the column'snumber_format/date_format(split_by doesn't respect the date formatting settings #2922), so header text changes. The plugin'scolumn_config_schemareceives a trailingroleargument ("column" | "group_by" | "split_by"), which is additive.sumfamilynullsemanticsPerspective has a family of
sumaggregates that exposed an asymmetry when used withtable(view), namely that asumof a column of exclusivelynullvalues is0, which caused empty groups from the parentView(caused by asplit_byeliminating a cell) to show as0in the derivedTable. This could be cured by usingsum not nullas an aggregate instead, butsumbeing the default makes this behavior surprising.As a fix,
sumnow works likesum not nullwas intended to work - returningnullfor groups with no valid values instead of0sumreturnsnullfor a group with no non-null values; it used to return0. This matches SQL, the SQL virtual servers, and the engine's ownmean/min/maxbehavior.sum or zerois new and keeps the old 0 behaviour.sum not nullis a parse-only legacy alias forsum.sum abs,abs sum,pct sum parentandpct sum totalfollow the samenull behavior change. A null sum used to render as 0%, eg.
All of these
sumaggregates used the incremental aggregate path now as well, making them all equally fast.Internal change, refactoring a a bunch of tests
viewer.restore()has been refactored to guarantee atomic-update-or-error semantics going forward (and has better error handling as a result for broken configs).