Add a statistics-aware selectivity estimator for agtype @> and @>> - #3
Open
NotHimmel wants to merge 1 commit into
Open
Add a statistics-aware selectivity estimator for agtype @> and @>>#3NotHimmel wants to merge 1 commit into
NotHimmel wants to merge 1 commit into
Conversation
A MATCH property constraint such as (n:Label {key: value}) is compiled to
`properties @> '{"key": value}'`. Neither stock estimator can see through
that. contsel returns a fixed 0.001. matchingsel, the binding before apache#2356,
consults the statistics of the whole properties column, which say nothing
about the distribution of any single key, and it is expensive: it calls
agtype_contains() once per MCV and histogram entry, which is why 639c8d5
reverted it. Both give {person_id: <unique value>} and {city: <one of 50>}
the same estimate.
On a multi-hop MATCH the resulting overestimate of the start vertex pushes
the planner away from per-vertex index probes and toward a full scan of
every edge label table joined with a hash or merge join. On a 3M-vertex
graph with three 3M-row edge tables, `(a:VTABLE {person_id: '...'})-->()-->(c)`
estimates the start at 3000 rows instead of 1 and degenerates into a
Parallel Seq Scan of all three edge tables: cost 156134, 691 ms. With this
estimator the start estimates at 1, the second hop becomes a parameterized
Bitmap Index Scan on start_id, and the same query costs 25932 and runs in
0.64 ms.
agtype_contains_sel() decomposes the containment constant exactly the way
the parser does when age.enable_containment = off
(transform_map_to_ind_recursive for @>, transform_map_to_ind_top_level for
@>>): one equality per leaf on
agtype_access_operator(VARIADIC ARRAY[properties, '"key"', ...])
That is the expression users index or attach extended statistics to.
build_access_expr() synthesizes the same node shape transform_A_Indirection
produces, so examine_variable() matches an expression index or a statistics
object by structural equality, and var_eq_const() turns the statistics into
the selectivity an explicit WHERE n.key = value gets. The two ways of
writing the filter now estimate identically. Nested keys extend the array;
per-leaf selectivities are multiplied.
The properties column's own statistics are never read and agtype_contains()
is never called at plan time.
Fallback contract, so that the apache#2356 planning regression cannot recur: the
estimator returns the 0.001 contsel produced when
age.enable_containment_statistics is off, when the operand is not a
constant non-empty object, when the relation has neither an expression
index nor extended statistics, or when no leaf finds statistics. The
relation gate only inspects rel->statlist and rel->indexlist, both already
in memory, so a relation without expression statistics performs no node
synthesis at all and gets byte-identical plans.
Only the RESTRICT bindings of @> and @>> change. JOIN selectivity stays
contjoinsel, and <@, <<@, ?, ?| and ?& are untouched.
Also adds age.enable_containment_statistics (boolean, PGC_USERSET, default
on) so the estimator can be disabled per session, and extends the
containment_selectivity regression test: the existing guard that no
matchingsel estimate leaks is kept, and assertions are added that an inline
map estimates the same as the equivalent WHERE clause for unique,
low-cardinality, nested and mixed constraints, and that a relation without
statistics, an unknown key, and the GUC turned off all still produce the
contsel estimate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A MATCH property constraint
(n:Label {key: value})compiles toproperties @> '{"key": value}', and neither stock estimator can see into the map:contsel(current binding) returns a fixed 0.001.matchingsel(the binding before Critical TPS drop in PG 18 branch caused by matchingsel selectivity for @> operator apache/age#2356) reads the statistics of the wholepropertiescolumn, which say nothing about any single key, and callsagtype_contains()once per MCV and histogram entry — the planning regressionthat 639c8d5 reverted.
Both give
{person_id: <unique>}and{city: <one of 50>}the same estimate.On a multi-hop MATCH the overestimated start vertex pushes the planner off
per-vertex index probes and onto a full scan of every edge label table. On 3M
vertices with three 3M-row edge tables,
(a:VTABLE {person_id: '...'})-->()-->(c)estimates the start at 3000 rows instead of 1:
contselApproach
agtype_contains_sel()decomposes the containment constant exactly the way theparser does when
age.enable_containment = off— one equality per leaf onagtype_access_operator(VARIADIC ARRAY[properties, '"key"', ...]), which is theexpression users already index or attach extended statistics to.
build_access_expr()synthesizes the node shapetransform_A_Indirectionproduces, so
examine_variable()matches by structural equality andvar_eq_const()yields the selectivity an explicitWHERE n.key = valuegets.Nested keys extend the array; per-leaf selectivities are multiplied.
The
propertiescolumn's own statistics are never read, andagtype_contains()is never called at plan time.