fix(server): clarify condition resolution semantics for label queries - #2994
fix(server): clarify condition resolution semantics for label queries#2994contrueCT wants to merge 40 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2994 +/- ##
============================================
- Coverage 37.77% 37.76% -0.01%
- Complexity 6560 6593 +33
============================================
Files 800 800
Lines 68960 69168 +208
Branches 9166 9222 +56
============================================
+ Hits 26052 26124 +72
- Misses 39841 39953 +112
- Partials 3067 3091 +24 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
4c42786 to
cc9af24
Compare
There was a problem hiding this comment.
I found one correctness issue in the latest revision. The CI failures were posted separately as a PR-level reminder.
CI/status checks are failing on the latest head (cc9af24929e42af1c90e1f55f3e60adc351e0318). Could you check the failed jobs before the next review round?
Failed checks include:
build-server (memory, 11): https://github.com/apache/hugegraph/actions/runs/26448131941/job/77861497015
cc9af24 to
2e82f83
Compare
There was a problem hiding this comment.
I don't see a clear blocking correctness issue in the latest head, and the previous LABEL-resolution comments look addressed. One remaining merge risk is that the latest checks are still red: hstore failed in VertexCoreTest#testQueryByDateProperty.
Since this PR also touches HstoreStore, could you rerun or clarify whether the hstore failure is an existing flaky/environment issue?
Add explicit condition resolution APIs to ConditionQuery while preserving the legacy condition() behavior. Introduce containsCondition(Object), conditionValues(Object), and conditionValue(Object) so callers can distinguish missing, empty, unique, and multi-value results without overloading null semantics. Migrate LABEL-specific consumers in graph/index transactions, serializers, traversers, and stores to use the new APIs for unique-label resolution and conservative fallback behavior. Extend QueryTest and VertexCoreTest to cover absent, conflicting, and multi-value label conditions as well as collectMatchedIndexes() behavior for multi-label and conflicting label queries.
94408b7 to
b10e3c2
Compare
801923a to
ebc31c8
Compare
|
Thanks for your patience. The hstore CI failure exposed an existing latent issue in hstore's range-index query path. For range-index scans with limit/paging, the upper layer assumed that backend scan results were globally ordered by the range-index key and that the returned page state could be reused as a HugeGraph range cursor. In hstore, multi-node/tablet scans can return entries in backend iterator order, and the page state is an internal storage cursor, so those assumptions may lead to unstable ordering or skipped results. This PR keeps the fix intentionally scoped: hstore range-index queries whose visible result depends on limit/offset/paging are sorted and sliced in the index layer, while unbounded scans still use the original streaming path to avoid disturbing count, joint-index, and cleanup paths. I think this is enough for the current PR, but the underlying hstore scan/page-state contract should be handled in a dedicated follow-up, ideally by defining whether range scans must be globally ordered and fixing the hstore iterator/page-state semantics at the storage-client layer. |
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: HStore range-index offset queries can skip too many sorted results. Evidence: static review of GraphIndexTransaction/query offset handling.
|
Thanks. I fixed this by resetting |
|
I will first create a separate PR to fix #3053 and merge it into this branch before continuing with this PR. |
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Summary: The condition() refactor is behaviour-preserving and the migrated label call sites each pick a defensible semantic (several latent multi-label ClassCastExceptions are removed), but the new hasUnsafeLabelInChain() guard scans a narrower step chain than the extraction loop it protects, and the new empty-label-intersection branch in collectMatchedIndexes() reaches callers as NoIndexException instead of an empty result. Evidence: static review of the exact head 7337d403b10e43ca0c9c64812e7e34ac949fa586 read via gh api -X GET repos/apache/hugegraph/pulls/2994/files and gh api -X GET repos/apache/hugegraph/contents/<path>?ref=7337d403 -H 'Accept: application/vnd.github.raw' (TraversalUtil.java:177,282-285,667; GraphIndexTransaction.java:487-491,779; ConditionQueryFlatten.java optimizeRelations/mergeRelations; Condition.java:143); all 24 check-runs at this head are green, and no local build or focused Maven run backs these findings because the PR head could not be fetched into the checkout in this session.
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The three findings from head 7337d40 are fixed, but the new hasUnsafeLabelInChain() pre-scan walks only HasStep/NoOpBarrierStep, so a RangeGlobalStep from limit()/range()/skip() between an indexed property filter and an unsafe label filter reverts to partial pushdown and silently drops matching elements whose label lacks the index. Evidence: end-to-end memory-backend run at this head on the fixture used by testQueryByNonEqLabelAndIndexedPropertyAcrossBarrier - the .barrier() form returns 4 vertices, the .skip(0) and .limit(1000) forms return 3.
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: A mixed-key logical label filter can still drop matching elements when property-index coverage differs by schema label. Evidence: exact-head static trace through TraversalUtil.java:677-699 and GraphIndexTransaction.collectMatchedIndexes(); no runtime regression test covers this path.
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: the ConditionQuery resolution API and the LABEL call-site migration read consistently, but hasUnsafeLabelInChain() still lets a negative-label predicate sit behind a step that TinkerPop will not hoist a has() across, while the earlier indexed property is pushed down anyway, so results go silently incomplete. One nit on a duplicated serializer helper. Evidence: built this head and ran a probe method on the memory backend (mvn -o test -pl hugegraph-server/hugegraph-test -am -P core-test,memory -Dtest='VertexCoreTest#<probe>'). On the testQueryByIndexedPropertyAndNonEqLabel fixture, g.V().has("city","Beijing").aggregate("x").has(T.label, P.neq("author")) returns 3 of the 4 expected names, and the coin(1.0) variant behaves the same, while the mixed-key or(), order() and dedup() shapes all return 4. Non-memory backends were not exercised. Step lists are in the inline comment.
|
CI follow-up: the Store retry passed completely (11m40s), confirming the earlier failure was the pre-checkout Zulu JDK download timeout. The HStore lane then failed twice, including repository-side rerun attempt 2, at the exact failure already tracked in #3180: |
|
Cross-backend results for this branch (head Before — master Both throw on RocksDB as well, so this is server-side condition resolution, not a backend issue. The other 33 cases are identical sets on both backends already on master. After — master + this branch (+ #3184 and the since-merged #3182): 35/35 identical sets; the two queries above return 0 and 100 vertices respectively, identically on both backends. Version axis (RocksDB master vs RocksDB with the branch, backend held constant): 168 of 174 queries across all four sections (sort-key pushdown, range-index paging, label semantics, Caveat: the "after" run was the combined branch, not #2994 alone; #3184 only touches the HStore pushdown path and #3182 is now in master, so attribution of the two fixes to this PR is by elimination. I can run #2994 alone on request. Raw reports (ids included) and the case list: https://github.com/SebastianGruza/hugegraph-oracle-suite — |
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Summary: The condition-resolution split reads well and all 35 earlier review threads are resolved at this head; I have one error-handling regression in the new local-filter path, one gap in the unsafe-label gate that the PR does not close, and one minor redundancy. Evidence: mvn -o -q compile -pl hugegraph-server/hugegraph-core -am passed; all 24 GitHub checks are green at 0ce61d0; findings below come from static review of git diff 98477f0f..0ce61d0 and the surrounding TraversalUtil / GraphIndexTransaction code. Not verified by a local test run.
Purpose of the PR
ConditionQuery.condition()historically combines several meanings in one API:INrelationThis PR preserves that legacy behavior, adds explicit condition-resolution APIs, and
migrates the high-risk
LABELcall sites to semantics that match each caller.Visual overview
The diagram contrasts strict and tolerant single-value resolution and shows why
negative or ambiguous label predicates stay local to preserve complete results.
Main Changes
Make condition resolution explicit
containsCondition(key)reports any top-level relation for the key.containsConditionValues(key)reports whether a top-levelEQ/INrelation exists,including an empty
INrelation.conditionValues(key)returns the resolvedEQ/INintersection. Pair it withcontainsConditionValues(key)when absence and an empty intersection must differ.conditionValue(key)returnsnullfor an empty result, returns the value for asingleton, and rejects a multi-value result.
singleConditionValueOrNull(key)returns the value only for a singleton and returnsnullfor both empty and multi-value results.condition(key)remains backward-compatible, including returning the raw list for asole
INrelation.Migrate label-sensitive callers
conditionValue()semantics where serializers and sort-key paths requireone resolved label.
singleConditionValueOrNull()where an optimization is valid only for exactly oneresolved label.
conflicting-label queries.
and HStore.
Preserve correctness for negative-label predicates
A downstream unsafe label predicate, including one after a range, side effect, or
inside a child traversal, keeps candidate-filtering predicates local. This avoids
losing matches from labels without equivalent index coverage.
The fallback handles query controls and SEARCH predicates separately:
~pageis consumed as query metadata. The backend page is bounded while localfilters and range steps keep their order. A filtered page can be empty while its
cursor still points to more data; callers must follow the cursor to exhaustion.
Text.contains()uses the same analyzer and term matcher as SEARCH indexes,including
(word),(word1|word2), and analyzed text. For example, searchingbody = "alpha"withText.contains("(alpha)")still matches when a negative labelfollows
limit().HugeGraph.searchPredicate(text)creates this matcher without exposing graphconfiguration; the authorization proxy verifies graph access before delegation.
This is deliberately conservative and can reduce property-index pushdown for those
chains. A nearby FIXME records the follow-up optimization: restore selective pushdown
only after proving compatible index coverage for every candidate schema label.
Verifying these changes
Regression coverage includes:
INrelation and non-EQ/INlabel predicatesconnectives or multiple label containers
filtered pages without missing or duplicate IDs
non-admin access to the SEARCH matcher
Targeted verification:
Does this PR potentially affect the following parts?
The public Java API of
ConditionQuerygains explicit resolution methods, andHugeGraph.searchPredicate(text)exposes the existing SEARCH matching semantics forlocal traversal filters. No REST API, configuration, or dependency changes are included.
Documentation Status
Doc - TODODoc - DoneDoc - No NeedThe API semantics are documented in Javadocs and exercised by regression tests.