fix: preserve groupless aggregate subquery cardinality

[ryux1]

Which issue does this PR close?

• Closes Correlated EXISTS/IN subqueries with groupless aggregates hit the count bug #24960.

Rationale for this change

A groupless aggregate always emits one row unless an operator such as HAVING or LIMIT 0 removes it. Correlated EXISTS and IN predicates currently decorrelate that aggregate into semi/anti joins, which loses the empty-input aggregate row. As a result, EXISTS drops outer rows without an inner match, IN misses comparisons against values such as count(*) = 0, and NOT IN can fail planning with a two-column null-aware anti join.

What changes are included in this PR?

• Recognize correlated, guaranteed-single-row subqueries through projection and alias wrappers.
• Fold EXISTS and NOT EXISTS according to that cardinality.
• Rewrite IN and NOT IN to scalar-subquery comparisons so the existing scalar decorrelator preserves empty-input aggregate values.
• Keep HAVING, LIMIT, grouped aggregates, and uncorrelated subqueries on their existing paths.
• Update the affected logical-plan snapshot.

What is the testing strategy for this PR?

The issue_24960.slt regression file covers top-level EXISTS, NOT EXISTS, IN, and NOT IN, plus embedded EXISTS. It also verifies that HAVING and LIMIT 0 are not treated as guaranteed-one-row cases.

Plan-level optimizer tests directly cover all four predicate forms at top level and inside another predicate, plus projection, alias, and HAVING cardinality boundaries. Codecov reports zero missing changed lines in the optimizer implementation.

Locally passing:

• cargo test --test sqllogictests -- issue_24960.slt subquery.slt
• cargo test -p datafusion-optimizer --lib (779 passed)
• PATH=/home/ryu/.cargo/bin:$PATH ./dev/rust_lint.sh
• cargo fmt --all -- --check

Are there any user-facing changes?

Yes. Correlated EXISTS and IN predicates over groupless aggregates now preserve SQL aggregate cardinality and return the correct rows. There are no public API changes.

Implementation and validation were completed with AI coding assistance under the account owner's direction.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 72.81553% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.68%. Comparing base (d25ffaa) to head (19ff841).

Files with missing lines	Patch %	Lines
...on/optimizer/src/decorrelate_predicate_subquery.rs	72.81%	0 Missing and 28 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25002      +/-   ##
==========================================
- Coverage   81.69%   81.68%   -0.01%     
==========================================
  Files        1126     1126              
  Lines      415193   415296     +103     
  Branches   415193   415296     +103     
==========================================
+ Hits       339182   339252      +70     
- Misses      56079    56082       +3     
- Partials    19932    19962      +30     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kumarUjjawal]

Hi @ryux1 I think @mohammadnaqvi04 was already planning to work on this and has a PR #25008

I will close this one. Please always coordinate with the original issue author before opening a PR to avoid duplicates.

[ryux1]

Understood—thanks for catching it. I missed #25008 when I checked for competing work. I’ll coordinate on the issue before starting similar work in DataFusion going forward.