[Enhancement] Make the flat_object read depth cap configurable

[RyanL1997]

Description

A flat_object was read to a fixed depth of 20 (FlatObjectValues.MAX_DEPTH); past it, the object that would have been descended into is kept as one leaf holding its JSON. As requested on #5830, the cap is now a setting:

• plugins.query.flat_object.max_depth: node scope, dynamic, default 20 (the default of index.mapping.depth.limit), minimum 1.

A pushed-down script flattens the field on the data node, where no query setting reaches it. So the value is held as a static in FlatObjectValues that every node keeps in step with the cluster setting, written at startup and on each update in SQLPlugin.createComponents. This is the pattern OpenSearch's SearchService uses for indices.query.bool.max_clause_count. The engine's read and a pushed-down script therefore apply the same cap.

Testing

• FlatObjectValuesTest: the cap moves both ways (30 reaches a 25-deep leaf, 3 cuts at 3). OpenSearchSettingsTest: default, dynamic, registered.
• New CalciteFlatObjectIT (also in CalciteNoPushdownIT), over a 25-deep document added to flat_object.json: at the default, the leaf reads as null and a script filter on it finds nothing; raised to 30, both find it. With the SQLPlugin hookup removed, the raised-depth case fails, so the IT covers the wiring.
• CalciteExplainIT and the full CalciteNoPushdownIT pass (0 failures).

Related Issues

Resolves #5842

Check List

• New functionality includes testing.
• New functionality has been documented.
• New functionality has javadoc added.
• New functionality has a user manual doc added.
• New PPL command checklist all confirmed.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff or -s.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Thread Safety The maxDepth field is declared volatile but is read twice in the fold method (line 144): once in the condition check and once implicitly when passed to flattenInto. Between these reads, another thread could update maxDepth via setMaxDepth, causing the depth check and the actual recursion to use different values. This could lead to inconsistent flattening behavior where the depth guard is bypassed or applied incorrectly during concurrent updates. if (value.isObject() && depth < maxDepth) { flattenInto(value, key, out, depth + 1); return;

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Use consistent settings source for initialization The initial setting read uses clusterService.getSettings() while the update consumer is registered with clusterSettings. This inconsistency could lead to the initial value being read from node-local settings instead of cluster settings, causing a mismatch between nodes in the cluster. plugin/src/main/java/org/opensearch/sql/plugin/SQLPlugin.java [443-448] FlatObjectValues.setMaxDepth( - OpenSearchSettings.FLAT_OBJECT_MAX_DEPTH_SETTING.get(clusterService.getSettings())); + clusterService.getClusterSettings().get(OpenSearchSettings.FLAT_OBJECT_MAX_DEPTH_SETTING)); clusterService .getClusterSettings() .addSettingsUpdateConsumer( OpenSearchSettings.FLAT_OBJECT_MAX_DEPTH_SETTING, FlatObjectValues::setMaxDepth); Suggestion importance[1-10]: 8 __ Why: This is a valid concern about consistency. Using clusterService.getSettings() for initialization and clusterService.getClusterSettings() for updates could lead to reading from different sources (node-local vs cluster settings), potentially causing inconsistency across nodes in a cluster.	Medium

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.79%. Comparing base (ccb58da) to head (9185302).

❌ Your project check has failed because the head coverage (63.79%) is below the target coverage (99.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #5861   +/-   ##
=========================================
  Coverage     63.78%   63.79%           
- Complexity     9063     9064    +1     
=========================================
  Files           962      962           
  Lines         40974    40982    +8     
  Branches       4653     4653           
=========================================
+ Hits          26136    26144    +8     
  Misses        13962    13962           
  Partials        876      876           

Flag	Coverage Δ
sql-engine	63.79% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.