Fix pandas conversion of dynamic data with heterogeneous properties

[jhonabreul]

Description

QuantBook.UniverseHistory with flatten=True fails when the universe constituents are DynamicData instances that don't all carry the same properties, e.g. per-security rows carrying security-specific factors plus a market-wide row carrying a different one:

ValueError: Length of values (1) does not match length of index (504)

The same call without flatten works, one row per day with the constituent objects in the cell:

symbol                        time
MY-UNIVERSE.MyFactorsData 2S  2026-08-01    [MyFactorsData: AAPL, MyFactorsData: SPY, MyF...
                              2026-08-02    [MyFactorsData: AAPL, MyFactorsData: SPY, MyF...

With the fix, flatten=True returns one row per constituent, with NaN for the properties a row doesn't carry:

                              factor_a  market_factor
time       symbol
2026-08-01 AAPL R735QTJ8XC9X  0.019643            NaN
           SPY R735QTJ8XC9X  -0.004292            NaN
           $MARKET 2S              NaN           3.78
2026-08-02 AAPL R735QTJ8XC9X  0.013058            NaN
...

Cause: DynamicData columns come from each instance's storage dictionary, so data points can contribute different column sets, which the pandas conversion assumed were homogeneous.

The fix:

• PandasData.ToPandasDataFrame(IEnumerable<PandasData>, ...) back-fills missing values for the symbols that don't have a given series, so every column stays aligned with the symbol index and missing values become NaN.
• PandasData.Add creates the series for a dynamic property on demand when it first appears in a later data point, instead of throwing ArgumentException: <name> key does not exist in series dictionary.

Related Issue

N/A

Motivation and Context

Universe and history requests for dynamic data types (custom C# DynamicData and PythonData) with per-row properties should produce NaN for the properties a row doesn't carry instead of crashing.

Requires Documentation Change

No

How Has This Been Tested?

• PandasConverterTests.FlattensBaseDataCollectionOfDynamicDataWithHeterogeneousProperties: flattens a universe collection with 3 constituents carrying different properties. Asserts row count, index names and NaN placement. Reproduces the ValueError without the fix.
• PandasConverterTests.HandlesDynamicDataWithPropertiesAddedInLaterDataPoints: a property first appears in the second data point of a series. Asserts values and NaN for the first point. Reproduces the ArgumentException without the fix.
• End-to-end QuantBook.UniverseHistory(universe, start, end, flatten=True) over a PythonData universe with 504 heterogeneous constituents per day: reproduces the exact reported error without the fix, returns the expected flattened frame with it.
• Regression suites (PandasConverter*, QuantBook*, PythonData, AlgorithmHistoryTests, AlgorithmChainsTests): 1698 passed, 0 failed.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which improves implementation)
• Performance (non-breaking change which improves performance. Please add associated performance test and results)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Non-functional change (xml comments/documentation/etc)

Checklist:

• My code follows the code style of this project.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>