Add optional net settlement of import/export within wall-clock windows (metric_net_settlement_window_minutes)

[piomar123]

Summary

Some tariffs don't bill every import and export separately. They net import against export within a fixed settlement window (each clock hour, or quarter hour) and only charge or pay for the net direction. The hourly balancing used for households with their own generation in Poland works this way. Predbat currently prices every 5-minute step independently, so on these tariffs it overvalues same-hour arbitrage and misreports cost.

This PR adds an opt-in apps.yaml setting:

  metric_net_settlement_window_minutes: 60   # 0 / absent = off (default)

When it's set, import and export in the same wall-clock window are netted before being priced. The net import is charged at the import rate, or the net export paid at the export rate. If a rate changes within the window, the winning direction's energy-weighted average rate is used. With the setting off, behaviour is bit-for-bit unchanged (see Testing).

What changes

• Both prediction engines (prediction.py hot loop and prediction_kernel.cpp) accumulate the window's import kWh/cost and export kWh/credit. They move metric to the window's settled value at every step, so every snapshot (final_metric at end_record, metric_time) is consistent. The setting is off by default, and when off the original pricing lines are executed unchanged.
• Only money is netted. Planned import/export kWh, carbon, standing charge, the car-charging premium and metric_keep stay physical. Export that bypasses the CT clamp into a car (car_energy_reported_load: false) is excluded from the netting, consistent with it already earning no export credit. The PV10 correction for an Intelligent dispatch assumed gone (the car's import priced at the rate nominal pays, not rate_max) is applied to the step's import cost before it enters the window, so it is netted like any other import.
• today_cost() nets the metered day per window, so cost_today_sofar and predbat.cost_today match a netted bill. A new attribute, cost_net_settlement_adjust, shows the difference from gross; cost_import/cost_export stay gross. predbat.cost_hour is a rolling 60 minutes spanning two windows, so it deliberately stays gross.
• Current window carry-over: today_cost() hands the in-progress window's metered totals to the prediction as net_settlement_seed (a NetSettlementSeed namedtuple; the debug dump writes it as a plain list and net_settlement_seed_from() rebuilds it on replay), so the forecast part of the hour nets against what has already happened. Both engines accept the seed only when its window id matches minutes_now // window. The seed is cleared in the paths that zero cost_today_sofar: compare.py, annual.py reset_sample_state and the yesterday simulation in output.py. fetch_sensor_data() also resets it every cycle.
• Validation (Fetch.fetch_net_settlement_config(), utils.net_settlement_window_from_arg()): the window must be a multiple of PREDICT_STEP that divides 1440, so the plan's windows (which run on past midnight) and today_cost's (which restart at midnight) line up. The raw value is validated, so 60.5 is rejected rather than truncated to a valid-looking 60. Anything invalid disables netting instead of raising: the Warn: is logged once per change of the setting, and predbat.status is flagged (had_errors) every cycle, like other configuration errors. The key is also added to APPS_SCHEMA.
• Shared formula: utils.net_settlement_value(), mirrored by the inline pk_net_settlement_value() in the kernel.
• Kernel ABI 7 → 8, parity 16 → 17. PkContext gains metric_net_settlement_window and the net_seed_* fields at the end.
• Docs: new "Net settlement of import and export" section in docs/energy-rates.md (including the plan-slot display, Energy Comparison, and the per-slot rate heuristics such as iBoost and car slots that still use gross rates), the setting listed in docs/apps-yaml.md, and cost_today vs the gross cost_today_import/cost_today_export in docs/output-data.md.

Kernel binaries

This PR comes from a fork, and the kernel-binaries job only runs for branches in this repository. So the six prediction_kernel_lib_*.so files are rebuilt in a separate commit. They were built with build_kernel_cross.sh and the same zig version the workflow pins (0.16.0), with no other changes. Feel free to drop that commit and let CI rebuild them if you prefer.

Testing

New tests/test_net_settlement_config.py: parsing and validation of the setting, logging once per change, predbat.status flagged on every invalid cycle, seed conversion, the debug dump round trip, and reset_sample_state dropping the seed.

Test additions in tests/test_kernel_parity.py, all wired into run_edge_case_tests so verify_kernel_binary.py runs them:

• Hand-computed expectations for a pinned, lossless battery (grid flow = load − PV), written independently of net_settlement_value:
  • windows of 15, 30 and 60 minutes, with flat rates and with rates changing every 15 minutes (averaging within a window);
  • a seeded current window;
  • clock offsets aligned and not aligned to the hour.
• Physical outputs unchanged: netting leaves every kWh and carbon output identical, and only final_metric moves.
• today_cost(): netted day cost for 15/30/60-minute windows matches a hand calculation; window 0 stays exactly gross; the seed equals the current window's metered totals; there's no seed at the top of the hour.
• Python/C++ parity edge cases, each under the nominal, pv10 and pv90 scenarios:
  • windows of 15, 30, 45, 1440 and 7 minutes (7 is rejected by config but still exercised by the engines);
  • windows mis-aligned with the clock;
  • end_record falling mid-window, and end_record = 0;
  • negative export rates;
  • IOG pv10 worst-case import rate;
  • car bypass and car premium;
  • an Intelligent dispatch assumed gone in PV10, with the car's load reported and not reported;
  • carbon plus metric_keep;
  • a seed from another window, and a seed with netting off (both must be ignored).
• Random sweep: each of the 150 seeds gets one extra netted run, 50% of them with a random seed. It draws from its own generator, so existing seeded scenarios are unchanged.
• Snapshot hygiene: the new tests snapshot and restore SCENARIO_STATE_ATTRS (which now includes the two new attributes), so later suites see the same fixture state as before.

Results:

• This branch (current main + patch), aarch64 (Raspberry Pi 4) in the predbat_addon image, the same way CI runs it: unit_test.py --quick with PREDBAT_KERNEL_REQUIRED=1 passed all 333 test groups, and verify_kernel_binary.py passed on the rebuilt prediction_kernel_lib_aarch64.so. I couldn't run the x86_64 binary locally; CI checks that one.
• Off means unchanged: 450 seeded random scenarios (the parity sweep's 150 seeds × nominal/pv10/pv90) were run through unpatched current main (with its shipped aarch64 kernel) and through this branch with the window at 0. The dumped result tuples, including per-step SoC, were byte-identical on all three paths: Python engine, save="best" (including metric_time) and kernel. As a control, the same run with the window at 60 differs in 420-427 of the 450 scenarios, depending on the path. (The first version of this patch passed the same check against v9.1.0, plus a full ./run_all with identical pass/fail, 377/377, with and without it.)
• Mutation checks: making the Python engine ignore the seed, or skipping the today_cost adjustment, makes the new tests fail. So does netting the gross import cost instead of the car-gone-corrected one (net60_car_dispatch_gone fails parity).
• In production: I've been running this (60-minute windows) on my own system since 22 September, first on v9.1.0 and on v9.3.4 since 3 October. It does what it's meant to do: the plan no longer schedules same-hour import/export pairs that net to nothing under hourly settlement. I can't compare against a bill yet, since I'm billed half-yearly.
• Replayed data: two live debug dumps replayed with the window at 0 and at 60. On my tariff the optimiser's choice moves only a little (about 0.6 kWh of same-hour import/export per plan), so the main visible effect is correct cost reporting. The benefit grows with same-hour mixing.

Known limitations

• A window only nets in-window flows. It doesn't try to model any settlement rule beyond "net per window, price the winning direction at its window rate".
• The Python fast mode (step > 5 min, only used when the kernel is unavailable) assigns each whole step to the window it starts in.
• I didn't have the GitNexus tooling mentioned in CLAUDE.md. Instead I checked the blast radius manually: all run_prediction callers (plan.py, compare.py, marginal.py, output.py, annual.py) and every site that zeroes cost_today_sofar.

🤖 Generated with Claude Code