Skip to content

fix(ev): preserve confirmed power history and verify charging recovery - #1675

Open
segran2 wants to merge 11 commits into
srcfl:masterfrom
segran2:diagnose/ev-import-stop-1674
Open

segran2 wants to merge 11 commits into
srcfl:masterfrom
segran2:diagnose/ev-import-stop-1674

Conversation

@segran2

@segran2 segran2 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem and result

Refs #1674. An Easee installation on v0.145.2-beta.1 repeatedly lost its EV offer under a configured import ceiling and could remain at zero measured charging after a positive offer was restored. The UI could show only a main-fuse limitation, obscuring the missing response.

This PR fixes the control evidence and recovery paths:

  • Retain fresh confirmations of change-only power samples in the bounded 64-point observation buffer. Update only the confirmation time of an identical source sample. Preserve original measurement time, observed stops and invalidation on missing/unconfirmed data; conflicting power values at one source time invalidate proof.
  • Expose confirmed intervals to time-aligned EV and battery headroom bounds through ConfirmedControlWindows. Independent response verification keeps using source-only ControlWindows; confirmations never manufacture distinct physical samples or usable response windows. Frozen copies cannot carry confirmation time beyond their cutoff.
  • End recovery after FTW's own zero offer only after ten seconds of fresh, advancing meaningful power observations. The threshold is 90% of the lower of the configured minimum and actual offer, allowing nominal-voltage variation and phase changes. A 130/800 W startup pulse or one high sample no longer clears recovery. The shared 100 W DeliveringW constant is unchanged. Existing session/device/generation, safety and plan gates remain authoritative; optional resumes stay bounded to three acknowledgements at 30-second spacing and are suppressed while meaningful delivery is being observed.
  • Keep historical EV brackets attached to their historical site observation. Retain already-qualified latest-site EV credit when no common bracket exists. Without positive eligible EV power, avoid retaining an older high grid sample from before a stop. Use a preceding fresh battery observation in the latest fallback while retaining conservative missing-history treatment for observations after the site read and reserving pending orders/loss of discharge.
  • Report sustained accepted positive offers with zero delivery as not-following/not-drawing after confirmation grace, and describe the shared clamp as site fuse/import limits. Retain rate-limited DEBUG evidence for below-minimum budget decisions; retain the operational warning when acknowledged resume attempts do not produce charging.

Evidence and limits

The previous personal revisions did not establish stable charging. Revision 3 was installed and checksum-verified, but still stopped. At 02:19 CEST household demand actually rose from about 0.9 to 2 kW, taking grid import to about 8.15 kW under a 7.5 kW cap. A reduction was justified. The later zero-cap decisions combined high grid power with little/no qualified EV credit.

Focused probes found a further reproducible defect: a 6183 W source measurement one minute old with a confirmation one second old was accepted as current, yet its observation ring was empty. The new integration regression covers a steady confirmed EV followed by a rate decrease. A separate probe showed a 130 W pulse deleting resume evidence despite a 4140 W charging minimum. Field history includes such pulses.

These establish code behavior; exact raw source/confirmation timestamps at every field stop were not retained. The complete field cause is not claimed proven. Cable disconnect/reconnect, rather than demonstrated software recovery, restarted the user's charge. With the import cap later disabled, charging reached 10.28 kW after another cable reconnect. The revision 4 controlled cap/stop/recovery field test below has now passed; longer operation and a separate real household load-step test remain outstanding.

No fuse/import ceiling, freshness threshold, phase rule or charging step is raised or removed. Requested power is never treated as measured power. No driver snapshot, Start/Stop authorization policy or automatic reboot is changed.

Verification

  • Before/after regression evidence for confirmed-anchor loss, rate-change credit, startup-pulse recovery and the earlier common-budget/fallback defects.
  • Telemetry, control, loadpoint and API suites pass. Focused regressions repeated 20 times pass.
  • Tests preserve source times and one physical point per sample, missing/conflicting-data invalidation, observed zero readings, frozen cutoffs, and separation of confirmed intervals from independent response evidence.
  • Recovery tests cover 130/800 W pulses, a brief high sample, cached/unavailable power, sustained charging, nominal-voltage tolerance and one-phase offers. Existing session, safety, plan-stop, retry-bound and dispatch tests pass.
  • Full make verify passes on the PR and cumulative v0.145.2-beta.1 personal source. Linux ARM64 build succeeds. No new web source changes in this revision; GitHub web checks run separately.
  • Signed source head: 8a02c0a. Personal revision 4 includes the earlier EV-cost/history patches. It is installed on the affected RPi; executable checksum verified, service restarted at 03:08:08 CEST on 17 October 2026.
  • The earlier CI timing failure in the streaming-phone test was corrected to wait for new windows and an advanced revision under the same delivery deadlines. Production code was unchanged by that test-only commit.

Scope and review

Checked open PRs for the newly changed observation/recovery/headroom paths: no other open branch overlaps them. #1537 overlaps the earlier feedback classifier/UI changes; preserve its load-balancer reporting on rebase. #888 has right of way for its dispatch.go PV-export work; this PR adds only a private diagnostic throttle field there. No other branch was rewritten or reviewer requested.

  • Regression tests, patch changesets and DCO sign-offs included.
  • Full local verification on both source trees.
  • Controlled revision 4 hardware test: 7,500 W cap, intentional stop at 3,000 W, and automatic restart after removing the cap, with cable connected.
  • Longer charging stability and a separate real household load-step test.
  • Human visual review of earlier UI wording/status changes. Local fixture opening was denied by the browser permission check; PR remains draft.

Revision 4 validation update: all executed GitHub checks on 8a02c0a pass, including core (Go) and go test + vet in 38988812967. Final focused regressions pass 20 repeats. The ARM64 revision 4 package is copied to the RPi, extracted and checksum-verified. It is now installed and checksum-verified. The controlled hardware test below passed; broader validation remains pending.

Revision 4 installed: controlled stop/recovery test — 17 October 2026 (CEST)

The personal cumulative v0.145.2-beta.1 ARM64 revision 4, derived from 8a02c0a, is now installed. The running executable SHA-256 matches the revision 4 manifest. FTW restarted at 03:08:08.

Observed on the affected Easee/Pixii installation:

  • Before the test, with the import cap disabled: 10.315 kW measured charging, 15 A offered, at 03:09:54.
  • With 7,500 W import cap: at 03:13:20, measured EV 6.209 kW, offer 9 A / 6,210 W, grid 7.058 kW, phase currents 9.85 / 11.16 / 9.88 A. Initial offers varied between 6–10 A, then held 9 A for approximately the last minute. No zero offer was recorded in that cap-on interval.
  • With 3,000 W cap: at 03:14:02, FTW offered 0 A, actual EV 0 W, grid 851 W. Easee reason 52, max dynamic charger current too low, was consistent with the intentional zero offer. The below-minimum diagnostic reported approximately 2.16 kW available against the 4.14 kW charging minimum.
  • After the user disabled the cap, without disconnecting the cable or manually sending Start/Resume: at 03:15:19, offer 15 A / 10,350 W, actual 10.328 kW, charging mode 3. Reason 78 was no longer reported. At 03:16:49, charging remained 10.328 kW, status Following, with no new stop logged in that follow-up interval.

This controlled cap-reduction / below-minimum stop / automatic-recovery scenario passed. It does not establish long-term stability, recovery from every charger state, or correct response to a separate real household load step. The observations do not isolate which individual revision 4 change caused the improved result or establish whether the bounded resume retry path was needed. Human review of the earlier UI changes also remains pending.

Leave the issue open and PR draft. The user has no current peak-tariff requirement, so the import ceiling is disabled for normal charging; configured 25 A per-phase protection remains active. Next validation is a longer charging period and inspection of history around these transitions. History coverage/invalid_balance work remains separately tracked in #1667/#1668; this charging test does not verify that fix.

Logging cleanup on 15847d0: detailed budget arithmetic is DEBUG (still limited to once per minute); exhausted-recovery warnings remain WARN. The existing incident regression explicitly checks DEBUG-level output. Full make verify passes (tests, vet and build). The installed personal revision 4 retains its INFO diagnostics for ongoing investigation; no new package was installed. The longer field/history findings are recorded in the follow-up comment; no claim of interruption-free stability or elimination of invalid_balance is made.

Signed-off-by: Segran <goran@andersson-stehag.se>
…t most once a minute

Signed-off-by: Segran <goran@andersson-stehag.se>
Signed-off-by: Segran <goran@andersson-stehag.se>
@segran2 segran2 changed the title diagnostics(control): capture EV import-limit stop evidence fix(ev): confirm resumed delivery and expose stalled limited charging Oct 16, 2026
Signed-off-by: Segran <goran@andersson-stehag.se>
@segran2 segran2 changed the title fix(ev): confirm resumed delivery and expose stalled limited charging fix(ev): align fuse-budget measurements and confirm resumed delivery Oct 16, 2026
Signed-off-by: Segran <goran@andersson-stehag.se>
@segran2
segran2 marked this pull request as ready for review October 17, 2026 00:00
@segran2
segran2 marked this pull request as draft October 17, 2026 00:00
Signed-off-by: Segran <goran@andersson-stehag.se>
Signed-off-by: Segran <goran@andersson-stehag.se>
@segran2 segran2 changed the title fix(ev): align fuse-budget measurements and confirm resumed delivery fix(ev): preserve confirmed power history and verify charging recovery Oct 17, 2026
@segran2

segran2 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

Longer field/history check — 17 October 2026, 12:27 CEST

The car now reports the requested 80% SoC (vehicle reading about 105 minutes old at inspection), and retained session delivery is approximately 42.73 kWh. Current command and actual charging are both zero; the command reason is no_plan_budget. The service has continued running since revision 4 installation. No exhausted-resume warning was found in the inspected post-test journal.

This is not a continuous, interruption-free charging result. At 05:49:15, with the import ceiling disabled, the controller issued a below-minimum zero offer: grid 12,323 W, eligible EV credit 0 W, phase bound approximately 2,449.5 W, minimum 4,140 W. Aggregated history shows EV zero from 05:49:25 through 05:50:25 and positive power again at 05:50:40 (10.230 kW by 05:50:45). This documents later recovery in telemetry; it does not independently establish whether any user action occurred or whether that zero decision was physically justified. Other short zero/low-power intervals also exist. The initial controlled test remains passed, but long-term stability and that later decision's raw alignment/phase evidence still need investigation.

History API review (range=12h&points=4320, 10-second aggregated rows):

  • 03:13:35–03:16:50, covering the controlled stop/restart: 20 returned rows, all complete.
  • 03:11–03:17, including the earlier rate reduction: 35 complete rows and one balance_mismatch / invalid_balance row.
  • 03:17–07:15: 1,428 returned rows, 14 with balance_mismatch, the remainder complete. These are aggregate rows, not counts of original failed measurements; averaged columns and last embedded driver values must not be treated as simultaneous raw samples.
  • The daily API still provides a non-null estimated household total for 17 October (15.268 kWh at this inspection), recorded coverage 44,855,239 ms against elapsed period 44,871,653 ms (about 16.4 seconds uncovered). 16 October remains estimated at 33.461 kWh with reported full-day coverage.

Thus #1668's deployed daily estimate is present while rejected instantaneous rows remain visible; live balance failures are not eliminated. This API inspection does not independently audit every estimate-table entry, exact missing interval or database provenance. No database repair or hardware/configuration change was made in this check.

Signed-off-by: Segran <goran@andersson-stehag.se>
Signed-off-by: Segran <goran@andersson-stehag.se>
@segran2

segran2 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

05:49 zero-credit investigation and remaining uncertainty

The retained per-driver series adds an important ordering detail. The Easee ev_w 10-second bucket has two samples, first receipt 05:49:13.164 and last receipt 05:49:18.275, min 0 W, max/last 10,241.9996 W. Thus the first sample was zero before the FTW zero-offer decision at 05:49:15.240; ev_dynamic_current_a was 15 A at both receipts. The preceding power sample at 05:49:08.117 was 10,241.9996 W. These timestamps are local recording/receipt times, not original Easee physical sample times.

Pixii's first receipt at 05:49:14.910 recorded 12,323 W and phase currents 20.45 / 17.05 / 15.87 A. The decision log used zero eligible EV credit and a 2,449.5 W phase bound, below the 4,140 W minimum. No sampled phase here exceeded the configured 25 A. This explains the conservative stop arithmetic; it does not prove a charger fault, physical pause or delayed/out-of-order cloud response. The zero was received from the Easee driver, not vehicle SoC telemetry.

Add per-EV evidence to the bounded DEBUG budget record using the same reading considered by the calculation: raw/control power, source time, receipt time, confirmation time, source validity, qualification, driver telemetry health, retained-point count and latest-site credit. Reasons distinguish observed nonpositive power, invalid/unavailable source, stale/future receipts or control observations, offline telemetry, and absent brackets/observed stops. No requested power is used as measured credit, and zero observations are not ignored.

This closes a diagnostic gap, not the uncertainty in the historical event or the runtime stop mechanism. Preventing that stop by retaining a previous positive EV value without contemporaneous proof could authorize excess current after a real EV stop plus household increase. No such hold/filter/debounce is introduced. A future recurrence must capture the new evidence with DEBUG enabled in a diagnostic build before a prevention fix can be justified. The currently installed personal revision 4 has not been replaced by this code.

Implemented on cde5cdb in #1675. Regression tests retain the declared source timestamp even when a source is unavailable, distinguish confirmed zero from unavailable data, and retain the below-minimum safety verdict. Full make verify passes (tests, vet and build). Hardware validation of this diagnostic addition is pending.

Signed-off-by: Segran <goran@andersson-stehag.se>
@segran2

segran2 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

CI telemetry-boundary correction

The previous cde5cdb head passed Go, race, web, full-stack, vet and build checks, but the direct cached-telemetry read ratchet rejected the new diagnostics (ev_fuse_budget.go 8→9 reads, ev_fuse_evidence.go 0→3). The macOS matrix job was cancelled after the Ubuntu failure.

Fixed in 4cae2e3: diagnostic snapshot parsing lives in go/internal/telemetry, explicitly exposes qualification/health for rejected readings and never authorizes dispatch. Control keeps only budget-specific evidence fields and logs the latest grid value captured by that same calculation, without rereading cached raw power for the diagnostic line. No exception or threshold change was added, and charging/recovery behaviour is unchanged.

Full local make verify and the exact upstream telemetry-read guard against base 3789764 pass. The new Ubuntu telemetry guard has passed on GitHub; the remaining jobs are still running. Installed personal revision 5 does not need replacement for this source-organisation change.

@frahlg

frahlg commented Oct 17, 2026

Copy link
Copy Markdown
Member

Thanks, Göran, for the careful work and the clear evidence. We read the stops from the code side. The cause sits in our control design, not in Easee or the car.

What happens. Since #1657, EVFuseBudget sets the EV cap on every tick as limit − grid + proven EV power. EV power counts as proven only when Easee samples bracket the meter reading. Easee reports through the cloud, only on change, seconds late, and now and then sends a lone 0 W. When the proof lapses, the budget counts the EV's own current as house load. The offer goes to 0, the "house load" disappears, the offer comes back, and the loop repeats. Since #1672 the reactive per-phase guard (nextFusePhaseCapA) steps aside whenever that budget exists, so nothing else holds the offer.

The 05:49 stop matches the formula. One 0 W sample gave zero EV credit: (25 − 1 − 20.45 A) × 230 V × 3 = 2,449.5 W, your logged phase bound, below the 4,140 W step. The meter showed every phase under 24 A. If the EV kept drawing, as the next sample and the meter total suggest, the house drew about 5.6 / 2.2 / 1.0 A. The 23:45 cycling under the 7.5 kW ceiling has the same shape.

Why more bookkeeping will not close it. Your agent reached the right conclusion: in this model the only way to keep the offer is to trust a stale EV value, which is unsafe. So the model has to change. A rule that needs proof from a slow sensor before it may keep an offer will stop the car each time that proof lapses.

Direction. The hard limit reads only the site meter and our own commands. It cuts at once by the measured overage, holds while inside the limit, and steps up only when the meter shows room and the last step has landed. Slow device power, from Easee or any other device, never enters the safety path. It may still feed status and the plan. We are building this as a separate draft PR on master and will link it here.

From this PR, please keep the resume check that waits for sustained delivery with bounded retries, the not-following status, and the diagnostics. The new guard will likely replace the confirmed-anchor and bracket changes in ev_fuse_budget.go and control_observations.go, so it may save you time to hold further work there until the new PR is up.

@segran2

segran2 commented Oct 17, 2026 via email

Copy link
Copy Markdown
Contributor Author

frahlg added a commit that referenced this pull request Oct 17, 2026
Fredrik's rule (17 Oct): never trust slow current values from an Easee
or any other device. The EV hard limit, the split between cars, the
EV/battery split and the #1536 allocator now read only the site meter
and FTW's own recent commands, kept for the 20 s a command takes to
reach the meter.

A rise needs a fresh reading with room for two steps and waits until
the meter shows the last change (import rose with it, or 20 s passed).
Every increase leaves a step free. An idle pool restarts only after 35 s
of measured room; a meter gap starts the count again, and a silent
meter's repeated reading never acts twice. Battery commands, not
readings, set the battery's baseline and pending range.

Removes the bracketed EV and battery credit and ev_pending_battery.go.
Overlaps #1675.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
frahlg added a commit that referenced this pull request Oct 17, 2026
…mands (#1674) (#1695)

* fix(control): the EV hard limit follows the site meter (#1674)

The EV hard cap credited a car only when its own readings bracketed a
site reading. Easee Cloud reports late and on change only, so missing
proof or one 0 W sample counted the car as house load and stopped the
charge (05:49: 2,449.5 W of credit at 20.45 A on a 25 A fuse), and an
import ceiling made the charger start and stop every half minute.

EVFuseBudget now cuts by the overage the meter measures on the worst
phase or in total import, taken from the highest offer the meter may
still contain. It holds inside the limits, rises one 1 A step when the
meter shows room and the cars have taken the last step, and restarts an
idle pool only with room for its minimum plus a step. EV readings only
gate a rise. In the loadpoint split, keeping an offer on the same
phases is no longer an increase, and the phase gate no longer cuts.

The bracket credit for the EV cap, the common battery reserve and the
dead reactive phase clamp are removed. Overlaps #1675.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>

* fix(control): the EV guard trusts only the meter and our own commands

Fredrik's rule (17 Oct): never trust slow current values from an Easee
or any other device. The EV hard limit, the split between cars, the
EV/battery split and the #1536 allocator now read only the site meter
and FTW's own recent commands, kept for the 20 s a command takes to
reach the meter.

A rise needs a fresh reading with room for two steps and waits until
the meter shows the last change (import rose with it, or 20 s passed).
Every increase leaves a step free. An idle pool restarts only after 35 s
of measured room; a meter gap starts the count again, and a silent
meter's repeated reading never acts twice. Battery commands, not
readings, set the battery's baseline and pending range.

Removes the bracketed EV and battery credit and ev_pending_battery.go.
Overlaps #1675.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>

* fix(control): keep our last offer across a restart; battery holds its order

Core restarts on every update. The pool counted a charging car as idle
on the first tick and, at 05:49 currents (2,449.5 W of room), stopped it.
FTW now saves its own last order per charger, keyed by hardware
identity, and restores it as the hold base while the same plug-in
session is connected: a matching session id, or a session energy
counter that has not gone back, within 10 minutes.

A commands-only battery counted its own repeated order twice and
hunted. Keeping a battery's latest order is now never an increase;
a measured shortfall takes charge back and new charge needs room after
our unseen increases. A command log younger than the settle time
counts the time before it as 0, so a new EV offer is not taken as
already in the meter.

The EV limit reads the meter once (telemetry-read check 8 -> 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>

---------

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit that referenced this pull request Oct 17, 2026
…1674)

The import ceiling acted every second. Under a 7.5 kW ceiling a short
house peak cut the car, and the 23:45 case held at 5,520 W. Swedish power
tariffs bill the mean import over each clock hour, the hours the
planner's demand charges use.

The control tick now adds the site meter's import to the local clock
hour. The hour's limit is the budget left over the time left; the EV
guard's cut, the battery fuse guard and the joint allocator use it. The
fuse stays an instant limit. Increases reach only a slower level: whole
steps with one in hand, at most once a minute, five minutes after a cut
and not in the hour's last quarter. A car starts only with room under the
ceiling itself, so it never starts on slack only to stop at the next
hour. A meter gap counts at no less than the hour's rate and never raises
a limit. At boot the hour's import comes from state.db. /api/status shows
the hour's budget left and the limit in force.

Overlaps #1675.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
frahlg added a commit that referenced this pull request Oct 17, 2026
…1674) (#1699)

* fix(control): the import ceiling caps each clock hour's mean import (#1674)

The import ceiling acted every second. Under a 7.5 kW ceiling a short
house peak cut the car, and the 23:45 case held at 5,520 W. Swedish power
tariffs bill the mean import over each clock hour, the hours the
planner's demand charges use.

The control tick now adds the site meter's import to the local clock
hour. The hour's limit is the budget left over the time left; the EV
guard's cut, the battery fuse guard and the joint allocator use it. The
fuse stays an instant limit. Increases reach only a slower level: whole
steps with one in hand, at most once a minute, five minutes after a cut
and not in the hour's last quarter. A car starts only with room under the
ceiling itself, so it never starts on slack only to stop at the next
hour. A meter gap counts at no less than the hour's rate and never raises
a limit. At boot the hour's import comes from state.db. /api/status shows
the hour's budget left and the limit in force.

Overlaps #1675.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>

* fix(control): one reading over the hour's ceiling no longer stops the car (#1674)

Göran, 17 October 14:27:59 UTC: with the car at its 4,140 W minimum beside
3.3 kW of house load, one reading 24 W over the 7.5 kW ceiling stopped it,
and the room left (4.2 kW) never met the minimum plus a step again.

The EV guard now keeps the fuse's part apart from the ceiling's part. The
fuse cuts on every reading. The ceiling cuts at once on an overage of a
step or more, after a meter gap, or while the hour is not measured; a
smaller overage must last 20 s. A restart judges the ceiling on the mean
import over the 35 s wait and needs room for the minimum only, under the
rise level and at most the ceiling. A ceiling set during the hour counts
from then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>

* fix(control): the hour's import reads the site meter through a health gate

The telemetry-read guard counted two direct reads of the cached meter in
import_hour.go. Store.OnlineMeterW returns the site meter's power and
arrival time only while its driver is still reporting, so a
watchdog-offline meter's last value never counts as a new reading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>

---------

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants