Skip to content

feat(stats): restore GitHub history and track FTW Cloud energy usage - #50

Merged
frahlg merged 17 commits into
mainfrom
agent/project-stats-dashboard
Oct 17, 2026
Merged

frahlg merged 17 commits into
mainfrom
agent/project-stats-dashboard

Conversation

@frahlg

@frahlg frahlg commented Aug 14, 2026 •

Copy link
Copy Markdown
Member

Update stats.ftw.energy for Core, drivers, Cloud, the native app, web app and website. Restore saved GitHub charts after the repository set grew from three to six. Preserve per-point coverage and break star lines when the set changes, so adding repos does not look like growth. Include beta releases when selecting the latest published release.

Use the public Cloud daily-report feed to estimate active reporting installations: 156 reports over the seven complete UTC days 10–16 October gives about 22. This is not unique people or all-time installations. Missing/withheld days hide the estimate; opt-outs, offline boxes, failed sends, restarts and test boxes can bias it. Replace overlapping imported relay history rather than adding it. Keep the retired relay archive clearly labelled.

Add an hourly, bounded collector for public Cloud usage cells, strict schema/cell checks, D1 migrations and 90-day retention. The page can show configured battery/charger power, battery and solar capacity, generation and recorded grid/battery/vehicle energy. Missing, withheld and stale values stay distinct from zero. Configured limits do not prove physical control. Household hourly collection remains off pending privacy review and policy activation; display that as a waiting state, not a fake zero or collector failure. The updated terms include statistics for every Cloud user; local FTW has a separate Yes/No question with Yes suggested and no upload before an answer.

Private diagnostics require Cloudflare Access plus Worker-side JWT checks. Public dimension labels each need the group minimum. Preserve the legacy signed ingest for compatibility; relay is retired and does not measure current Cloud access.

Validation after merging current main: type checks, 1,133 app tests (two skipped), 24 Worker tests, app build and Worker dry run pass. Resolve the relay type-import conflict by keeping both main's existing Watch exchange and aggregate-export types. Browser review confirms six repos, restored charts, the live estimate and paused energy collection. The deployed Worker is version 04536a74-3012-4ba5-93a6-83691744d2dc from commit 64188dc; all three D1 migrations are applied.

Paired terms changes: srcfl/ftw#1700 (merged), srcfl/ftw-cloud#32, srcfl/ftw-app#87 and srcfl/ftw-web#23 (merged). The original Core/Cloud ingestion foundation is already merged and Cloud is deployed with hourly collection off. This PR does not activate collection or claim a signed DPIA.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ftw-webapp 9001b8d Commit Preview URL

Branch Preview URL
Oct 17 2026, 02:57 PM

@frahlg

frahlg commented Aug 16, 2026

Copy link
Copy Markdown
Member Author

Stable-release audit update: the reviewed privacy and retention fixes are now on this PR at head ddc7469.

The new head:

  • uses exactly 30 UTC dates including today;
  • hides each version and driver label until that label has at least 10 reports;
  • hides every label while the total is below 10;
  • prevents an older replay from replacing a newer daily aggregate;
  • prunes all stored source tables on their stated retention windows; and
  • safely counts labels such as __proto__ and constructor.

npm run verify is green: 884 app tests passed with 1 skipped, 17 stats tests passed, Svelte and TypeScript checks passed, and both app build and stats Worker dry-run passed.

Keep this PR in draft. The live stats Worker still runs the old head and was observed hiding the total while exposing low-count version and driver labels. Before merge or production deploy we still need:

  1. CI green on exact head ddc7469.
  2. Human browser review of the changed dashboard copy.
  3. A separate ftw-project-stats deploy from the reviewed SHA.
  4. Live checks for the low-count gates, Access on /api/admin, and a 401 response for unsigned ingest.

The threshold counts reports, not unique boxes; the UI and README now say that directly.

@frahlg frahlg changed the title feat(stats): add privacy-bounded project dashboard feat(stats): track the public FTW stack and Cloud transition Oct 17, 2026
@frahlg frahlg changed the title feat(stats): track the public FTW stack and Cloud transition feat(stats): restore GitHub history and track FTW Cloud energy usage Oct 17, 2026
@frahlg
frahlg marked this pull request as ready for review October 17, 2026 14:57
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T14:59:41.176909Z 9001b8d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@frahlg
frahlg merged commit cc9f2ad into main Oct 17, 2026
3 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9001b8d886

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread stats/src/dashboard.ts
} else if (estimate && typeof estimate.value === 'number') {
setText('overview-installations','≈ ' + number(estimate.value)); setText('installation-period',estimate.window_start + ' to ' + estimate.window_end + (estimate.state === 'stale' ? ' · stale source' : ''));
}
if (estimate && typeof estimate.value === 'number') setText('installation-method',estimate.reports + ' reports over ' + estimate.days + ' complete days. About one successful report per installation per day. Counts reporting active installations, not people or all-time installs; opt-outs, offline boxes, failed sends, restarts and test boxes affect the estimate. Source: ' + estimate.source + '.');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match the installation explanation to the displayed source

When both collectors have data, the branch above displays the 24-hour usage.installations value, but this unconditional assignment replaces its explanation with the unrelated seven-day fleet estimate methodology and source. The dashboard can therefore show one installation count while claiming it was derived from a different report set; only render this estimate explanation when the estimate is the value selected, and provide the usage-cell provenance otherwise.

Useful? React with 👍 / 👎.

Comment thread stats/src/dashboard.ts
Comment on lines +272 to +275
if (u.installations && u.installations.state === 'visible') {
setText('overview-installations', format(u.installations)); setText('installation-label','Reporting installations'); setText('installation-period','Opted in · last 24 closed hours');
} else if (estimate && typeof estimate.value === 'number') {
setText('overview-installations','≈ ' + number(estimate.value)); setText('installation-period',estimate.window_start + ' to ' + estimate.window_end + (estimate.state === 'stale' ? ' · stale source' : ''));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not prefer historical installations while usage is paused

If a usage snapshot was saved before the source began returning the documented disabled state, dashboardData retains that snapshot while setting u.state to paused; this condition still selects its historical installation cell and labels it as the current “last 24 closed hours” value, even when a fresh fleet estimate exists. Gate this branch on a connected, sufficiently fresh usage state so a pause cannot present an old count as current.

Useful? React with 👍 / 👎.

Comment thread stats/src/store.ts
Comment on lines +247 to +249
installation_estimate: installationEstimate(feed?.days ?? (fleetView['days'] as {date:string;reports:number|null}[] ?? []), nowMs, cloudFleet?.observed_at ?? freshness['fleet'] ?? null, feed ? 'cloud.ftw.energy' : 'retired relay archive'),
fleet_current: feed,
usage: { ...(usage ? JSON.parse(usage.view_json) as Record<string, unknown> : { minimum: 20 }), state: usageState },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply the configured threshold to the public Cloud feed

When PUBLIC_MIN_REPORTS is configured above 10, the legacy fleet view honors it, but /api/public also returns fleet_current verbatim from a feed whose parser fixes its minimum at 10. Counts and dimension labels between 10 and the configured threshold are consequently exposed through the public API, bypassing the operator-selected privacy floor; omit this raw feed or re-filter it using publicMinimum(env.PUBLIC_MIN_REPORTS).

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant