feat(statistics): require Cloud terms and allow local contribution

CLAUDE.md

@@ -32,8 +32,8 @@ maintains it.
   never from a body field. Every later box message but `tun` must verify
   with that key. `tun` is unsigned: its `d` is a phone's ciphertext, which
   the service routes by `dev` and `sid` and never reads.
-- **Live readings stay in memory.** The separate, versioned project statistics
-  consent permits rounded closed-hour energy and configured capacities on
+- **Live readings stay in memory.** The versioned project statistics
+  scope covers rounded closed-hour energy and configured capacities on
   disk for 13 months. Only group cells pass `/v1/stats`; no staff route reads
   a home. Withdrawal and home purge delete its unpublished statistics.
   Memory holds live readings, and only while the roster's

README.md

@@ -257,7 +257,7 @@ Not built yet:
 | Step | What | Where |
 |---|---|---|
 | C6 | Tiles | `internal/server` |
-| C7–C8 | Statistics | not started |
+| C7–C8 | Statistics | implemented with collection off; review pending |
 
 ## How it holds a home
 
@@ -384,42 +384,44 @@ The sender refuses redirects, proxies and local destinations.
 ## Project usage statistics
 
 `GET /v1/stats` publishes only group cells under
-`ftw-project-stats-2026-10-v1`. Boxes send signed `stats` hours only after the
-owner accepts that policy and the service advertises it in signed `hello_ok`.
-The service starts with collection off. After the signed C8 DPIA review,
-operators can set `FTW_CLOUD_STATISTICS_POLICY` to that exact policy in the
-host's compose environment; empty keeps it off. An unapproved policy makes
-`/v1/stats` return 503 with `FTW-Statistics-State: disabled`. Withdrawal,
-purge and retention remain active while collection is off. The review draft
-is in [privacy/project-statistics-dpia.md](privacy/project-statistics-dpia.md). The previous statistics policy sends no new
-fields. Reporting installations in the last 24 closed hours are not people
-or total FTW installs. Power and capacity are configured; energy sums
-recorded hours, including partial hours. Missing data is unknown.
-
-Cloud stores consented hourly values for 13 months, upserts reconnects, and
-deletes household rows on a signed withdrawal or purge. Cells need 20 homes
-and no home above 30 percent of the sum. The stats Worker polls this public
-view; it receives no home id or per-home value. One instance supplies its
-local shard only; public multi-shard totals need an explicit safe collector.
-
-Owner acceptance uses the existing box API. On the box's loopback interface,
-read `GET /api/cloud/stats-policy`, accept its terms, then
-`POST /api/cloud/stats` with `{"policy":"ftw-project-stats-2026-10-v1"}`.
-The readable Cloud grant must already be on. `DELETE /api/cloud/stats`
-withdraws this choice. Do not enable it for someone else or reuse an older
-consent. Native apps keep the old scope until their consent screen adopts
-this policy. The first contributing hour starts after consent; the sender
-does not upload earlier ledger history.
-
-Enabling this collection scope needs the signed C8 DPIA and the updated
-consent text described by Core's ADR 0008. A deployment with collection off
-does not claim that sign-off or change an owner's saved choice. The paired
-native consent change requires a fresh choice for this policy.
-
+`ftw-project-stats-2026-10-v2`. Collection stays off until the signed C8 DPIA
+and the legal basis for both flows have been reviewed. Empty
+`FTW_CLOUD_STATISTICS_POLICY` keeps it off; an exact approved policy is needed
+to advertise collection. While paused, `/v1/stats` returns 503 with
+`FTW-Statistics-State: disabled`. Withdrawal, purge and retention still run.
+The unsigned [review draft](privacy/project-statistics-dpia.md) records the
+open decisions. Clear terms alone do not establish a lawful basis.
+
+Statistics are included for every FTW Cloud user under
+`ftw-cloud-2026-10-v2`. Owners accept the current scope through
+`POST /api/cloud/grant`. There is no separate statistics opt-out within Cloud;
+`DELETE /api/cloud` ends Cloud and requests deletion. Old terms do not accept
+the new scope. Deploy compatible Core and clients, show the new terms, and
+review the migration before policy activation. When activated, the service
+refuses legacy rosters and does not restore their cached access after restart.
+Until activation, existing Cloud traffic continues under the old scope.
+
+Local FTW users can instead answer Yes or No on the box. Yes is the suggested
+action, but no new report starts without an answer. `POST /api/cloud/stats`
+with the current statistics policy creates a statistics-only signed uplink:
+no phone keys, pairing codes, Noise key, live cache or remote access.
+`DELETE /api/cloud/stats` stops this local contribution. A current Cloud grant
+refuses that separate stop with 409. Never answer for another household.
+
+Cloud stores the approved hourly fields for 13 months and deletes unpublished
+household rows on withdrawal or purge. Only closed hours starting after the
+new choice contribute; no earlier ledger history is uploaded. Group values
+need 20 homes and no home above 30 percent of a sum. The stats Worker receives
+no home id or per-home value. Counts describe reporting installations, not
+people or all-time installs. Capacity and power are configured limits, not
+proof of physical control. Energy includes only recorded hours, which can be
+partial; missing data is unknown. One instance supplies its local shard only;
+public multi-shard totals need an explicit safe collector.
 
 The separate anonymous daily-report feed is `GET /v1/fleet/public`. It exposes
 only daily counts and dimension labels with at least 10 reports, for at most
 90 days. It contains imported relay history and new reports received by Cloud;
 consumers replace the old archive rather than adding both. It needs no hourly
-energy policy. Averaging seven complete UTC days gives an estimate of active
+energy activation. Updated Core sends it only after the current project
+statistics answer; legacy versions retain their existing daily-report setting. Averaging seven complete UTC days gives an estimate of active
 reporting installations, never people or all installations ever made.

go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/fxamacker/cbor/v2 v2.9.4
 	github.com/gorilla/websocket v1.5.3
 	github.com/quic-go/quic-go v0.63.0
-	github.com/srcfl/ftw/go/protocol v0.0.0-20261017140802-6335dbcea60e
+	github.com/srcfl/ftw/go/protocol v0.0.0-20261017145623-2b1cc2563972
 	golang.org/x/crypto v0.57.0
 	golang.org/x/net v0.58.0
 	golang.org/x/sys v0.48.0

go.sum

@@ -52,8 +52,8 @@ github.com/quic-go/quic-go v0.63.0 h1:LIFGHI4PFUhhw2dDD1ARHdCff143ffMHwZtbnbuJ78
 github.com/quic-go/quic-go v0.63.0/go.mod h1:RAro2j2yN9a9EiPACLHT9IB2NXCvGQmmo/alT0yYI0w=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
-github.com/srcfl/ftw/go/protocol v0.0.0-20261017140802-6335dbcea60e h1:9J9k985aF1/Drp5GHOZFkLmVq2ImuWRPIzUlXiPyoqY=
-github.com/srcfl/ftw/go/protocol v0.0.0-20261017140802-6335dbcea60e/go.mod h1:VEjr/o0yZijJ1nApuJK+nak6FxJ/fubAUBNOunqNrUU=
+github.com/srcfl/ftw/go/protocol v0.0.0-20261017145623-2b1cc2563972 h1:+YZbUeomGm9bDdpVICXMJS8AbgaTaeWrRA+QZjcuNAc=
+github.com/srcfl/ftw/go/protocol v0.0.0-20261017145623-2b1cc2563972/go.mod h1:VEjr/o0yZijJ1nApuJK+nak6FxJ/fubAUBNOunqNrUU=
 github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
 github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
 github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=

internal/registry/statistics.go

@@ -12,7 +12,7 @@ import (
 	"github.com/srcfl/ftw/go/protocol/cloudwire"
 )
 
-// Statistics tables contain only separately consented, closed-hour values.
+// Statistics tables contain only versioned, owner-approved closed-hour values.
 // They have no staff read route, and leave with the home's purge.
 func (r *Registry) StatisticsInit() error {
 	_, err := r.db.Exec(`CREATE TABLE IF NOT EXISTS statistics_grants (
@@ -37,9 +37,10 @@ func (r *Registry) StatisticsGrant(home string, g cloudwire.Grant) error {
 	if err != nil && !errors.Is(err, sql.ErrNoRows) {
 		return err
 	}
-	on := g.Controller == cloudwire.ControllerSourceful && g.StatsPolicy == cloudwire.StatsPolicy &&
-		len(g.StatsID) == 16 && g.StatsSinceMs > 0 && slices.Contains(g.Purposes, protocol.CloudPurposeStats) &&
-		slices.Contains(g.Purposes, protocol.CloudPurposeServe)
+	terms := (g.TermsPolicy == cloudwire.CloudTermsPolicy && slices.Contains(g.Purposes, protocol.CloudPurposeServe)) ||
+		(g.TermsPolicy == cloudwire.StatsPolicy && len(g.Scopes) == 0 && len(g.Purposes) == 1 && g.Purposes[0] == protocol.CloudPurposeStats)
+	on := terms && g.Controller == cloudwire.ControllerSourceful && g.StatsPolicy == cloudwire.StatsPolicy &&
+		len(g.StatsID) == 16 && g.StatsSinceMs > 0 && slices.Contains(g.Purposes, protocol.CloudPurposeStats)
 	if !on || !bytes.Equal(old, g.StatsID) {
 		if _, err := tx.Exec(`DELETE FROM statistics_hours WHERE home=?`, home); err != nil {
 			return err

internal/registry/statistics_test.go

@@ -18,7 +18,7 @@ func statsFixture(t *testing.T) (*Registry, time.Time, cloudwire.Grant) {
 		t.Fatal(err)
 	}
 	now := time.Date(2026, 10, 17, 12, 30, 0, 0, time.UTC)
-	g := cloudwire.Grant{Controller: cloudwire.ControllerSourceful, Purposes: []string{protocol.CloudPurposeServe, protocol.CloudPurposeStats}, StatsID: bytes.Repeat([]byte{1}, 16), StatsPolicy: cloudwire.StatsPolicy, StatsSinceMs: now.Add(-25 * time.Hour).UnixMilli()}
+	g := cloudwire.Grant{TermsPolicy: cloudwire.CloudTermsPolicy, Controller: cloudwire.ControllerSourceful, Purposes: []string{protocol.CloudPurposeServe, protocol.CloudPurposeStats}, StatsID: bytes.Repeat([]byte{1}, 16), StatsPolicy: cloudwire.StatsPolicy, StatsSinceMs: now.Add(-25 * time.Hour).UnixMilli()}
 	return r, now, g
 }
 func statsHome(t *testing.T, r *Registry, now time.Time, g cloudwire.Grant, home string, value int64) cloudwire.Stats {

internal/server/quic_e2e_test.go

@@ -230,10 +230,23 @@ func TestHbAndLiveTravelAsDatagrams(t *testing.T) {
 		t.Fatalf("before the phone spoke, an hb came as %d", hb[0])
 	}
 	phone.say(cloudwire.MsgWatch, cloudwire.Watch{Hz: cloudwire.HzLooking})
-	time.Sleep(50 * time.Millisecond)
-	box.writeAs(wire.Datagram, mustSeal(t, box, cloudwire.MsgLive,
-		cloudwire.Live{Header: box.header(), Fields: map[string]int64{"2": 2000}, SrcRev: 1, Ages: map[string]int64{"m": 50}}))
-	box.writeAs(wire.Datagram, mustSeal(t, box, cloudwire.MsgHB, cloudwire.HB{Header: box.header(), Clock: cloudwire.ClockNTP, SrcRev: 1}))
+	// Observing a datagram proves the phone's watch was processed. A sleep
+	// does not order the independent control stream and datagram path.
+	eventually(t, "an hb datagram after the phone spoke", func() bool {
+		if slices.Contains(phone.kindsOf(cloudwire.MsgHB), wire.Datagram) {
+			return true
+		}
+		box.writeAs(wire.Datagram, mustSeal(t, box, cloudwire.MsgHB, cloudwire.HB{Header: box.header(), Clock: cloudwire.ClockNTP, SrcRev: 1}))
+		return false
+	})
+	eventually(t, "a live datagram before the later source table", func() bool {
+		if len(phone.kindsOf(cloudwire.MsgLive)) > 0 {
+			return true
+		}
+		box.writeAs(wire.Datagram, mustSeal(t, box, cloudwire.MsgLive,
+			cloudwire.Live{Header: box.header(), Fields: map[string]int64{"2": 2000}, SrcRev: 1, Ages: map[string]int64{"m": 50}}))
+		return false
+	})
 	box.send(cloudwire.MsgLive, full2(box))
 	eventually(t, "two lives and an hb at the phone", func() bool {
 		return len(phone.kindsOf(cloudwire.MsgLive)) >= 2 && len(phone.kindsOf(cloudwire.MsgHB)) >= 1
@@ -284,10 +297,24 @@ func TestALateTableStillCounts(t *testing.T) {
 	box.roster(2, grantServe, nil, key.Public())
 	table := map[string]cloudwire.Source{"m": {Kind: "driver", State: protocol.SourceLive, StaleAfterMs: 10_000}}
 	box.send(cloudwire.MsgLive, cloudwire.Live{Header: box.header(), Fields: map[string]int64{"2": 1}, Sources: table, SrcRev: 1, Ages: map[string]int64{}})
+	// The grant and first table must arrive before the overtaking datagram.
+	waitHeldN(t, s, box.home, key, 1)
 	withTable := mustSeal(t, box, cloudwire.MsgLive, cloudwire.Live{Header: box.header(), Fields: map[string]int64{"2": 2},
 		Sources: table, FieldSources: map[string][]string{"2": {"m"}}, SrcRev: 2, Ages: map[string]int64{}})
 	box.writeAs(wire.Datagram, mustSeal(t, box, cloudwire.MsgLive, cloudwire.Live{Header: box.header(), Fields: map[string]int64{"2": 3}, SrcRev: 2, Ages: map[string]int64{}}))
-	time.Sleep(100 * time.Millisecond)
+	eventually(t, "the overtaking live held before its late table", func() bool {
+		for _, raw := range heldSnapshot(t, s, box.home, key).Envelopes {
+			env, _ := cloudwire.DecodeEnvelope(raw)
+			if env.T != cloudwire.MsgLive {
+				continue
+			}
+			var live cloudwire.Live
+			if cloudwire.Unmarshal(env.B, &live) == nil && live.Fields["2"] == 3 {
+				return true
+			}
+		}
+		return false
+	})
 	box.writeAs(websocket.BinaryMessage, withTable)
 	box.send(cloudwire.MsgHB, cloudwire.HB{Header: box.header(), Clock: cloudwire.ClockNTP, SrcRev: 2})
 	waitHeldN(t, s, box.home, key, 3)

internal/store/store.go

@@ -615,7 +615,8 @@ func (s *Store) Seed(id string, r *cloudwire.Roster, at time.Time) {
 }
 
 // Forget drops a home the registry has deleted, as after 90 days without a
-// roster. It is called by upkeep, never by a request.
+// roster. Upkeep uses it after expiry; a signed roster without required Cloud terms
+// also loses every cached session until its box submits the current terms.
 func (s *Store) Forget(id string) {
 	s.mu.Lock()
 	h := s.homes[id]