feat(core): snapshot an aggregate's state, valid only for a fingerprint of the code that folds it

.changeset/aggregate-snapshots.md

@@ -0,0 +1,7 @@
+---
+"@bounda-dev/core": minor
+"@bounda-dev/cli": minor
+"@bounda-dev/postgresql": minor
+---
+
+Snapshots of an aggregate's state, opt-in per aggregate with `runtime.overrides.<aggregate>.snapshot: { every }`. A `snapshot:<aggregate>` subscriber writes them in the background, and a command loads the snapshot and folds only the events after it. A snapshot is a cache of the fold and never the source of truth: it is valid only for the `stateFingerprint` the generator now writes to the registry, a digest of the modules that shape the state and of the type inferred for it, plus the configured `snapshot.version`; one that is missing, from other code, unreadable or past the stream's head is ignored, and a share of the loads (`snapshot.verify`, `0.01` by default) is checked against a fold of the whole stream, which drops a snapshot that folds to another state. An app from `createTestApp` checks every load and throws `SnapshotError`. The state is kept with its types (`Date`, `Map`, `Set`, `bigint`). Every adapter gains a `snapshots` table and `Storage.snapshotStore`; `load` with `fromVersion` in the SQL event stores now reads the stream's head before the events, bounded by it, so an append landing between the two reads can no longer report a head past the events returned.

docs/src/content/docs/adapters/index.mdx

@@ -88,6 +88,7 @@ On first use the adapter creates its tables, prefixed with `bounda_` by default:
 | `bounda_inbox` | Which handler already completed for which event, so a retry does not run it again |
 | `bounda_scheduled_commands` | Scheduled commands, process deadlines and timeouts, and delayed policy runs |
 | `bounda_dead_letters` | Handler runs that gave up, with the error and the attempt count |
+| `bounda_snapshots` | The last snapshot of every stream of an aggregate that takes them, with the fingerprint it is valid for |
 
 Read models get one table each, named after the read model in snake_case behind `rm_`, so no
 read model can take one of the names above: `orderSummary` becomes `bounda_rm_order_summary`.

docs/src/content/docs/concepts/long-streams.md

@@ -1,6 +1,6 @@
 ---
 title: Long streams close their books
-description: What to do in Bounda with a stream that keeps growing, an account or the stock of a SKU, why closing the books comes before snapshots, and why snapshots will only come with a validity the runtime checks itself.
+description: What to do in Bounda with a stream that keeps growing, an account or the stock of a SKU, why closing the books comes before snapshots, and how snapshots work when a stream has nothing to close, with a validity the runtime checks itself.
 sidebar:
   order: 51
 ---
@@ -74,7 +74,7 @@ command folds that month alone, while the closed ones stay in the event store fo
 that shows the account across years. The same shape fits a SKU's stock with a stock-take per
 period, or a cashier's shift.
 
-## Why there are no snapshots yet
+## Snapshots
 
 A snapshot stores the folded state so that the next load starts from it instead of from the first
 event. It is a cache, and a cache can be stale. The question is what a stale one breaks.
@@ -94,15 +94,74 @@ reference says to adjust on every deployment that changes the aggregate's state.
 inferred by the generator and carries no version a person maintains, and a version someone has to
 remember to bump is the bug waiting for the one deploy where they forget.
 
-So the stance is: **snapshots only with a validity the runtime checks itself**. They are planned as
-a cache of the fold and never the source of truth: one that is missing, unreadable or doubtful is
-ignored, and the stream is folded from the start, with the same result. A snapshot will be valid
-only for a fingerprint of the modules that shape the state, so that a change to any of them makes
-the old ones unusable without anyone deciding so. `bounda rebuild` already works that way: a paused
-rebuild resumes only under a digest of the same view and projections, and starts again otherwise.
-There is no date and no API for snapshots yet.
+So Bounda's snapshots are **a cache of the fold, never the source of truth, with a validity the
+runtime checks itself**. One that is missing, written by other code, unreadable or doubtful is
+ignored, and the stream is folded from the start, with the same result. They are opt-in per
+aggregate, for the stream that has nothing to close.
 
-## What snapshots would not do
+### Opting in
+
+```ts
+// bounda.config.ts
+runtime: {
+  overrides: {
+    account: { snapshot: { every: 200 } },
+  },
+},
+```
+
+`every` is how many events a stream gathers past its last snapshot before the next one is written.
+A subscriber named `snapshot:account` follows the global stream like a projection does and writes
+the snapshots in the background, in the `worker` role or an app from `createTestApp` after
+`runUntilIdle()`, so no command ever waits for one: a command loads the snapshot, folds the events
+after it and appends as before. The first deploy with snapshots walks the whole history and writes
+one for every stream long enough, so the streams that hurt today are covered without a migration.
+`app.getLag()` reports the subscriber like any other.
+
+### What a snapshot is valid for
+
+The generator computes a fingerprint of every aggregate's state and writes it to the registry as
+`stateFingerprint`: a digest of `state.ts`, of every event module with its upcasts, of every module
+those reach through relative imports, and of the state type it inferred. A snapshot is written
+under that fingerprint and used only under the same one, so a change to any of those files, a
+reformat or a comment included, retires every snapshot of the aggregate at the next deploy: each
+stream is folded from its first event once, and the subscriber writes a fresh snapshot at its next
+event. Nobody decides when; the cost is one slow load per stream, and a wrong state is never
+decided from.
+
+What the fingerprint cannot see is a module reached through a package or a path alias such as
+`~/lib/money`, since the generator follows relative paths only. When one of those changes what the
+state folds to, raise `snapshot.version`, which is part of what the snapshot is valid for too:
+
+```ts
+account: { snapshot: { every: 200, version: 2 } },
+```
+
+### What the runtime checks
+
+A snapshot is used only when its fingerprint is the aggregate's, it reads back, and the stream
+reaches its version; one that reads badly or is past the stream's head is dropped. On top of that,
+`snapshot.verify`, `0.01` by default, is the share of loads from a snapshot that are checked
+against a fold of the whole stream: a snapshot that folds to another state is logged, dropped and
+not used, and the command goes on with the state of the stream. Such a difference means the fold is
+not a function of the events alone, an `evolve` reading the clock or a module variable, which is
+worth knowing however the state is loaded. The `bounda.snapshots` counter says what every load
+found: `used`, `verified`, `mismatch`, `missing`, `stale`, `unreadable` or `ahead`
+([observability](/reference/observability/)).
+
+An app from `createTestApp` checks every load and throws `SnapshotError` instead of dropping the
+snapshot, from the command that loaded it or from `runUntilIdle()` when the subscriber met it. A
+test suite that runs its aggregates with `snapshot` configured therefore proves their folds
+deterministic ([testing](/guides/testing/#snapshots-are-checked)).
+
+### What a snapshot holds
+
+The state is kept with its types: `Date`, `Map`, `Set`, `bigint`, `undefined`, `RegExp` and
+nested plain objects and arrays come back as they were. A class instance or a function in the
+state cannot be kept: the subscriber logs it and writes no snapshot for that stream, and a test app
+throws `SnapshotError` from `runUntilIdle()`. Keep the state to data.
+
+## What snapshots do not do
 
 - **Speed up a rebuild.** A rebuild runs projections over the global stream; it never folds an
   aggregate, so an aggregate's snapshot saves it nothing. What would make a large rebuild faster
@@ -111,6 +170,9 @@ There is no date and no API for snapshots yet.
   Object, with everything that hangs off it) has one writer at a time, and a snapshot saves a
   read, not a commit. More throughput comes from
   [one store per tenant](/concepts/how-it-runs/#the-way-out-one-store-per-tenant).
+- **Replace closing the books.** A stream with a period has a natural end and a state the next
+  period starts from; a snapshot keeps a stream without one loadable, while the stream itself
+  keeps growing and every projection of it still reads it whole.
 
 ## How others do it
 
@@ -134,8 +196,9 @@ There is no date and no API for snapshots yet.
 
 ## What stays outside
 
-- **A stream with no period and many events.** If the business has nothing to close, the fold is
-  paid on every command until snapshots exist.
+- **A stream with no period and many events.** If the business has nothing to close, a snapshot
+  keeps each command's fold to the events since the last one; a projection or a read of the stream
+  still goes through all of it.
 - **The moment between periods.** A command writes one stream, so the next ledger opens when the
   policy runs, after the closing commits, not in the same transaction. An entry posted in between
   is rejected and retried by its caller.
@@ -145,9 +208,8 @@ There is no date and no API for snapshots yet.
 
 ## Where to read more
 
-- [What is not there yet](/reference/limitations/), the short version of this
-  page, and [Changing an event's shape](/guides/changing-events/) for the upcasters every fold
-  runs.
+- [`snapshot`](/reference/configuration/#per-aggregate) in the configuration reference, and
+  [Changing an event's shape](/guides/changing-events/) for the upcasters every fold runs.
 - [State](/guides/project-layout/#state) and [policies](/guides/project-layout/#policies-policies),
   for the modules used above.
 - Oskar Dudycz,

docs/src/content/docs/guides/changing-events.md

@@ -94,9 +94,10 @@ what keeps the chain straight.
   new event type and keep handling the old one; upcasters transform, they do not invent.
 
 Upcasters are cheap but not free: they run on every read of an old event. An aggregate with
-thousands of old events re-runs the chain on every command, since Bounda has no snapshots yet
-([what is not there yet](/reference/limitations/)), and a rebuild runs it over
-the whole global stream once.
+thousands of old events re-runs the chain on every command, unless it takes
+[snapshots](/concepts/long-streams/#snapshots), which fold the old events once, and a rebuild runs
+it over the whole global stream once. Adding an upcaster retires the aggregate's snapshots, as any
+change to how its state is folded does.
 
 ## What is not covered yet
 

docs/src/content/docs/guides/deployment.mdx

@@ -290,6 +290,11 @@ events a batch carries. Rebuilds honour it too.
 `backoff` sets how long a subscriber whose batches fail is left alone before the next try: see
 [A projection that keeps failing](#a-projection-that-keeps-failing).
 
+An aggregate whose streams grow without a period to close takes
+[snapshots](/concepts/long-streams/#snapshots) with `runtime.overrides.<aggregate>.snapshot`; a
+subscriber named `snapshot:<aggregate>` writes them in the background, and shows in the lag like
+any other.
+
 ## Observability
 
 The runtime reports OpenTelemetry spans and metrics with no adapter to write: register an SDK

docs/src/content/docs/guides/testing.mdx

@@ -276,6 +276,16 @@ stopped reacting:
 expect((await app.getLag()).maxLag).toBe(0);
 ```
 
+## Snapshots are checked
+
+An aggregate configured with [`snapshot`](/concepts/long-streams/#snapshots) gets them in a test
+app too: `runUntilIdle()` writes them, and every later command loads from one. A test app checks
+every such load against a fold of the whole stream, where a production app checks a sample, and
+throws `SnapshotError` instead of dropping the snapshot: from the command whose state differs, or
+from `runUntilIdle()` when the subscriber met the difference or a state it cannot keep. A suite
+that passes with `snapshot` configured has proved the aggregate's fold a function of its events
+alone, and its state made of data.
+
 ## What is worth testing
 
 - **The rules**, through commands: what is accepted, what is rejected and with which code, and

docs/src/content/docs/reference/cli.md

@@ -17,8 +17,10 @@ aggregates without `state.ts`.
 ## `bounda generate`
 
 Reads the project layout and writes `.bounda/registry.ts`, `.bounda/register.d.ts`,
-`.bounda/types.ts` and one `+types/<name>.ts` next to every module. Files whose content did not
-change are left alone; `+types` files whose module is gone are removed, and so are the folders
+`.bounda/types.ts` and one `+types/<name>.ts` next to every module. The registry carries, for every
+aggregate, a `stateFingerprint` of the modules that shape its state and of the type inferred for
+it, which its [snapshots](/concepts/long-streams/#what-a-snapshot-is-valid-for) are valid for.
+Files whose content did not change are left alone; `+types` files whose module is gone are removed, and so are the folders
 their removal empties; a folder holding nothing but `+types` (and Finder's `.DS_Store`) is not
 taken for a module.
 

docs/src/content/docs/reference/configuration.md

@@ -100,8 +100,20 @@ a stopped one left: see [delivery guarantees](/concepts/delivery-guarantees/#the
 ### Per aggregate
 
 - **`overrides`**, default `{}`. Settings for one aggregate, by aggregate name: `commands.timeout`,
-  and `policies` and `processes` with the keys above. What an override leaves out comes from the
-  app's settings.
+  `policies` and `processes` with the keys above, and `snapshot`. What an override leaves out comes
+  from the app's settings.
+- **`overrides.<aggregate>.snapshot`**, none by default. Opts the aggregate into
+  [snapshots](/concepts/long-streams/#snapshots) of its state, written in the background and loaded
+  from instead of the stream's first event:
+  - **`every`**, required. How many events a stream gathers past its last snapshot before the next
+    one is written.
+  - **`verify`**, default `0.01`. The share of the loads from a snapshot, 0 to 1, checked against a
+    fold of the whole stream. A snapshot that folds to another state is logged, dropped and not
+    used. An app from `createTestApp` checks every load and throws `SnapshotError` instead.
+  - **`version`**, default `1`. Part of what a snapshot is valid for, next to the
+    `stateFingerprint` the generator computes from the modules that shape the state. Raise it to
+    retire every snapshot of the aggregate when the fold changed through something the fingerprint
+    cannot see, such as a package or a module behind a path alias.
 
 A process can set its own lifetime in its `config`,
 `({ events }) => ({ startedBy: [...], timeout: "48h" })`, which wins over `processes.timeout` and

docs/src/content/docs/reference/errors.md

@@ -72,6 +72,11 @@ dead letter with `errorType: "retriable_exhausted"`.
   event, or its process failed on another step first. The letter stays `failed`.
 - **`REBUILD_SUPERSEDED`** (`RebuildSupersededError`; an operator's answer). Another rebuild of the
   same read model took over; this one stopped without writing. `readModel`.
+- **`SNAPSHOT_FAILED`** (`SnapshotError`; only in an app from `createTestApp`). A
+  [snapshot](/concepts/long-streams/#snapshots) did not stand for its stream: the state folded from
+  it differs from the one folded from the first event, or the state cannot be serialized. Thrown by
+  the command that loaded it or by `runUntilIdle()`. Any other app logs it, drops the snapshot and
+  folds the stream from the start.
 
 The last three are answers to an operator, from `app.deadLetters`, `bounda dead-letters` and
 `bounda rebuild`, not failures of a handler.

docs/src/content/docs/reference/limitations.md

@@ -8,12 +8,6 @@ sidebar:
 Bounda is 0.x, and this is the honest list of what a production app might want and does not
 get today. Each item says why, so nobody discovers it the hard way:
 
-- **Snapshots.** An aggregate is folded from its whole stream on every command. That is fine for
-  the hundreds of events per instance that Bounda's kind of app produces, and it is not fine for
-  hundreds of thousands. Snapshots are deliberately not built yet: the state is inferred and
-  carries no version, so a snapshot written by yesterday's `evolve` would silently be wrong after
-  today's deploy. They come with a versioning story or not at all; until then, close the books
-  of an aggregate that would grow forever ([Long streams](/concepts/long-streams/)).
 - **Changing the shape of a process's state.** A process keeps its state in its own lifecycle
   events, so a change to that shape has the same problem an event payload has, and no
   `state.upcast.ts` yet. See
@@ -31,5 +25,6 @@ get today. Each item says why, so nobody discovers it the hard way:
 What a production app does get, and where it is explained:
 [rebuilding a read model](/guides/deployment/#rebuilding-a-read-model) without taking it offline,
 [dead letters with a way out](/guides/dead-letters/), [upcasters](/guides/changing-events/) for
-events whose payload changed, [observability](/reference/observability/) through OpenTelemetry, and
-a dispatcher that reacts in milliseconds on PostgreSQL.
+events whose payload changed, [snapshots](/concepts/long-streams/#snapshots) whose validity the
+runtime checks itself, [observability](/reference/observability/) through OpenTelemetry, and a
+dispatcher that reacts in milliseconds on PostgreSQL.

docs/src/content/docs/reference/observability.md

@@ -25,7 +25,7 @@ Everything is reported under the scope `@bounda-dev/core`. Spans:
 | Span | When | Attributes |
 | --- | --- | --- |
 | `bounda.command <Type>` | a command is dispatched | `bounda.command.type`, `bounda.aggregate.type`, `bounda.aggregate.id`, `bounda.correlation_id`, `bounda.causation_id`, `bounda.outcome` (`stored`, `scheduled`, `rejected`), `bounda.event.count`; a rejection adds the event `bounda.command.rejected`, its code in `bounda.rejected` |
-| `bounda.subscriber <name>` | the dispatcher hands a batch to a projection, the policy runner or the process runner; idle passes produce none | `bounda.subscriber`, `bounda.subscriber.kind`, `bounda.position.after`, `bounda.event.count`, `bounda.outcome` (`advanced`, `held`, `failed`, `moved`) |
+| `bounda.subscriber <name>` | the dispatcher hands a batch to a projection, the policy runner, the process runner or a snapshot subscriber; idle passes produce none | `bounda.subscriber`, `bounda.subscriber.kind` (`projection`, `policy`, `process`, `snapshot`), `bounda.position.after`, `bounda.event.count`, `bounda.outcome` (`advanced`, `held`, `failed`, `moved`) |
 | `bounda.projection <readModel>.<projection>` | a projection handles one event | `bounda.read_model`, `bounda.projection`, the event's id, type and aggregate, `bounda.correlation_id` |
 | `bounda.policy <aggregate>.<policy>` | a policy handler runs | `bounda.policy`, the event's id, type and aggregate, `bounda.correlation_id`, `bounda.attempt` |
 | `bounda.process <aggregate>.<process>` | a process handler runs; `… at <field>` for an `at-<field>.ts`, `… at timeout` for `at-timeout.ts` | `bounda.process`, the event's id, type and aggregate, `bounda.correlation_id`, `bounda.attempt`; for a deadline, `bounda.process`, the process's aggregate type and id and `bounda.correlation_id` |
@@ -46,8 +46,11 @@ Metrics:
 | `bounda.dispatcher.lag` | observable gauge, events each subscriber is behind the head | `bounda.subscriber` |
 | `bounda.commands` | counter | `bounda.command.type`, `bounda.outcome` (`stored`, `scheduled`, `rejected`, `failed`) |
 | `bounda.dead_letters` | counter | `bounda.handler.kind`, `bounda.handler`, `bounda.outcome` (`terminal`, `retriable_exhausted`) |
+| `bounda.snapshots` | counter, loads of an aggregate with [snapshots](/concepts/long-streams/#snapshots) | `bounda.aggregate.type`, `bounda.outcome` (`used`, `verified`, `mismatch`, `missing`, `stale`, `unreadable`, `ahead`) |
 
 The lag gauge is what to alert on: a subscriber whose lag grows is a projection or a policy that is
 failing or stuck, and `app.getLag()` returns the same numbers for a health endpoint, with what a
 failing subscriber is stuck on (see
-[A projection that keeps failing](/guides/deployment/#a-projection-that-keeps-failing)).
+[A projection that keeps failing](/guides/deployment/#a-projection-that-keeps-failing)). A
+`mismatch` on `bounda.snapshots` is worth one too: a snapshot folded to a state the stream does not
+give, so an `evolve` of that aggregate depends on something other than the events.