Skip to content

feat(core): snapshot an aggregate's state, valid only for a fingerprint of the code that folds it - #211

Open
benatespina wants to merge 2 commits into
mainfrom
feat/aggregate-snapshots
Open

benatespina wants to merge 2 commits into
mainfrom
feat/aggregate-snapshots

Conversation

@benatespina

Copy link
Copy Markdown
Member

What

Snapshots of an aggregate's state: a cache of the fold, never the source of truth, opt-in per aggregate, with a validity the runtime checks itself. Closes the "no snapshots yet" item of the limitations page.

  • Opt-in with runtime.overrides.<aggregate>.snapshot: { every, verify?, version? }. Per-aggregate runtime settings already live in overrides, and the generator never imports modules, so the setting lives next to timeouts and retries instead of in a module.
  • Written outside the command path by a snapshot:<aggregate> subscriber (new SubscriberKind "snapshot"), checkpointed like policies and processes, idempotent across instances. The first deploy walks the history and covers existing long streams.
  • Loaded by a new AggregateLoader the command pipeline uses: snapshot plus the events after it (load({ fromVersion })). A snapshot that is missing, written under another fingerprint, unreadable or past the stream's head is ignored, the last two dropped.
  • Validity by fingerprint, computed by the generator: stateFingerprint in .bounda/registry.ts, a sha256 prefix over state.ts, every event module, its upcasts, everything those reach through relative imports (not packages or path aliases) and the inferred state type. snapshot.version is the manual hatch for what the fingerprint cannot see. The registry is now written after inference, like types.ts.
  • Verification: snapshot.verify (default 0.01) is the share of loads from a snapshot folded again from the first event and compared structurally; a mismatch is logged, counted and the snapshot dropped. createTestApp verifies every load and throws SnapshotError (from the command, or from runUntilIdle() for the subscriber's path), so a test suite proves the folds deterministic and the state serializable.
  • Serialization with devalue (new dependency of core, 6.0.3): Date, Map, Set, bigint, undefined and the rest survive; a class instance cannot be snapshotted and is logged (tests throw).
  • Stores: Storage.snapshotStore and a snapshots table in memory, SQLite (libSQL and the Durable Object through core/adapter/sqlite) and PostgreSQL, with a storage contract run in every adapter, workerd included.
  • Telemetry: bounda.snapshots counter with bounda.aggregate.type and bounda.outcome (used, verified, mismatch, missing, stale, unreadable, ahead).

Also fixed

load({ fromVersion }) in the SQLite and PostgreSQL event stores read the events and then the head in two statements; outside a transaction an append in between returned a head past the events, which a fold from a snapshot would have turned into a lost update. The head is read first now and the events bounded by it; a test pins it in both stores.

Measured

Stream of 10 050 events on SQLite (libSQL, file), one command appending one event, mean over 50 commands:

ms per command
without snapshots 63.03
with snapshot: { every: 500 } 0.46

×138. The fold is O(stream) per command without snapshots and O(every) with them.

Docs

concepts/long-streams (the "Why there are no snapshots yet" section becomes "Snapshots"), reference/configuration, reference/errors, reference/observability, reference/limitations, reference/cli, guides/testing, guides/deployment, guides/changing-events, adapters/index, the core README.

Checks

  • pnpm check green locally; the PostgreSQL suite skipped itself locally (Docker was not running) and runs in CI.
  • Stryker: see the mutation jobs.

…nt of the code that folds it

Opt-in per aggregate with `runtime.overrides.<aggregate>.snapshot: { every }`.
A `snapshot:<aggregate>` subscriber writes the snapshots in the background and
a command loads the snapshot and folds the events after it. A snapshot is a
cache of the fold, never the source of truth: it is valid only for the
`stateFingerprint` the generator writes to the registry, a digest of the
modules that shape the state and of the inferred type, plus the configured
`snapshot.version`; one that is missing, from other code, unreadable or past
the stream's head is ignored, and a share of the loads (`snapshot.verify`) is
checked against a fold of the whole stream, dropping a snapshot that folds to
another state. An app from `createTestApp` checks every load and throws
`SnapshotError`. The state is kept with its types through devalue.

Every adapter gains a `snapshots` table and `Storage.snapshotStore`, with a
storage contract run in each one, workerd included. `load({ fromVersion })` in
the SQL event stores now reads the stream's head before the events, bounded by
it, so an append landing between the two reads can no longer report a head
past the events returned.

With 10 050 events on SQLite a command goes from 63 ms to 0.46 ms.
…he state fingerprint

On Windows the candidate built with a forward slash named the same file as
the one resolved from another importer with a backslash, so the module was
read twice and the fingerprint walk counted it as two.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant