Repository navigation
feat(core): snapshot an aggregate's state, valid only for a fingerprint of the code that folds it - #211
Open
benatespina wants to merge 2 commits into
Open
feat(core): snapshot an aggregate's state, valid only for a fingerprint of the code that folds it#211benatespina wants to merge 2 commits into
benatespina wants to merge 2 commits into
Conversation
…nt of the code that folds it
Opt-in per aggregate with `runtime.overrides.<aggregate>.snapshot: { every }`.
A `snapshot:<aggregate>` subscriber writes the snapshots in the background and
a command loads the snapshot and folds the events after it. A snapshot is a
cache of the fold, never the source of truth: it is valid only for the
`stateFingerprint` the generator writes to the registry, a digest of the
modules that shape the state and of the inferred type, plus the configured
`snapshot.version`; one that is missing, from other code, unreadable or past
the stream's head is ignored, and a share of the loads (`snapshot.verify`) is
checked against a fold of the whole stream, dropping a snapshot that folds to
another state. An app from `createTestApp` checks every load and throws
`SnapshotError`. The state is kept with its types through devalue.
Every adapter gains a `snapshots` table and `Storage.snapshotStore`, with a
storage contract run in each one, workerd included. `load({ fromVersion })` in
the SQL event stores now reads the stream's head before the events, bounded by
it, so an append landing between the two reads can no longer report a head
past the events returned.
With 10 050 events on SQLite a command goes from 63 ms to 0.46 ms.
…he state fingerprint On Windows the candidate built with a forward slash named the same file as the one resolved from another importer with a backslash, so the module was read twice and the fingerprint walk counted it as two.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Snapshots of an aggregate's state: a cache of the fold, never the source of truth, opt-in per aggregate, with a validity the runtime checks itself. Closes the "no snapshots yet" item of the limitations page.
runtime.overrides.<aggregate>.snapshot: { every, verify?, version? }. Per-aggregate runtime settings already live inoverrides, and the generator never imports modules, so the setting lives next to timeouts and retries instead of in a module.snapshot:<aggregate>subscriber (newSubscriberKind"snapshot"), checkpointed like policies and processes, idempotent across instances. The first deploy walks the history and covers existing long streams.AggregateLoaderthe command pipeline uses: snapshot plus the events after it (load({ fromVersion })). A snapshot that is missing, written under another fingerprint, unreadable or past the stream's head is ignored, the last two dropped.stateFingerprintin.bounda/registry.ts, a sha256 prefix overstate.ts, every event module, its upcasts, everything those reach through relative imports (not packages or path aliases) and the inferred state type.snapshot.versionis the manual hatch for what the fingerprint cannot see. The registry is now written after inference, liketypes.ts.snapshot.verify(default0.01) is the share of loads from a snapshot folded again from the first event and compared structurally; a mismatch is logged, counted and the snapshot dropped.createTestAppverifies every load and throwsSnapshotError(from the command, or fromrunUntilIdle()for the subscriber's path), so a test suite proves the folds deterministic and the state serializable.devalue(new dependency ofcore, 6.0.3):Date,Map,Set,bigint,undefinedand the rest survive; a class instance cannot be snapshotted and is logged (tests throw).Storage.snapshotStoreand asnapshotstable in memory, SQLite (libSQL and the Durable Object throughcore/adapter/sqlite) and PostgreSQL, with a storage contract run in every adapter, workerd included.bounda.snapshotscounter withbounda.aggregate.typeandbounda.outcome(used,verified,mismatch,missing,stale,unreadable,ahead).Also fixed
load({ fromVersion })in the SQLite and PostgreSQL event stores read the events and then the head in two statements; outside a transaction an append in between returned a head past the events, which a fold from a snapshot would have turned into a lost update. The head is read first now and the events bounded by it; a test pins it in both stores.Measured
Stream of 10 050 events on SQLite (libSQL, file), one command appending one event, mean over 50 commands:
snapshot: { every: 500 }×138. The fold is O(stream) per command without snapshots and O(
every) with them.Docs
concepts/long-streams(the "Why there are no snapshots yet" section becomes "Snapshots"),reference/configuration,reference/errors,reference/observability,reference/limitations,reference/cli,guides/testing,guides/deployment,guides/changing-events,adapters/index, the core README.Checks
pnpm checkgreen locally; the PostgreSQL suite skipped itself locally (Docker was not running) and runs in CI.