[abedell-phantom]

Base: perf/fast-delivery (#71, 73c8aba). Branch: fix/perf-stack-regressions, head 3fa63ba. Eight commits.

Why

The perf stack (#67 → #64 → #61 → #69 → #70 → #71) gets its speed from the entity snapshot fast path, from skipping notify and persist when applied data is unchanged, and from writing an entity once per streamed event. A differential hunt against the tree before the snapshot fast path found 17 regressions a client can observe, R1 to R17. Some come from those changes and some from how they interact with #69's synchronous hydration. The fix was first written against main + #61 + #64. This PR is that fix merged onto #71's head. It keeps every one of the stack's performance mechanisms.

The ones most likely to hit an app:

• R6/R7. Streamed create/update events and mutation effects stopped refreshing the persisted cache. On a cold start, items vanished or showed old values.
• R2, and R1 in dev. Hydrating a cached query while its entity was already in memory re-parsed the entity in place. In dev, the snapshot guard's throw escaped Signalium's watcher flush and froze every reactive consumer on the page. In prod, consumers kept a stale nested value.
• R8. A store write that threw was never retried. The query cache was then dropped as corrupt on the next cold start.

Commits

commit	closes
89f54d6 fix(fetchium): optional store hooks for acknowledged, merged and checked entity writes	The store-side half of R9–R12, R14 and R15. Adds onPersisted, isSettled, hasEntity, mergeEntity, and an unsubscribe returned from onDelete, all optional on QueryStore. A store without them keeps the old behaviour: every apply writes the whole record. Compiles and passes the unit suite on its own.
331d9db fix(fetchium): close the snapshot, hydration and event-persistence regressions	R1–R17. This is the fix merged onto #71 with git apply --3way. 24 files applied cleanly, QueryClient.ts included, so #69–#71's lease, stagger and retry code sits next to the fix's applyMutationEvent rewrite. 3 conflicts were resolved by hand (below).
947c88c perf(fetchium): skip the per-snapshot key-cache lookup and an event's unused Set	A cost the fix had added. Moving the static/dynamic key split to a per-client cache made every entity snapshot do a WeakMap lookup. It now compares the cache's identity instead. That gives +10–11% on the one-field re-snapshot cases against the fix as first written, back to stack parity.
768b137 perf(fetchium): take the extra store work back out of streamed events	Costs the fix had added to streamed events, found by a later perf review. (1) An entity built from events stopped being partial only when a full fetch replaced it, so every update to a stream-born row was a mergeEntity that read the stored record back and parsed it twice: about half the apply rate. It is now written whole once its fields are known to be the whole record: the store held no record when it was first written. (This commit also treated events that carried every field of the entity's class as the whole record; the last commit takes that back, see below.) (2) With the AsyncQueryStore writer, any queued write turned write skipping off for every entity (!isSettled()), so identical events during a burst all wrote again. The store now reports only queued work that can delete a record (new optional hasQueuedDeletes(), which excludes the client's own entity writes); an entity whose own write is queued counts as written; an entity a queued write stops referencing is rewritten if referenced again before that write lands; and a stale record can no longer be marked persisted by an older write's acknowledgement. (3) An unrouted create is routed once, not twice, and the duplicate-slot check's lookup is reused. (4) Dev builds skip the snapshot key-list check for frozen snapshots.
2c42511 perf(fetchium): stop re-running a shared entity's consumers, copying unchanged fetch data and asking the store twice	Three more costs found by a second perf review. (1) The R17 notify re-ran every consumer of a shared entity, and broke its list's snapshot identity, when a query whose result is that entity first applied, even with identical data. It is gone: R17 is left as it was before #60 (below). (2) The parser copied every nested t.object/t.record of every fetch (refetch of 1000 entities with nested values: −15…−21%). A fetch result or cached record is now copied only where a parsed value differs, arrays included (an array of primitives is no longer rebuilt each fetch; an empty one is always new, because a live array grows the array it holds); event payloads and mutation effects still always are. The parser still never writes into its input: a reified value is written back only when it changed, and a merge into a nested value an adapter froze goes into a copy. (3) A streamed create asked the store hasEntity even when a live array retained the root, and a partial one asked again in save(). Routed creates now make no kv.has call (partial: one instead of two); an unrouted create keeps its one.
chore	chore(fetchium): major changeset listing the behavior changes an upgrade brings (.changeset/upgrade-behavior-changes.md). No source change.
f83dfb5 fix(fetchium): keep merging an event-built entity whose events carried every field of its class	A regression 768b137 added, found by the third review. With two classes on one typename (a list class and a detail class with more fields) and only the list class registered in a session, a streamed full update for an entity not in memory was parsed with the list class alone, counted as covering the record and written with saveEntity, which deleted the detail's fields: at the next launch the detail's cached query could not hydrate and waited for the network. The same happened to an entity an event created under a root in memory. An event-built entity now stays partial and merges into its record unless the store held no record of it at its first write. Covering a class's fields says nothing about fields another class, possibly one not registered this session, wrote to the record.
chore	chore(fetchium): shorten this change's changesets for app developers: .changeset/skip-write-and-snapshot-fixes.md and the major upgrade notes rewritten as short, user-facing bullets. The full detail stays in this description. No source change.
chore	chore(fetchium): rewrite this change's changesets for React developers: the same two files, rewritten again for an app developer. The upgrade notes now list breaking changes only, and add one line from #74: code that awaits a query whose last reader unmounts before its first data now waits for the next mount, gcTime collection or destroy() instead of getting an AbortError. No source change.

Conflicts resolved by hand in 331d9db:

• QueryResult.ts. perf(fetchium): hydrate synchronously from a sync store #69's hydrate() catch already clears updatedAt and discards a corrupt entry, so the fix's own catch hunk was dropped. The fix throws CachedEntityMismatchError from inside hydrate()'s try, so a cached query whose in-memory entity doesn't match its shape is dropped and fetched. The R17 notify after query extras are attached applied as is; 2c42511 removes it.
• The two API docs pages. They take the fix's frozen-snapshot and destroy() wording and keep perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71's prefetch/retain table, its "useQuery never suspends" bullet and its useSuspenseQuery section.

What changes, by area

• Snapshot fast path and dev guard (EntityInstance.ts).
  • Dev snapshots are frozen. A consumer that mutates one, such as a sort() in render, now fails at its own line (R4).
  • The drift guard serves the re-read value and reports the drift out of band: through log.error, then as an uncaught error. It no longer throws inside a snapshot (R1).
  • Values are compared with Object.is plus a structural fallback (R3).
  • Static and dynamic fields are classified per client, so classes that share a typename stay fresh (R5).
  • Narrowed arrays consume the members they leave out (R13).
  • Proxies from a hot-reloaded module are walked instead of returned as they are (R16).
• Hydration and the parser (parseEntities.ts, QueryResult.ts).
  • An in-memory entity is never re-parsed. Its data must satisfy the query's shape structurally. If it doesn't, the whole cached query is dropped and fetched (R2).
  • The parser no longer writes into the objects it is given. Event payloads and mutation effects are parsed into copies. A fetch result or cached record is copied only where a parsed value differs from it, and a later merge into a nested value an adapter froze goes into a copy.
  • R17 is not fixed on purpose: query extras attached to a shared entity reach the query's own consumers, and other consumers on their next recompute, as before perf: don't notify or persist when applied data is unchanged #60. Notifying them cost every consumer of the entity a render.
• Streamed events and persistence (applyEntities.ts, QueryClient.ts, EntityStore.ts, LiveCollection.ts).
  • Changed existing entities are written during the apply.
  • Created entities are written once a written record references them, and the root is written before routing. Anything that nothing references is evicted.
  • Event-built entities merge only the fields their events carried, through mergeEntity.
  • A record the store holds (hasEntity) is refreshed even when its entity isn't in memory (R6, R7).
  • _persisted is set only after the write succeeds (R8).
  • No write is skipped while an async writer has operations queued (R9).
• Stores.
  • AsyncQueryStore writers acknowledge their writes and report failed ones (R10).
  • They resize a persisted LRU queue into a new buffer instead of throwing or orphaning entries (R11).
  • They keep the queue size an older reader persisted (R12).
  • destroy() unsubscribes the store's delete listener (R14).
  • SyncQueryStore shares delete listeners between instances over one kv (R15).

The changesets are .changeset/skip-write-and-snapshot-fixes.md and .changeset/upgrade-behavior-changes.md (major: the upgrade notes for the whole stack).

Verification

Trees: base = main bb255ca on Signalium 3.0.3. stack = #71 on Signalium #254. fix = this branch on Signalium #254.

• Round 2 (2c42511): unit 1454 passed / 2 skipped (+1: a frozen fetch result applied without a write and merged later; the R17 test now asserts the query's own consumer sees the extras and the other consumer is not re-run, and fails with the notify back). Review probes, dev and prod, against 768b137: only guard-misc › "late-attached query extras…" and hy-shared-entity › H9/R17 change (pass → fail, the R17 revert above). zz-vp2-shared-root: 0 added row runs, 0 added list snapshots, identity kept (was +1/+1/lost). zz-vp2-store-ops: routed create 0 kv.has (was 1), partial routed create 1 (was 2), unrouted 1.
• Round 3 (f83dfb5): unit on Signalium 3.0.3 1456 passed / 2 skipped, browser 114/114; on Signalium #254 unit the same, browser 112/114 (the two #252 counts). New tests in event-persistence.test.ts (2): an update for an entity not in memory, and an entity an event builds under a root in memory, each with only the list class registered in the session: the record keeps the detail's fields and the detail's cache is served at the next launch. Both fail on 2c42511. Review probe zz-vc31 T4/T5 pass in dev and prod (fail on 2c42511); the kit and every earlier probe give the same results as on 2c42511.
• Round 3 bench (5 interleaved runs vs 2c42511): a full-payload update for an entity not in memory whose record the store holds, single-class typename, −16…−26% (vp3 257k → 216k, W1 250k → 184k): each such event merges again, reading and parsing the record, instead of replacing it. Two-class typenames and in-memory events are unchanged (within ±3%). The next PR's merge cache takes the repeated read out, and the PR after it writes such an event whole again when no other class of the typename declares more fields.
• Round 2 bench (zz-vp2 apply/held-event benches, 7 interleaved runs vs perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71, with the next PR on top): refetch of 1000 entities with nested values, identical payload −6.7% (was −13.6%), list membership −2.8% (was −9.6%); shared-entity consumers no longer re-run; routed create −8% (overlapping); unrouted create −22% and an event for an entity not in memory −16% (R7's hasEntity and record write, inherent).
• The fix's acceptance tests: skip-write-and-snapshot-fixes.test.ts and event-persistence.test.ts pass 55/55. 50 of them fail on perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71.
• Maintainers' suites:
  • On Signalium 3.0.3: unit 1448 passed, 2 skipped; browser 114/114.
  • On Signalium #254: unit is the same. Browser is 112/114, and the 2 failures are the known #252 render counts in component.test.tsx, the same as on perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71.
• Differential suite (51 files, before = base, after = stack, fixed = this branch): the output is byte-identical to the reference tables in the fix kit.
  • Node: 294 tests. 113 regressed, 54 resolved, 59 intended differences remain, 3 newly differ.
  • Browser: 14 regressed, 14 intended, 2 newly differ.
  • The 5 that newly differ are the NaN rule and the frozen-snapshot dev contract.
• Review probes, stack → fix, dev and prod:
  • No probe that passed on stack fails here, apart from the 5 above.
  • In each pair below, the first number is the dev result and the second is the prod result.

finding	probes on this branch
R6/R7	store-h1-subtree 9/9, store-h2-baseline 8/8, live-event-routing 10/13 (the remaining 3 are in the reference's intended list)
R1/R2	guard-hydrate 6/6. In hy-shared-entity, H8 and H9 pass. In rx-shared, S8 and S8b now show no errors and keep updating; their assertions still encode base's "the cache overwrites memory" and its loading frame.
R8	store-h11-flaky 1/2 (matches the reference)
R13–R17	cd-narrowing 2/2, store-h4-multiclient 3/3, api-module-instances 2/2, guard-misc 9/9
R3/R4/R5	guard-nan 7/7, snap-nan 6/6 (dev too), guard-shared-typename 1/1. snap-consumer and guard-consumer-mutation match the frozen-snapshot contract.
R9–R12	async-race 10/10. async-roles 7/9, async-maxcount-upgrade 2/3 and api-async-maxcount 1/3 match the reference.

• tsc, eslint and prettier: clean on every changed file.

Bench

npm run bench (snapshot.bench.ts), hz, median of 6 interleaved runs on Apple Silicon:

case	#71	this PR
100 entities, one field changes	72.4k	73.2k
100 entities, identical event	1.23M	1.27M
1000 entities, one field changes	8.2k	8.6k
1000 entities, identical event	1.21M	1.26M
cold query 100 / 1000	730 / 132	739 / 124

All within noise. The cold rows are ±10–15% rme.

With the default ~100 ms warmup, the first case in the file has read 7–12% below #71 in some sessions. With a 1 s warmup it does not reproduce (+2.7%), and CPU profiles of the two trees have the same shape. It looks like JIT tier-up timing, not steady-state work.

The fix adds this work to the hot path:

• Object.is instead of === in the snapshot compare.
• Per event: one copy of the root payload, plus a shallow copy of each nested object and record the parser builds. Per fetch: a shallow copy only of a nested object, record or array whose parsed value differs from the response's (one holding a formatted value or an entity), plus one Object.isFrozen per in-place nested merge.
• Per apply: an _applying try/finally, a parseId check (reusing the entity lookup) and, only when nothing changed, a storeIsSettled() call.
• Per created event entity: a for…in over its shape keys. Per created root no live array retains: one hasEntity call, whose answer the write reuses.
• Freezing and static-field verification run in dev only.

Intended changes a client will notice

• Dev snapshots are frozen. Code that mutates a snapshot throws a TypeError at its own call site in dev. Prod snapshots are not frozen, but should be treated as read-only.
• Snapshot drift in dev is reported out of band (log.error, then an uncaught error) instead of from inside the snapshot.
• Re-applying NaN no longer notifies, and -0 over 0 is applied.
• A cached query whose in-memory entity no longer matches its shape is dropped and fetched instead of being re-parsed.
• With a store that implements mergeEntity, an entity built from events writes only the fields its events carried, merged over its record, until a fetch replaces it or the store turns out to hold no record of it. An event that carries every field of the entity's class is still merged, so fields another class of the typename wrote survive. A store without the new hooks writes every apply, as before 0.6.

Residuals, deliberately left

• Probes that encode base's behaviour. Some R2 probes still assert base's "cached data overwrites memory" and its loading frame: parts of hy-shared-entity, hy-react-hydration and rx-shared. This PR replaces that behaviour with trusting memory, on purpose.
• Production keeps a consumer's mutation of a snapshot until the entity's version moves.
• The async writer and silently dropped writes. A store that silently drops a write can't be detected without an acknowledgement. While an async writer has a deletion-capable operation queued (a query save, activation or delete, or a reader's message), write skipping is off for every entity. A burst of the client's own entity writes no longer turns it off.
• An unrouted create costs one hasEntity call (kv.has on MMKV, one JSI call). That is what lets an event for an entity on disk but not in memory refresh its record (R7). The second perf review measured an unrouted create −25…−28% against perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71, about two-thirds of it that call; for a high-rate stream of creates nothing routes (a price feed), that is the price of R7. A routed create no longer asks the store.
• A full update for an entity not in memory whose record the store holds costs about 40% against perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71 when two classes share the typename (W1: 341k → 141k here, 197k at the top of the stack), against the −16% measured earlier with one class. The record write is R7 (perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71 did not write the record at all); its cost scales with the record's size because the merge reads, splits and rewrites it. On this PR alone every such event also reads the record back; the next PR caches that.
• R17 is left as before perf: don't notify or persist when applied data is unchanged #60. An existing consumer of a shared entity sees another query's extras (__refetch, __hasNext, …) on its next recompute. guard-misc › "late-attached query extras…" and hy-shared-entity › H9/R17 fail again, in dev and prod, and the differential tables differ from the kit's reference by those rows.
• An adapter that hands out the same response object more than once (a TopicQueryAdapter topic's fulfilled data, an adapter caching its last response) can now share an unchanged nested object between the entities that apply it, and a later in-place merge changes that object for all of them, as on perf(fetchium): microtask refetches, prefetch/retain, sync lease starts, shouldRetry hook #71 and every earlier version. Event payloads are not affected.
• t.optional set to null by an event for an entity that isn't in memory keeps its stored value.
• An event-built entity that a cached query then fills in from its record becomes whole, so its next write replaces the record and drops fields another class of its typename wrote, as every list-class write does on this PR. The cached-first-paint PR at the top of the stack hands the record over at that fill, so those writes keep the fields.
• Pre-existing, unchanged:
  • A shared-typename entity first reached through a narrow shape keeps the narrow parse.
  • Entity unions don't hydrate from the cache.
  • The LRU queue is persisted only when it is created or resized.
  • Signalium's flushWatchers should still catch errors per signal.
• Not in this PR. The October review's new findings are N1–N8, S1 and S2: activation timing, nested-value reactivity, config keys and useSuspenseQuery holds. They are fixed in the next PR in the stack (fix/stack-timing-and-reactivity).

🤖 Generated with Claude Code

[jimmy-phantom]

I don't think the error message should carry such a long description with it, although this looks like a pre-existing problem.

[jimmy-phantom]

This sounds like a boolean check and is confusing to me at callsites
E.g.

if (persist) this.writeDropsRef(child);

Maybe something likeinvalidateChildPersistence would be clearer.

[jimmy-phantom]

This function now does multiple things and it's hard to figure out exactly what it's doing just by looking at it. The cleanest would be to use named helper functions, but even just having clear section headers like // Save children before their parent (rather than the long and rambling thing the agent wrote), // Defer incomplete entities, etc. would help.