Repository navigation
fix: re-read the index from zero when the store left its boot line - #166
Merged
Merged
Conversation
index.sync now takes the client's last boot and, when the store left that line (a restored backup), streams from zero with reset on the first Synced; every Synced carries the head's boot. usage.sync shares the same resume. The app server keeps the index's boot on machine.boot, sends it on resume and treats reset as a full sync, so Mirror.unseen sweeps the lost timeline in the first batch's transaction. Before, a restore whose head passed the cursor resumed across two timelines and stuck in a foreign-key retry loop. A thread's first open now replaces its stored entries and inbox in its first batch: after a reset, entries of the lost line held the seqs the restored line writes, and the open failed on entry (thread_id, seq). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The app server's index mirror only treated a restored backup as a full re-read while the server's rev was still behind the app's cursor. Once a restored store wrote past the cursor,
index.syncresumed across two timelines: rows the restored store wrote between the backup and the cursor never arrived, a thread came before its session, and the machine's sync retried aFOREIGN KEY constraint failedbatch every second forever.Fix
Same boot contract
usage.syncalready uses (resync after a version that is no longer valid, as in the PubSub paper, section 4.2.1):index.synctakes an optionalboot. Whenafter > 0andStore.onLine(boot, after)is false, it streams from zero, and the firstSyncedcarriesreset: true. EverySyncedcarries the head'sboot.index.syncandusage.syncnow shareRows.resume. The desktop's copy of the contract (apps/widget/src/codework.ts) hasboot/resetoptional, for older servers.machine.boot(the index's boot, asrollup_bootis the rollup's). The app server sends it on resume.resetruns the full-sync path:Mirror.unseenruns in the first batch's transaction, so nothing from the lost timeline survives, and a failed re-read leaves the old rows. An older server without boots keeps therev < cursorrule.unseendoes reset thread cursors, but the E2E test showed that this alone is not enough. On a first open, the restored line's entries failed onentry (thread_id, seq)against the lost line's entries at the same seqs ("Thread sync stopped"). So a thread's first open (cursor NULL) now replaces its stored entries and inbox in its first batch.Projects and spaces from a lost timeline still stay (no tombstones yet), as before.
Tests
E2E (
apps/server/test/sync.test.ts, new case "a restored store is read again in full once its head passed the cursor…"), with the app up and a thread open:fullnotice.constraint failedorThread sync stoppedlines appear.boot(today's behaviour), the same test times out in the FK loop.Updated:
sync.restored.json(its second restore, a copy from another line, is now a full re-read that removes nothing; returning rows keep their desktop columns),schema.json(new column),host.test.ts(rolls back the new migration too), codeworksync.index.json(bootonSynced).Live, on OrbStack
cw-aandcw-b, each suite run alone:metrics.simulation.live.test.ts: 9/9. The restore case now cuts cw-a's network while the restored store works past the cursor, so the app first sees it there. It waits on the index cursor too and checks the lost session is removed, the restored ones are present, and no constraint failures occur. The "index does not recover yet" comment and theindex: falseoption are gone.metrics.simulation.jsonis regenerated. Its seeded totals depend on the clock:seed.tsusesDate.now(), an extra random draw depends on the time of day, and the weekend weighting depends on the date. So it drifts between days with or without this change.usage.machines.live.test.ts: 6/6.Also run:
vp check,vp run typecheck, the apps/server suite (42/42) and the codework suite (151 passed, 22 skipped), all without*.live. Harness is untouched.🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.