Skip to content

fix: re-read the index from zero when the store left its boot line - #166

Merged
sanchitrk merged 1 commit into
canaryfrom
claude/index-sync-boot
Nov 14, 2026
Merged

sanchitrk merged 1 commit into
canaryfrom
claude/index-sync-boot

Conversation

@sanchitrk

@sanchitrk sanchitrk commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The app server's index mirror only treated a restored backup as a full re-read while the server's rev was still behind the app's cursor. Once a restored store wrote past the cursor, index.sync resumed across two timelines: rows the restored store wrote between the backup and the cursor never arrived, a thread came before its session, and the machine's sync retried a FOREIGN KEY constraint failed batch every second forever.

Fix

Same boot contract usage.sync already uses (resync after a version that is no longer valid, as in the PubSub paper, section 4.2.1):

  • codework: index.sync takes an optional boot. When after > 0 and Store.onLine(boot, after) is false, it streams from zero, and the first Synced carries reset: true. Every Synced carries the head's boot. index.sync and usage.sync now share Rows.resume. The desktop's copy of the contract (apps/widget/src/codework.ts) has boot/reset optional, for older servers.
  • apps/server: a new migration adds machine.boot (the index's boot, as rollup_boot is the rollup's). The app server sends it on resume. reset runs the full-sync path: Mirror.unseen runs in the first batch's transaction, so nothing from the lost timeline survives, and a failed re-read leaves the old rows. An older server without boots keeps the rev < cursor rule.
  • thread.sync: unseen does reset thread cursors, but the E2E test showed that this alone is not enough. On a first open, the restored line's entries failed on entry (thread_id, seq) against the lost line's entries at the same seqs ("Thread sync stopped"). So a thread's first open (cursor NULL) now replaces its stored entries and inbox in its first batch.

Projects and spaces from a lost timeline still stay (no tombstones yet), as before.

Tests

E2E (apps/server/test/sync.test.ts, new case "a restored store is read again in full once its head passed the cursor…"), with the app up and a thread open:

  • A plain restart, with work done while the app could not see the server, only resumes: no full notice.
  • A restore that works past the cursor before the app sees it is re-read in full. The lost sessions are removed; a session deleted only in the lost timeline comes back; the restored store's sessions are all there. The opened thread ends with exactly the restored line's entries, and no constraint failed or Thread sync stopped lines appear.
  • With the client not sending boot (today's behaviour), the same test times out in the FK loop.

Updated: sync.restored.json (its second restore, a copy from another line, is now a full re-read that removes nothing; returning rows keep their desktop columns), schema.json (new column), host.test.ts (rolls back the new migration too), codework sync.index.json (boot on Synced).

Live, on OrbStack cw-a and cw-b, each suite run alone:

  • metrics.simulation.live.test.ts: 9/9. The restore case now cuts cw-a's network while the restored store works past the cursor, so the app first sees it there. It waits on the index cursor too and checks the lost session is removed, the restored ones are present, and no constraint failures occur. The "index does not recover yet" comment and the index: false option are gone. metrics.simulation.json is regenerated. Its seeded totals depend on the clock: seed.ts uses Date.now(), an extra random draw depends on the time of day, and the weekend weighting depends on the date. So it drifts between days with or without this change.
  • usage.machines.live.test.ts: 6/6.

Also run: vp check, vp run typecheck, the apps/server suite (42/42) and the codework suite (151 passed, 22 skipped), all without *.live. Harness is untouched.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

index.sync now takes the client's last boot and, when the store left that
line (a restored backup), streams from zero with reset on the first Synced;
every Synced carries the head's boot. usage.sync shares the same resume.

The app server keeps the index's boot on machine.boot, sends it on resume and
treats reset as a full sync, so Mirror.unseen sweeps the lost timeline in the
first batch's transaction. Before, a restore whose head passed the cursor
resumed across two timelines and stuck in a foreign-key retry loop.

A thread's first open now replaces its stored entries and inbox in its first
batch: after a reset, entries of the lost line held the seqs the restored line
writes, and the open failed on entry (thread_id, seq).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sanchitrk
sanchitrk merged commit 0c0579e into canary Nov 14, 2026
1 check passed
@sanchitrk
sanchitrk deleted the claude/index-sync-boot branch November 14, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant