Skip to content

fix(remixer): speed up publish for large books with small edits - #1011

Open
yghaemi wants to merge 1 commit into
stagingfrom
fix/remixer-batch-skip-saving
Open

yghaemi wants to merge 1 commit into
stagingfrom
fix/remixer-batch-skip-saving

Conversation

@yghaemi

@yghaemi yghaemi commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Publish time scaled with book size rather than with the number of changed pages: every unchanged page cost a job save, a 100ms sleep, and a MindTouch order call in the final sibling pass.

  • Count unchanged pages once in a new skippedPages job field instead of logging and saving each one; the progress bar adds that count.
  • Batch job message writes (at most ~1s / 25 lines), still saving createdPages immediately and forcing a flush at phase boundaries.
  • Drop the fixed per-page 100ms sleep; retry ORDER_PAGES on transient failures so a rate-limited call no longer silently misplaces a page.
  • Limit the final ordering pass to the book's first layer plus sibling groups this run created, imported or moved a page into.
  • Replace per-page whole-book scans (hasSubpages, sibling lookups, parentID rewrites, re-snapshot findIndex) with existing indexes.

Publish time scaled with book size rather than with the number of changed
pages: every unchanged page cost a job save, a 100ms sleep, and a
MindTouch order call in the final sibling pass.

- Count unchanged pages once in a new `skippedPages` job field instead of
  logging and saving each one; the progress bar adds that count.
- Batch job message writes (at most ~1s / 25 lines), still saving
  createdPages immediately and forcing a flush at phase boundaries.
- Drop the fixed per-page 100ms sleep; retry ORDER_PAGES on transient
  failures so a rate-limited call no longer silently misplaces a page.
- Limit the final ordering pass to the book's first layer plus sibling
  groups this run created, imported or moved a page into.
- Replace per-page whole-book scans (hasSubpages, sibling lookups,
  parentID rewrites, re-snapshot findIndex) with existing indexes.
@yghaemi
yghaemi requested a review from jakeaturner October 23, 2026 19:05
@github-actions

Copy link
Copy Markdown

No actionable bugs found in the PR changes.

The commits have no merge base in this checkout, so I reviewed the direct diff between the two supplied revisions.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant