Repository navigation
Preserve archives and delete only successfully exported node states - #590
robertexternbrink wants to merge 6 commits into
Conversation
Audit finding: BUG-0017
Audit finding: BUG-0018
Audit finding: BUG-0019
Audit findings: BUG-0017, BUG-0018, BUG-0019, BUG-0020
Replace cutoff-wide cleanup after a node-state export with exact archived IDs, timestamps and node identities. Recheck every node latest timestamp at deletion time and bound SQL parameter usage for large completed selections. Cover concurrent late inserts, reused identities, newly latest ties, unexported orphan rows, partial export failure, deletion failure and cancellation. Port only GB5 safety from the deployed 2026-10-26 source; retain the existing selection and archive format. Depends on BE14 archive-integrity.
Order the shared latest-state subquery by timestamp and then row ID. Two updates in the same second must expose the later state in detail, list and map reads, while an actually newer timestamp keeps priority over a higher ID backfill. Port the deployed GB6 correction without query batching or indexing features.
|
This review took a bit longer, as it found two larger discussion points which should be decided before merging, even though the PR itself builds cleanly and all tests pass (also with Blocking Job RetentionDescription: With the
Proposal: Skip jobs with Deletion Timing IncreaseDescription: AI-assisted timing tests (freshly migrated DB, bundled SQLite 3.53.4, WAL, 1,000 nodes × 288 rows, 287k rows to delete, single runs with about ±1 s noise) show a notable increase for the new
The existing Proposal: This keeps the introduced fix while reducing the measured time:
|
Publish complete archives without replacing existing output, preserve portable metadata, and stop retention cleanup after transfer failures. Delete only the exact node-state identities successfully exported so concurrent backfills and replacements remain intact, and prefer the latest inserted state when timestamps tie.
Validation: Archive, repository and retention package tests, including race checks, and build/vet passed locally. Checks used the published cc-lib v2.17.0 and line-protocol v2.4.0 dependencies without local replacements.