feat(data): deferred durable blob writes + combined local/remote blob refs - #187
Merged
Conversation
… refs
blobStore now defers the storage-tier write off the critical path via a new
createDeferredFallbackAsyncCache (memory -> pendingWrites -> storage read
order); getRef returns after the memory write and flush() is the durability
barrier that drains pending writes and surfaces failures. serialize-to-storage
flushes before persisting refs.
BlobRefSchema is relaxed to allow a combined { localBlobRef, remoteBlobRef }
shape; getBlob/hasBlob/borrowUrl are local-first with self-healing remote
fallback, opt-in remote-hash verification, a withRemoteUrl constructor, and
hash-canonicalized identity.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The canonicalized-key borrow test now verifies revokeObjectURL fires exactly once, only after the last borrow across both ref shapes is returned. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two related changes to
@adobe/data'sblobStore, sharing one durability model.Part 1 — Deferred durable writes +
flush()createDeferredFallbackAsyncCache(memory, storage):putawaits the fast tier and schedules the durable write in the background;matchreads memory →pendingWrites→ storage;flush()drains pending writes and rejects with anAggregateErroron failure (no silent swallowing); repeated writes of the same key coalesce;deletewaits out any in-flight write so it can't be resurrected.blobStoreuses this wrapper, so every consumer gets deferred durability by default.getRefreturns after the memory write;BlobStore.flush()is the documented durability barrier.serialize-to-storagenow awaitsblobStore.flush()before persisting refs.Part 2 — Combined BlobRef (local + remote)
BlobRefSchemarelaxed to a 3-memberoneOf(remote-only / local-only / combined), eachadditionalProperties: false.getBlob/hasBlob/borrowUrlare now local-first (fixes the old dispatch that sent combined refs down the remote path); remote fallback self-heals by repopulating the local cache under the known hash.verifyRemoteHash, awithRemoteUrl(ref, url)constructor, and hash-canonicalized identity so a combined ref and a local-only ref with the same hash dedup.Test evidence
withRemoteUrlvalidation, identity canonicalization).pnpm run lint,pnpm run typecheck, andpnpm testall green (3225 tests pass).Related PRs
None.
Jira
N/A — internal library refactor; no ticket.