[rickykaare]

Problem

Cosmos DB emulator (and occasionally the cloud service) can respond to a container/database read with 404 NotFound, substatus 1013 ('Collection is not yet available for read') for a short window right after it is created. This happens when two or more processes race to create the same database or container concurrently — common in local Aspire setups where multiple services/processors start simultaneously against the same emulator instance.

This crashed the host on startup (\DocumentStoreService.StartAsync\ -> \SubscriptionInitializer.InitializeAsync), e.g. observed in load-management's Processors project.

Fix

• Added \CosmosRetryHelper\ with exponential backoff (1s -> 10s, up to 6 attempts) that retries only on the specific 404/1013 signature.
• Applied it in \DocumentStoreInitializer\ (database creation), \DocumentInitializer\ (document container creation) and \SubscriptionInitializer\ (change-feed lease container creation).
• Added unit tests covering: first-try success, retry-then-succeed, giving up after max attempts, and not retrying unrelated Cosmos exceptions.
• Updated CHANGELOG.md.

All 221 existing + new tests pass.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Copilot review overview

🟡 Changes recommended

Hardcoded wall-clock delays add six seconds to tests and prevent efficient deterministic testing.

Review effort: Balanced
Findings: 1

Open (1)

• Inject controllable time to avoid slow retry-path tests · New

What changed in this PR

Adds targeted retries for transient Cosmos DB 404/1013 initialization races.

Changes:

• Adds exponential-backoff retry handling.
• Applies retries to database, document-container, and lease-container creation.
• Adds unit tests and release notes.

File	Description
CosmosRetryHelperTests.cs	Tests retry behavior and limits.
SubscriptionInitializer.cs	Retries lease-container creation.
DocumentStoreInitializer.cs	Retries database creation.
DocumentInitializer.cs	Retries document-container creation.
CosmosRetryHelper.cs	Implements targeted exponential backoff.
CHANGELOG.md	Documents the initialization fix.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🔵 Needs a closer look

The default six-attempt schedule and 10-second delay cap need explicit test coverage.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Inject controllable time to avoid slow retry-path tests

Previously missed (1)

In code that hasn't changed since last review

Add retry test covering six attempts and capped backoff

src/​Chronicles/​Documents/​Internal/​CosmosRetryHelper.cs:61

The retry tests stop after three attempts, so neither this 10-second cap nor the default six-attempt policy is exercised. A regression here would still leave the suite green; add a default-policy failure test asserting six calls and delays of 1, 2, 4, 8, and 10 seconds.