Repository navigation
feat(cas): verify complete objects before atomic output promotion - #769
Conversation
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 59 minutes. View limit detailsLimit details: You’ve used the included review currently available. Review configuration: ⚙️ Run configuration
📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 834196cfe0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adversarial Independent Review: PR #769 (
|
| Operation / Code Path | Entry Point | Staging / Execution | Verification / Sealing | Promotion / Emission |
|---|---|---|---|---|
| Staged Expected Ingestion | StagedContent::read_expected |
PrivateContentBuffer::write via io::copy bounded by byte_limit |
VerifiedContent::seal (checks target.length and BLAKE3 target.hash) |
Produces private StagedContent; dropping publishes nothing |
| Memory Tier Publication | MemoryTier::publish_content |
Transfers bytes from content.0.bytes() |
MemoryTier::put_verified verifies hash before inserting into blobs |
Emits ContentReceipt (establishes_durability: false) |
| Disk Tier Publication | DiskTier::publish_content |
Writes .tmp file in blob partition directory |
DiskTier::put_verified verifies hash before write; atomic rename |
Emits ContentReceipt (establishes_durability: false) |
| Memory View Reconstruction | MemoryContentView::reconstruct |
Calls reconstruct_quarantined with in-memory lookup |
PrivateContentBuffer stages bytes; VerifiedContent::seal checks identity |
MemoryContentDestination::promote swaps visible buffer atomically |
| Disk View Reconstruction | DiskContentView::reconstruct |
Streams from std::fs::File via io::copy into staging |
Re-hashes file bytes; rejects corrupted/truncated files via VerifiedContent::seal |
TransactionalContentDestination::promote only on seal success |
Merges and Integration Audit
- Branch Divergence Structure:
- Merge Base:
18b22e362e986f3e2509856433d040f33dc81bd2 - PR Head:
834196cfe0244819d452f5a03bf0f0ea59b98c06(1 commit ahead of base) - Target Main:
7dde48b223ed105c13a1b70afb6a1a07d1dc308e(6 commits ahead of base: PR fix: bound Action WAL parent replay and retained states #767 fix/study-recovery-replay) - No merge commits exist within
feat/physical-content-port.
- Merge Base:
- Target Divergence Semantic Check:
- Main added WAL recovery replay cursor reuse in
warp-core(trusted_runtime_host.rs,provenance_store.rs,WAL.md). feat/physical-content-portmodifies onlyecho-casand documentation.- The only shared file between branch and target main is
CHANGELOG.md. - In
CHANGELOG.md,7dde48b2added an entry under### Fixed, while834196cfadded an entry under### Added. git merge-tree 18b22e36 834196cf 7dde48b2completes cleanly with exit code 0 and zero conflict markers.- No recovery call sites in
warp-coreor runtime call physical-content APIs; zero integration friction or broken invariants.
- Main added WAL recovery replay cursor reuse in
Constants & Numeric Evidence Verification
| Claim / Constant | Location | Raw Evidence / Coordinate | Audit Result |
|---|---|---|---|
| All 34 echo-cas tests pass | PR #769 body, line 5 | retained evidence: k02-green.log:6,29,39,48,60 (17 unit + 4 disk + 3 physical + 10 retention) |
Verified Exact (34 passed, 0 failed) |
Initial Clippy failure on unused Write |
PR #769 body, line 5 | retained evidence: k02-initial.log:4,24-28 (unused import: Write in physical_content.rs:15:21, exit code 101) |
Verified Exact |
| Three new tests added | PR #769 body, line 5 | retained evidence: k02-green.log:39-44 (prefix_and_ignored_writer_failure..., memory_complete_object..., disk_complete_object...) |
Verified Exact |
| Rust toolchain 1.90.0 / 1.96.0 | PR #769 body, line 5 | retained evidence: k02-green.sh:1-3 (cargo +1.90.0 test, cargo +1.90.0 clippy, cargo +1.96.0 fmt) |
Verified Exact |
Chunk-boundary test constant 262_145 |
crates/echo-cas/tests/common/physical_content.rs:20 |
262_145 bytes = 256 * 1024 + 1 (256 KiB chunk boundary + 1 byte) |
Verified Exact |
| Build cache usage: 13,348,293,696 B | retained evidence: k02-green.result.json:3 |
13.35 GiB <= 20 GiB host build budget | Verified Within Bound |
| Test data usage: 4,255,566,912 B | retained evidence: k02-green.result.json:4 |
3.96 GiB <= 4 GiB aggregate data budget | Verified Within Bound |
| Log output usage: 16,029,981 B | retained evidence: k02-green.result.json:5 |
15.29 MiB <= 128 MiB log budget | Verified Within Bound |
| Free disk floors: 728 GB host / 692 GB VM | retained evidence: k02-green.result.json:6-7 |
Both well above 50 GiB safety floor | Verified Within Bound |
| Code SHA256 hashes matching HEAD | retained evidence: k02-green.manifest.json:5-7 |
physical_content.rs: cb5140d6...common/physical_content.rs: 4158a0e1...tests/physical_content.rs: ff12bab4... |
Verified 100% Identical to HEAD |
| Post-gate documentation additions | Git commit 834196cf vs k02-green.manifest.json |
CHANGELOG.md, crates/echo-cas/README.md, and echo-keep-physical-content-boundary.md updated after gate |
Verified Documented & Grounded |
Invariant & Contract Analysis
- Quarantine & Output Invisibility:
PrivateContentBufferis private tophysical_content.rs. The reconstruction closure only receives a&mut dyn Write. Visible output is held inMemoryContentDestination::visibleand is never touched untilVerifiedContent::sealhas cryptographically verified the BLAKE3 digest and exact byte count. In case of any error (source error, hash mismatch, length mismatch, allocation failure, or promotion error), previous visible output is completely preserved. - Ignored Writer / Backend Overrun Protection:
InPrivateContentBuffer::write, if length exceedslimitor allocation fails,self.resource_failureis latched totrue. Inreconstruct_quarantined, even if a misbehaving reconstruction callback catches and swallows the write error,staging.resource_failureis checked independently before sealing. - Allocation Refusal & Resource Discipline:
Allocation is checked usingself.bytes.try_reserve_exact(bytes.len())before extending the buffer. Host allocation refusal latchesresource_failureand maps toContentError::ResourceLimitwithout panicking. - Missing Content vs Absence Refusal:
Missing content returnsContentError::CapabilityUnavailable. It does not return an authenticated absence refusal, adhering to the boundary architecture thatDiskTier/MemoryTiercannot prove non-membership. - No Synchronization or Durability Claims:
ContentReceipt::establishes_durabilityandContentReceipt::establishes_complete_viewreturnfalse.DiskTier::publish_contentrelies on POSIX directoryrenamewithoutfsync, matching its explicit lack of crash-durability claims. - No Causal Authority:
VerifiedContentandContentReceiptbind onlyContentTarget(BLAKE3 hash and byte length). They grant no causal authority, worldline state, or commit validity. - SPDX Headers:
All 8 modified files conform to repository SPDX policy (Apache-2.0 for Rust code, Apache-2.0 OR LicenseRef-MIND-UCAL-1.0 for Markdown). Validated viascripts/check_spdx.sh --check.
Mandatory Verification Checklist
- Every code path traced (file:line to file:line):
- Staging:
physical_content.rs:171->physical_content.rs:282->physical_content.rs:57->physical_content.rs:165. - Memory publication:
physical_content.rs:321->memory.rs:109->physical_content.rs:136. - Disk publication:
physical_content.rs:355->disk.rs:72->physical_content.rs:136. - Memory reconstruction:
physical_content.rs:301->physical_content.rs:229->physical_content.rs:125. - Disk reconstruction:
physical_content.rs:332->physical_content.rs:229->physical_content.rs:125.
- Staging:
- Every merge audited:
- Base
18b22e36to target main7dde48b2(PR fix: bound Action WAL parent replay and retained states #767) audited. - Three-way merge tree simulated via
git merge-tree; 0 conflicts. No overlapping callers or recovery interactions.
- Base
- Constants and claims verified against raw evidence:
- 34 tests, 3 new tests, 1 unused import clippy failure in initial gate verified against
retained evidence: k02-initial.logandk02-green.log. - Resource consumption verified against
k02-green.result.json.
- 34 tests, 3 new tests, 1 unused import clippy failure in initial gate verified against
- Documentation accuracy verified:
CHANGELOG.md,crates/echo-cas/README.md, anddocs/architecture/echo-keep-physical-content-boundary.mdaccurately reflect the implementation posture.- One-line paragraph structure and SPDX headers confirmed.
- Execution vs static inspection boundaries declared:
- Executed: static line-by-line inspection, SHA256 checksum comparison,
git merge-tree,git diff --check,scripts/check_spdx.sh --check. - Inspected only: Docker worker test logs (
k02-initial.log,k02-green.log) and telemetry results (k02-green.result.json). - Skipped / Unavailable: Host test execution and Docker container execution (per user directive and resource discipline rules). No physical power-loss injection was performed.
- Executed: static line-by-line inspection, SHA256 checksum comparison,
Verdict
APPROVE
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e0494292a7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adversarial Independent Review: PR #769 (
|
| Operation / Path | Entry Point | Staging & Bounds | Verification & Sealing | Promotion & Output |
|---|---|---|---|---|
| Staged Ingestion | StagedContent::read_expected |
PrivateContentBuffer::write via io::copy, checked against byte_limit and expected_length |
VerifiedContent::seal (checks target.length and BLAKE3 target.hash) |
Produces private StagedContent; dropping publishes nothing |
| Memory Publication | MemoryTier::publish_content |
N/A (consumes verified StagedContent) |
Checked by MemoryTier::put_verified |
Stores in memory map; returns ContentReceipt (durability=false, complete_view=false) |
| Disk Publication | DiskTier::publish_content |
N/A (consumes verified StagedContent) |
Checked by DiskTier::put_verified |
Writes temp file, atomic fs::rename; returns ContentReceipt (durability=false, complete_view=false) |
| Memory View Reconstruct | MemoryContentView::reconstruct |
reconstruct_quarantined with private PrivateContentBuffer |
Reads from MemoryTier::get, sealed via VerifiedContent::seal |
Calls TransactionalContentDestination::promote; emits ContentReceipt |
| Disk View Reconstruct | DiskContentView::reconstruct |
reconstruct_quarantined with private PrivateContentBuffer |
Streams file via DiskTier::blob_path, sealed via VerifiedContent::seal |
Calls TransactionalContentDestination::promote; emits ContentReceipt |
| Memory Destination Promotion | MemoryContentDestination::promote |
Bound check against limit |
Sealed handle ownership transfer | Replaces visible buffer atomically; leaves previous bytes unchanged on any error |
- Parallel Path Parity:
- Upfront Preflight:
read_expected(physical_content.rs:178) andreconstruct_quarantined(physical_content.rs:245) enforce identical length-versus-limit rejection (ResourceLimit) prior to I/O or closure invocation. - Missing Content: Both
MemoryContentView(physical_content.rs:325) andDiskContentView(physical_content.rs:356) returnCapabilityUnavailablerather than asserting authenticated absence. - Shared Conformance: Both tiers execute identical assertions in
crates/echo-cas/tests/common/physical_content.rs:16-104.
- Upfront Preflight:
2. Merges and Integration Audit
- Branch Commits: 3 linear commits (
834196cf,e0494292,7802d898) on base18b22e36. No internal merge commits exist on the branch. - Target Main Divergence (
18b22e36..7dde48b2): PR fix: bound Action WAL parent replay and retained states #767 introduced WAL recovery replay bounds incrates/warp-core/src/trusted_runtime_host.rsanddocs/topics/WAL.md. - Target Integration Diff: PR feat(cas): verify complete objects before atomic output promotion #769 touches exclusively
crates/echo-cas,docs/architecture/echo-keep-physical-content-boundary.md, andCHANGELOG.md. Zero overlap withwarp-corerecovery paths.git merge-tree 18b22e36 7802d898 7dde48b2completes cleanly with 0 conflicts. No existing callers are rerouted.
3. Claims Line-by-Line Audit
- Rebuttal of Windows Rename Defect: Confirmed by pinned Rust 1.90 standard library source. Duplicate publication idempotency verified by test.
- Overlong Rejection Invariant: Confirmed that
identity_failureis latched beforeresource_failureinPrivateContentBuffer::writeand takes precedence infailure(). - Test Isolation Invariant: Confirmed that
fresh_disk_fixtureloops withfs::create_dirand does not delete or reuse preexisting paths.
4. Constants Against Raw Evidence
| Constant / Parameter | Declared Location | Bound / Value | Raw Evidence Coordinate | Verified Status |
|---|---|---|---|---|
| Max Fixture Search Slots | tests/physical_content.rs:53 |
1024 slots | k02-fresh-fixture-gate.log |
Bounded iteration; terminates with AlreadyExists if exhausted |
| Test Conformance Payload | tests/common/physical_content.rs:20 |
262,145 bytes (256 KiB + 1) | k02-fresh-fixture-gate.log |
Verifies chunk boundary crossing |
| Worker Build Budget | Launch Contract | 20 GiB (21,474,836,480 B) | Measured: 13,355,391,484 B | Pass (62.2% of budget) |
| Worker Data Budget | Launch Contract | 4 GiB (4,294,967,296 B) | Measured: 4,262,595,068 B | Pass (99.2% of budget) |
| Worker Log Budget | Launch Contract | 128 MiB (134,217,728 B) | Measured: 18,819,936 B | Pass (14.0% of budget) |
| Host Disk Floor | Workstation Guard | >= 50 GiB free | Measured: 727,068,368,896 B | Pass (~677 GiB free) |
| VM Disk Floor | Workstation Guard | >= 50 GiB free | Measured: 691,347,476,480 B | Pass (~643 GiB free) |
5. Document Figures and Numeric Claims
- Test Counts:
k02-green.log: 34 tests passed (17 unit + 4 disk + 3 physical_content + 10 semantic_retention).k02-fresh-fixture-gate.log: 36 tests passed (17 unit + 4 disk + 5 physical_content + 10 semantic_retention). Two new witnesses added:impossible_staging_budget_does_not_read_sourceandoverlong_reconstruction_is_mismatch_at_any_sufficient_budget.
- Working Tree File Hashes:
Every file in the HEAD working tree exactly matchesk02-fresh-fixture-gate.manifest.json:CHANGELOG.md:fe418497cc72ddae13cc05c85067d8473c466b8a7ec11823ae899e54d56f8bcbcrates/echo-cas/README.md:701598accf72c8eef0a3973e9839c08038c7064e30b19c2fe45aaad3d80c9b3bcrates/echo-cas/src/disk.rs:641f915138d47eab2e56ead3ee472bd3993c4854e81f2bba8c162e0768dc9f17crates/echo-cas/src/lib.rs:e11a7a5c38c35676a8e44702a61e97272a3ea780296bb5f3622201d2636cd601crates/echo-cas/src/physical_content.rs:1136f9c1b8f4914e9db9c770b7c8ffe332484e07ec997c3d80edcd0b264d095ccrates/echo-cas/tests/common/physical_content.rs:0db74fe6aa86eaf8a8e30e6a8293bb1d65d7d19c4142a2ac1da9082c5bfd4479crates/echo-cas/tests/physical_content.rs:28f373eb9737b11a0b2c137cf3cd501cf8b9aed2b539103ece5b46bc418b679bdocs/architecture/echo-keep-physical-content-boundary.md:f9cddefbc202943992f55bac16ef731dc6c954732a55ee91d2019bc6f7f6d427
6. State Machine and Error Transitions
- Preflight Refusal:
target.length > byte_limitimmediately yieldsResourceLimitwithout source reads or callback execution. - Mid-stream Source Interruption: Propagates underlying
io::Errordirectly. Latched failures take precedence. - Overlong Output / Swallowed Writer Failure: Callback writing beyond
expected_lengthlatchesidentity_failure = true.reconstruct_quarantinedchecksstaging.failure()before examining callback return, ensuringMismatchis returned even if the callback swallowed the error. - Hash / Length Corruption: Recomputing BLAKE3 on sealed buffer detects truncation or alteration and yields
Mismatch. Destination is not touched. - Destination Atomic Promotion Failure:
TransactionalContentDestination::promotefailure returns the operational error while preserving prior visible content. - Missing Content: Returns
CapabilityUnavailable. No authenticated absence claim is emitted.
7. Repository Standards Audit
- AGENTS.md / Documentation Rules:
- One physical line per paragraph observed in
echo-keep-physical-content-boundary.md:119,121,123,crates/echo-cas/README.md:16, andCHANGELOG.md:9. - Canonical owners updated; no new numbered ADR allocated.
- SPDX license identifiers present on all 8 modified files.
- One physical line per paragraph observed in
- Rust Standards:
- Edition 2021/2024 compliance; strict Clippy with
-D warningspasses without warnings. - Zero
unwrap(),expect(),panic!, orunsafeblocks in library code. Checked constructors and typed errors maintained.
- Edition 2021/2024 compliance; strict Clippy with
3. Verification Checklist & Coverage Status
| Check Item | Status | Method / Coordinates | Evidence / Coverage Notes |
|---|---|---|---|
| All Code Paths Traced | Verified | Static Inspection (physical_content.rs:1-376) |
All runtime ingestion, publication, view reconstruction, and destination paths mapped |
| Merge History Audited | Verified | Git Tree Audit (git log, git merge-tree) |
Linear branch (3 commits); clean 3-way merge into target main 7dde48b2 |
| Resolved Review Threads | Verified | Code Inspection & Thread History | All 4 GitHub review threads verified resolved in source |
| Constants & Budget Limits | Verified | Manifest & Result Audit | Resource limits, timeouts, and byte limits checked against raw logs |
| Source Hash Parity | Verified | Python SHA256 Check | All 8 working tree files match k02-fresh-fixture-gate.manifest.json |
| Cargo Test Suite (36 tests) | Inspected | Sealed Gate (k02-fresh-fixture-gate.log) |
All 36 CAS tests pass in guarded Linux container |
| Clippy & Workspace Fmt | Inspected | Sealed Gate (k02-fresh-fixture-gate.log) |
-D warnings on Rust 1.90 and cargo fmt --check pass |
| Git Diff Whitespace | Verified | Git Read Query (git diff --check) |
0 whitespace or formatting errors |
| Windows Native Execution | Unavailable | Static Source Inspection Only | Verified stdlib source (sys/fs/windows.rs); native Windows kernel execution unrun |
| Hardware Power-Loss Durability | Unavailable | Protocol Review Only | Explicitly out of scope: receipts declare establishes_durability() == false |
APPROVE
|
Code Lawyer audit at7802d898a933e41fdccd7a8e4651a1e295b4e3b8: all complete-object ingestion, publication, borrowed-view, quarantine, sealing and promotion paths were inspected. The shared suite passes on memory and disk, including repeat publication, source/promotion failure, resource refusal and corruption. Impossible budgets refuse before I/O; excess bytes are Mismatch at any sufficient budget. Test roots are exclusively created and never reuse stale directories. Guarded Docker evidence at the exact candidate:36 CAS tests, strict all-target Clippy on Rust1.90, workspace fmt and whitespace pass. Canonical boundary, package README and changelog describe actual per-object bounds and unsupported evidence. Existing consumers remain compatible. The Windows rename claim is contradicted by pinned Rust1.90 source; native Windows execution is unrun. The earlier P5 wrapping observation is a lawful diagnostic distinction, independently reconciled by agy: reads preserve stream I/O, while publication preserves existing DiskTier path/operation context. Both retain original causes and carry no content proposition on error. Current-head independent agy APPROVE includes its full checklist. All four actionable/disputed threads are reconciled. No authenticated absence, pinned filesystem generation, retention, synchronization, crash-durability or aggregate MemoryTier-budget claim is made. Final merge is conditioned on live head, CI, reviews and protections. |
Existing CAS APIs do not provide a common fallible complete-object boundary with atomic caller output. This additive port stages under explicit per-object byte limits, verifies the Echo hash and exact length, and promotes only a sealed complete object. MemoryTier and DiskTier implement the same borrowed-view and explicit-publication contract. Existing consumers retain compatible APIs.
Closes #760. The shared backend-neutral conformance suite covers staging invisibility, empty/text/chunk-boundary bytes, missing content, wrong length, source failure, capacity refusal, unavailable atomic output, and failed promotion. Disk corruption and publication failure, plus partial and ignored writer failures, preserve previous visible output.
Validation in the reusable guarded Docker worker: all34 echo-cas tests pass on Rust1.90, strict all-target Clippy and workspace fmt pass. The initial narrow run passed all three new tests but Clippy caught an unused test import, corrected before the full gate. Canonical boundary, package README, and changelog state the actual behavior.
Limits: private payload staging is bounded per object, not aggregate retained MemoryTier capacity or RSS. Missing content is an operational capability error; no authenticated absence, pinned filesystem generation, synchronization, retention, or crash-durability evidence is claimed. Keep is not a dependency and current consumers are not rerouted. #761 owns the optional experimental backend.