Skip to content

fix: expose bounded typed Action diagnostics from the runner - #766

Merged
flyingrobots merged 2 commits into
mainfrom
fix/study-operation-diagnostics
Oct 7, 2026
Merged

flyingrobots merged 2 commits into
mainfrom
fix/study-operation-diagnostics

Conversation

@flyingrobots

Copy link
Copy Markdown
Owner

Closes #756.

The runner hid typed non-committed Action outcomes behind a generic scheduler error. Both the first Action and unexpected duplicate paths now report a bounded summary: obstruction kind, footprint-conflict category and blocker count, or a distinct missing-outcome category. The errors omit raw records and invocation data. Retained obstruction encoding stays unchanged.

Docker RED: a real compiler fixture with a 512-byte output bound returned the generic error for a wider projection; the assertion requiring ResultProjectionInvalid failed. Docker GREEN: all 13 runner tests passed, including the new error-kind, 256-byte length, and no-input/no-raw-record checks. Workspace rustfmt and strict Clippy for the changed binary/integration-test targets passed.

Canonical documentation and CHANGELOG describe the diagnostic boundary. No provider artifact, admission rule, WAL schema or runtime semantic change. The previously reported unrelated all-targets xtask unit-test lint is outside this scope; changed targets and the CI binary route pass.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T13:44:06.177387Z 08a2416 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: flyingrobots/echo/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 2736f8dc-31a9-4245-be8e-7ce390604d86
📥 Commits

Reviewing files that changed from the base of the PR and between 6ef53c4 and 08a2416.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • docs/architecture/application-contract-hosting.md
  • xtask/src/run_edict_operation.rs
  • xtask/tests/run_edict_operation.rs
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7fcdb85aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread xtask/tests/run_edict_operation.rs
@flyingrobots

Copy link
Copy Markdown
Owner Author

The outcome summary is the bounded contract; opted-in Rust backtraces are separate diagnostics. The runner test builder now clears RUST_BACKTRACE and RUST_LIB_BACKTRACE for its child, without changing user/global settings or suppressing production backtrace opt-in. Canonical documentation states this scope.

The Docker regression passes when both variables are enabled in the parent. All 13 runner tests, workspace formatting and strict changed-target Clippy pass. Source manifests and the final raw log are retained as s03-hermetic-green evidence.

@flyingrobots

Copy link
Copy Markdown
Owner Author

Independent Adversarial Review: PR #766 (flyingrobots/echo)

  • Branch: fix/study-operation-diagnostics
  • Exact Head: 08a2416315bc014b65fc605839a5984b33c269a7
  • Prior Reviewed Head: b7fcdb85aa5f51b14788dba71f3421a527b80ef4
  • Target / Base: main (6ef53c42db04ef16fae9e90e847203453a211af3)
  • Review Mode: Read-only inspection of clean repository checkout (the reviewed checkout), Git integration history, conversation records, and primary Docker execution evidence. No host tests, Docker runs, commits, merges, or configuration mutations were performed.

1. Findings (P0–P5)

No blocking defects or regressions identified (0 findings).

All changes in the PR diff across both commits satisfy boundary, privacy, hermeticity, and durability requirements:

  1. Unchanged Production Bytes: Production code bytes in xtask/src/run_edict_operation.rs are identical between b7fcdb85 and 08a24163 (git diff b7fcdb85..08a24163 xtask/src/run_edict_operation.rs is empty).
  2. Hermetic Test Child: The follow-up commit 08a24163 explicitly removes RUST_BACKTRACE and RUST_LIB_BACKTRACE from the spawned child runner in runner_command_with_closure. This ensures test isolation against parent/host backtrace configurations without altering global process state or suppressing opted-in user backtraces in production.
  3. Bounded Single-Line Summaries: Error outputs on both Action paths are bounded to concise summaries (~30–95 bytes total message), preventing stderr amplification.
  4. Zero Invocation Data Leakage: Fieldless obstruction categories via EchoOperationObstructionKindV1 omit private invocation keys, raw payload records, and memory addresses.
  5. Raw Record Elimination: The prior raw debug dump (EchoOperationObstructionV1 { ... }) on the duplicate outcome path was eliminated in favor of the shared action_outcome_summary formatter.
  6. No Retained Schema Mutation: The change remains strictly a CLI diagnostic boundary; no WAL schema, provider format, or consensus encoding was altered.

2. Mandatory Verification Checklist

2.1 Every Code Path Traced

  • First Action Failure Path:
    • Production Location: xtask/src/run_edict_operation.rs:379-388.
    • Traced To: action_outcome_summary.
    • Logic: Matches Some(EchoOperationActionOutcomeV1::Committed(receipt)) for success; routes non-committed outcomes (Obstructed, RejectedFootprintConflict, None) to action_outcome_summary(outcome) under error prefix "scheduler did not publish a committed typed Action outcome: ...".
  • Unexpected Duplicate Action Outcome Path:
    • Production Location: xtask/src/run_edict_operation.rs:523-536.
    • Traced To: action_outcome_summary.
    • Logic: Matches Some(EchoOperationActionOutcomeV1::Obstructed(o)) with o.kind() == PreconditionMismatch for expected idempotency rejection; routes all other outcomes (Committed, unexpected Obstructed, RejectedFootprintConflict, None) to action_outcome_summary(outcome) under error prefix "duplicate Action produced unexpected outcome: ...". Replaced previous {outcome:?} debug formatting.
  • Shared Formatter Branches:
    • Production Location: xtask/src/run_edict_operation.rs:967-981.
    • Some(EchoOperationActionOutcomeV1::Committed(_)) -> "committed" (static string, 9 bytes).
    • Some(EchoOperationActionOutcomeV1::Obstructed(obstruction)) -> format!("obstructed: {:?}", obstruction.kind()) (uses fieldless EchoOperationObstructionKindV1, max 39 bytes).
    • Some(EchoOperationActionOutcomeV1::RejectedFootprintConflict(conflict)) -> format!("footprint conflict ({} blockers)", conflict.blocked_by().len()) (uses public slice accessor blocked_by(), ~30 bytes).
    • None -> "missing typed Action outcome" (static string, 28 bytes).
  • Test Child Environment Isolation:
  • Regression Assertion Path:
    • Test Location: xtask/tests/run_edict_operation.rs:416-447 (result_projection_obstruction_reports_a_bounded_kind).
    • Logic: Submits 5,120-byte key exceeding the package's 512-byte output bound; verifies ResultProjectionInvalid is returned in stderr, stderr.len() <= 256, and verifies complete absence of "private-input-marker", "invocation_admission", and "EchoOperationObstructionV1 {".

2.2 Merges and Integration Invariants Audited

2.3 Constants and Claims Checked Against Evidence

  • 512-byte Projection Output Bound: Declared in fixture executable-operation-package.cbor (max_output_bytes: 512). Verified via [s03-red.log:13-16] where 5,120-byte key input triggered ResultProjectionInvalid.
  • 256-byte Diagnostic Stderr Bound: Asserted in xtask/tests/run_edict_operation.rs:440. Actual formatted diagnostic error is ~96 bytes (Error: scheduler did not publish a committed typed Action outcome: obstructed: ResultProjectionInvalid\n), well below the 256-byte limit.
  • Resource Limits in Launch Contract:
    • Build cache limit: 20 GiB (21,474,836,480 B); actual build in [s03-hermetic-green.result.json:3]: 13,201,960,552 B (~12.3 GiB).
    • Runtime data limit: 4 GiB (4,294,967,296 B); actual data in [s03-hermetic-green.result.json:4]: 4,251,475,560 B (~3.96 GiB).
    • Log output limit: 128 MiB (134,217,728 B); actual logs in [s03-hermetic-green.result.json:5]: 15,571,921 B (~14.8 MiB).
    • Free space threshold: Host free 729,564,708,864 B (~679 GiB), VM free 693,547,126,784 B (~645 GiB), well above the 50 GiB minimum.
    • Concurrency/Timeouts: 4 CPUs, 6 GiB RAM, 1000s timeout, monitored fail-closed 2-second watchdog.

2.4 Numeric and Documentation Claims Checked

  • Runner Test Count: Exactly 13 tests reported in [s03-hermetic-green.log:16-31]; exactly 13 tests documented in PR fix: expose bounded typed Action diagnostics from the runner #766 description and [s03-review-followup.md:3].
  • Source Manifest Files and SHA Comparison:
    • Manifest [s03-hermetic-green.manifest.json] tracks 964 files.
    • Manifest header records head: "b7fcdb85aa5f51b14788dba71f3421a527b80ef4", corresponding to the candidate pre-commit state.
    • Verification: SHA-256 hashes for all 964 files in the manifest were compared against the current clean checkout at committed head 08a2416315bc014b65fc605839a5984b33c269a7: 0 mismatches across all 964 files.
  • Documentation and Markdown Standards:

2.5 Error Transitions, Privacy, and State Machine Invariants

  • No Swallowed Errors: Neither bail! swallows the underlying failure context; typed outcomes are surfaced with specific category names.
  • Privacy and Non-leakage: The raw EchoOperationObstructionV1 fields (invocation_id, invocation_admission, evaluation_basis_id, decision_coordinate) are omitted from error formatting.
  • Fail-closed Semantics: Any non-committed outcome during singleton execution triggers an immediate bail, preventing uncommitted state or invalid receipts from advancing execution.

3. Check Execution Status and Coverage Boundaries

  1. Executed in Primary Evidence (s03-hermetic-green.log, s03-red.log):
    • RED witness: [s03-red.log] (exit 101, confirmed generic error previously hid ResultProjectionInvalid).
    • Hermetic regression test run with parent RUST_BACKTRACE=1 RUST_LIB_BACKTRACE=1: [s03-hermetic-green.log:8-11] passed in 0.04s.
    • Complete runner suite: all 13 tests passed in 0.68s ([s03-hermetic-green.log:16-31]).
    • Workspace format: cargo +1.96.0 fmt --all -- --check passed.
    • Strict Clippy: cargo +1.96.0 clippy --locked -p xtask --bin xtask --test run_edict_operation -- -D warnings -D missing_docs passed.
    • Git whitespace: git diff --check passed.
  2. Superseded Intermediate Artifacts:
    • s03-green.log: Compile error (conflict.blocked_by private field access). Superseded.
    • s03-final-green.log: Formatting error (cargo fmt multi-line wrap on line 974). Superseded.
    • s03-stable-green.log: Non-hermetic candidate run on b7fcdb85. Superseded by s03-hermetic-green.log.
  3. Statically Inspected Only (No Execution Fabricated):
    • Formatter branches for Committed, RejectedFootprintConflict, None, and non-ResultProjectionInvalid obstruction kinds (e.g. PreconditionMismatch, BasisChanged, etc.).
    • Unexpected outcome path on duplicate submission (run_edict_operation.rs:532-536).
  4. Unavailable / Skipped:
    • Physical power-loss and hardware crash injection (software process execution only).
    • Broad workspace-wide --all-targets Clippy (scoped to changed binary and test target due to pre-existing unrelated main.rs unit-test lint).
    • Original study raw benchmarks (absent from tree as expected).

4. Final Verdict

APPROVE

@flyingrobots

Copy link
Copy Markdown
Owner Author

Code Lawyer activity summary at 08a2416315bc014b65fc605839a5984b33c269a7: the actual projection refusal names its fieldless kind, both Action error paths share a bounded summary, raw records and invocation values are omitted, and missing outcomes have a distinct category. The backtrace-environment review finding is fixed and published; the parent can opt into backtraces while the test child clears those flags. All 13 runner tests, fmt, strict changed-target Clippy and whitespace checks pass in Docker. No runtime or retained schema change. The exact committed files match the final candidate manifest. All paginated findings are resolved; current-head agy APPROVE includes a full checklist. Final live head/CI/rule verification precedes the already authorized merge.

@flyingrobots
flyingrobots merged commit 18b22e3 into main Oct 7, 2026
40 checks passed
@flyingrobots
flyingrobots deleted the fix/study-operation-diagnostics branch October 7, 2026 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report the typed Action obstruction when the operation runner cannot commit

1 participant