Skip to content

feat(harness): keep partial replies and end crash loops visibly - #124

Merged
sanchitrk merged 4 commits into
canaryfrom
claude/cod-118-recovery-08e2c8
Nov 1, 2026
Merged

sanchitrk merged 4 commits into
canaryfrom
claude/cod-118-recovery-08e2c8

Conversation

@sanchitrk

Copy link
Copy Markdown
Contributor

Summary

  • Aborted, failed and crash-healed replies keep their streamed parts. Unfinished tool calls in them settle aborted and never run. Aborted replies stay out of model context.
  • Streaming replies are checkpointed durably (session.llm.progress) at most once a second, so a crash loses at most about 1 s of text.
  • A reply cut off at the output limit gives its tool calls a "re-issue" error and the run continues. Two such replies in a row end the run as truncated.
  • The attempt of the request in flight is now persisted (thread.attempt, added in place to the init migration). A request cut off by a host crash counts as a failed retry attempt; with no retries left, the run ends as interrupted. A clean shutdown is not counted.
  • A crash while tools ran commits the turn, with the unfinished tools aborted and finished results kept. Nothing re-runs, and this commit does not reset the restart count.
  • At the restart limit the run ends with a durable session.run.failed (crashloop) instead of silently parking the thread. parked is removed from the thread info.
  • codework run warns and exits 1 when the final response is truncated.
  • Fixes a race in the MCP OAuth test fixture: the revoke trigger also fired on the keyed server's listing.

Existing databases need to be recreated, because the init migration was edited in place.

Test plan

  • vp check, vp run typecheck
  • packages/codework/test/recovery.test.ts: 7 cases that SIGKILL a real server and reopen it (mid-stream, mid-tool, retries exhausted, crashloop, tool crash loop, truncated loop, loaded tools surviving a restart), each ending in an __artifacts__ snapshot
  • cli.test.ts (truncated run), cli.mcp.oauth.test.ts (3 runs in a row), and the harness runner/thread tests
  • Unrelated failures that exist on the base too: the model.list tests in server.test.ts and the model-catalog tests in settings.llm.test.ts

🤖 Generated with Claude Code

sanchitrk and others added 4 commits November 1, 2026 16:17
…sting

The fixture's afterList trigger also fired on the keyed server's listing,
which usually connects first, so tokens were revoked before the secure
server connected and its tool was never offered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Aborted, failed and crash-healed replies keep their streamed parts;
  unfinished tool calls in them settle aborted and never run.
- Streaming replies are checkpointed durably (session.llm.progress) at
  most once a second, so a crash keeps the partial.
- A reply cut off at the output limit gives its tool calls a re-issue
  error and the run continues; two in a row end it as truncated.
- The in-flight attempt is persisted: a request cut off by a host death
  counts as a failed retry attempt, and with none left the run ends as
  interrupted.
- A crash while tools ran commits the turn with the unfinished tools
  aborted; that commit does not reset the restart count.
- At the restart limit the run ends with session.run.failed crashloop
  instead of parking the thread.
- Kill/reopen E2E against the real server process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sanchitrk
sanchitrk merged commit 5b425b5 into canary Nov 1, 2026
2 checks passed
@sanchitrk
sanchitrk deleted the claude/cod-118-recovery-08e2c8 branch November 1, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant