Skip to content

fix: preserve reviewer results after tool failures - #585

Merged
zzwong merged 2 commits into
mainfrom
zzwong/issue-550/retain-reviewer-results
Sep 7, 2026
Merged

fix: preserve reviewer results after tool failures#585
zzwong merged 2 commits into
mainfrom
zzwong/issue-550/retain-reviewer-results

Conversation

@zzwong

@zzwong zzwong commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #550.

The pipeline assigns incomplete_tool after obtaining a reviewer result when the required diff tool failed, did not finish, or was not invoked. The shared coverageResultProduced helper omitted that status, so consumers incorrectly treated a returned result as missing: the findings table displayed “did not run,” JSON emitted ran: false, and coverage details omitted known-empty skipped-file and constraint lists.

This recognizes incomplete_tool as a produced result. Readers can see what the reviewer actually returned while the coverage warning explains why the review cannot approve.

Before and after

For a reviewer that returned two findings, inspected its assigned file, and reported no skips or constraints, but whose diff-tool evidence was unsuccessful:

Surface Before After
Findings table ⚠️ did not run 2
JSON reviewer row "findings": 2, "ran": false "findings": 2, "ran": true
Coverage details Empty skipped/constraint fields omitted skipped: none; constraints: none

The same distinction applies to a real zero-finding result: it displays 0. A reviewer that produced no result still displays “did not run.” The tool-failure warning remains visible, and incomplete coverage still clamps an otherwise approving review to comment.

Implementation and scope

The production change adds incomplete_tool to the existing result-produced classification shared by Markdown rendering and the JSON view. Regression coverage exercises those consumers with zero and nonzero counts, known-empty lists, and genuinely missing results. Existing tests continue to cover the warning and approval withholding.

This is a focused reporting correction. #584 separately documents the persisted tool-evidence contract.

Verification

  • The new Markdown and JSON regressions failed with the previous classification and passed with the fix.
  • All 143 tests in internal/reviewplan and internal/view passed. Formatting, lint, and build checks passed with Go 1.26.3.
  • CI passed, including tests, the 1Password opt-out configuration, static smoke tests, lint, and Linux/macOS/Windows builds.

The local full-suite run encountered a separate validation limitation: TestPiRPCReviewerExtensionLoadsInInstalledPi timed out during the installed Pi help preflight, and a focused retry reproduced that timeout. The affected-package tests and the CI test jobs above passed.

@monit-reviewer monit-reviewer left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR Review

Reviewed commit: 2228b4107641
Profile: open-cli-collective-review - Posting as: monit-reviewer

Summary

Reviewer Findings
go:implementation-tests 0
structure:repo-health 0

Reviewer Coverage

  • go:implementation-tests — complete (broad); skipped: none; constraints: Review limited to the assigned changed Go files and read-only repository inspection; tests were not executed.
  • structure:repo-health — complete (broad); skipped: none; constraints: Review limited to the three assigned changed files and their immediate status-producer/JSON-renderer context.
Inspected files (3)
  • internal/reviewplan/summary.go
  • internal/reviewplan/summary_test.go
  • internal/view/review_reviewer_ran_test.go

0 PR discussion threads considered. 0 summarized; 0 resolved.


Completed in 1m 46s | $0.04 | openai-codex/gpt-5.6-terra | cr 0.10.299
Field Value
Model openai-codex/gpt-5.6-terra
Reviewers go:implementation-tests, structure:repo-health
Engine pi_rpc · openai-codex/gpt-5.6-terra
Reviewed by cr · monit-reviewer
Duration 1m 46s wall · 2m 47s compute
Cost $0.04
Tokens 14.2k in / 617 out

Per-workstream usage

  • orchestrator-selection — openai-codex/gpt-5.6-terra
    • In: 5.8k
    • Out: 178
    • Cache read: 0
    • Cache create: 0
    • Cost: $0.01
    • Duration: 5s
  • go:implementation-tests — openai-codex/gpt-5.6-terra
    • In: 4.5k
    • Out: 203
    • Cache read: 20.0k
    • Cache create: 0
    • Cost: $0.02
    • Duration: 1m 27s
  • structure:repo-health — openai-codex/gpt-5.6-terra
    • In: 2.6k
    • Out: 199
    • Cache read: 20.0k
    • Cache create: 0
    • Cost: $0.01
    • Duration: 1m 11s
  • orchestrator-rollup — openai-codex/gpt-5.6-terra
    • In: 1.2k
    • Out: 37
    • Cache read: 2.8k
    • Cache create: 0
    • Cost: $0.00
    • Duration: 2s

@zzwong
zzwong marked this pull request as ready for review September 7, 2026 23:40
@zzwong
zzwong merged commit c5f35b9 into main Sep 7, 2026
19 of 20 checks passed
@zzwong
zzwong deleted the zzwong/issue-550/retain-reviewer-results branch September 7, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: Preserve reviewer results in incomplete_tool summaries

2 participants