feat(seidroid-review): publish the review check under both names

[bdchatham]

Every review run publishes its check run twice, as review and as AI Review. Both publish sites carry both names. A caller can turn the copy off.

What changed

Three sites in .github/workflows/seidroid-review.yml.

Publish the review check run sends the same conclusion, title and summary under both names. It reads the check artifact once, so the two cannot differ. review is unchanged: same name, same conclusion rule, same error posture.

Post the verdict, on the path where the verdict comment fails to post, publishes its failure check run under both names as well. It named only review before, so the copy kept the review's earlier conclusion and a rule naming AI Review read green on a verdict that never arrived.

publish-ai-review-check, a new boolean input, default true.

The decision

The repository owner decided to publish both names (PLT-1152). A branch-protection rule matches a check by its name, so only a check called AI Review satisfies a rule that requires AI Review. A repository that retires ai-review.yml under such a rule waits forever on a check nothing publishes, and the rule reads as pending, not failed. The rules on the calling repositories are org-level, and no token in this session can read them. Publishing both names is the safe answer under a rule nobody can see, so the input defaults to on. Delete the copy once someone who can read those rules confirms that none names AI Review.

publish-ai-review-check: false is for a repository that runs both tools and knows its own ruleset. Both tools then publish AI Review, GitHub lists only the check run that completed last, and this copy can hide the incumbent's verdict. The switch does not remove that race. It makes the race a choice, and the description says so.

Error posture

Site	review	AI Review
Publish the review check run	exits the step under set -e, and continue-on-error decides the job, as today	warns, and the step still succeeds
Post the verdict fallback	warns, as today	warns

review publishes first at both sites. The copy can never be the reason the primary fails to post. The missing-head_sha paths are unchanged: the publish step fails the job, the fallback warns.

The two-publisher collision

The ticket asked me to confirm this rather than assume it. Half of the premise holds.

Two check runs of one name on one commit are legal. GitHub does not show both. GET /repos/{owner}/{repo}/commits/{ref}/check-runs defaults to filter=latest, which returns one check run per name per check suite. The checks list on the pull request reads that default view.

Measured on sei-protocol/sei-chain commit 1a086bc, app codecov, check suite 91901537092:

filter=all   codecov/project x3   ids 101144306086, 101144546091, 101145910119
default      codecov/project x1   id 101145910119, the latest completed_at

One name plus one app plus one commit therefore means that the later publish supersedes the earlier one in the view that matters. sei-internal-skills posts both ai-review.yml and seidroid under the App slug seidroid: its AI Review check run on commit 530588c carries "app": {"slug": "seidroid"}. On that repository the two AI Review publishes race for the name, and the checks list shows whichever completed last. That measurement is why the input exists.

Duplicates do sit side by side across check suites. Commit 530588c carries ai-review / Claude nine times, one per workflow run.

Verification

I extracted both steps with a YAML parser and ran them under bash with a gh stub on PATH.

Publish the review check run:

Case	Exit	Check runs posted	Result
both publish	0	review, AI Review	one conclusion, title and summary on both; log reads published check runs review and AI Review: success — review: 0 blockers
the copy fails	0	review	warning raised; log reads published check run review: success — ...
review fails	1	none	the step exits, and the copy is not attempted
no head_sha	1	none	::error::the reviewed commit was not recorded on ...
no verdict	0	review, AI Review	both carry failure, and the file says success
empty check file	0	none	no check run to publish
copy off	0	review	log reads published check run review: ...
malformed check file	0	review, AI Review	title reads review; on the base branch it is empty

Post the verdict, the failure fallback:

Case	Exit	Check runs posted	Result
the comment posts	0	none	posted the verdict on ...
the comment fails, copy on	0	review, AI Review	both failure, both titled review produced but not published; the base posts review alone
the comment fails, copy off	0	review	failure, as the base does
the comment fails, no head_sha	0	none	::warning::no reviewed commit was recorded ...

The base branch's scripts exit the same way on all twelve cases.

The harness reads the workflow-level env: out of the file rather than restating it. #86 hoisted VERDICT_MARKER from the Post the verdict step env to a workflow-level env:, and a harness that named the old key modelled a step that no longer exists. All four fb- cases run through the line that reads it.

actionlint over .github/workflows/*.yml, base 5f5fd78 and this branch: 6 action, 30 shellcheck (1x SC1102, 25x SC2086, 4x SC2102), 1 syntax-check. Identical. Both new gh api calls sit inside shell functions, so the output[title] and output[summary] literals stay at four.

Not verified

• Nothing ran on a GitHub runner. The stub proves the scripts; the API did not.
• I read no org ruleset. Nobody in this session can read one. Whether a rule names AI Review is still unknown.
• I measured the superseding behaviour on the list endpoint. I did not create two check runs of one name and watch a branch-protection rule resolve them.

🤖 Generated with Claude Code

[cursor]

PR Summary

Medium Risk
Changes CI merge-gate signaling and can collide with legacy ai-review.yml on the AI Review check name, though the new input and primary-vs-copy error handling limit blast radius.

Overview
Adds publish-ai-review-check (default on) so repos retiring ai-review.yml can still satisfy branch protection that requires a check named AI Review, while callers still running both workflows can turn the duplicate off to avoid two publishers racing for the same name.

Publish the review check run now posts the same conclusion, title, and summary twice when enabled: review remains the merge gate (failure fails the step); AI Review is best-effort (warn only). Post the verdict’s “could not post comment” fallback uses the same name set so a stale green AI Review cannot hide a missing verdict.

^{Reviewed by Cursor Bugbot for commit c4a006f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit a909f42. Configure here.}

[github-actions]

The dual publish itself is implemented cleanly (single artifact read, review first and authoritative, copy best-effort), but the failure fallback further down the workflow still supersedes only review, so the AI Review copy can remain green when the review failed to post — defeating the branch-protection rule this PR is meant to satisfy.

Findings: 1 blocking | 1 non-blocking | 1 posted inline

Blockers

• None at the file/PR level.
• 1 blocking issue(s) flagged inline on specific lines.

Non-blocking

• The copy can mask a genuine AI Review failure during the transition. On a repo still running ai-review.yml under the same App, if seidroid concludes success and completes after a real AI Review failure, filter=latest shows green — the race the step's own comment describes, but in the direction that weakens a gate rather than just reordering it. Consider gating the copy behind a workflow input (default off, or off for repos still running ai-review.yml) so the transition is opt-in per caller rather than unconditional.

Comments that couldn't be anchored to the diff

• .github/workflows/seidloid-review.yml:1137 -- [nit] || true swallows a jq failure and leaves title empty, which the check-runs API rejects (output.title is required), so a malformed check file now fails with a 422 from gh instead of jq's own error. A default keeps the publish working: title="${title:-review}" after the assignment.

[github-actions]

[blocker] The AI Review copy is never superseded by the failure fallback, so it can stay green on the exact path the fallback exists to red-flag.

The "State the review's position on the pull request" step (~line 1751) publishes a second check run when the verdict comment could not be posted:

gh api -X POST "repos/$REPO/check-runs" -f name=review ... -f conclusion=failure \
  -f output[title]="review produced but not published"

Its comment says it posts "under the same name as the check published above, so it supersedes that conclusion on this commit rather than sitting beside it." That reasoning held when this step published exactly one name. It now publishes two, and the fallback only supersedes review.

Result on that path: review = failure / "review produced but not published", while AI Review still carries the original conclusion — success when the review was clean. A branch-protection rule that requires AI Review (the whole premise of this PR) is then satisfied by a review that never reached the pull request. The same gap applies to the no-verdict path, where this step forces conclusion=failure before publishing but the fallback path can't correct the copy.

Fix: make the fallback publish both names too, e.g. reuse the same list of published names (write them to $GITHUB_OUTPUT here and loop over them there), so any name this step published is superseded by any name the fallback publishes.

[bdchatham]

Fixed in a4f0cc3. Post the verdict publishes the fallback under both names now, review first and both best-effort. I did not pass the names through $GITHUB_OUTPUT. Both sites read the same publish-ai-review-check input instead, so one switch decides the set of names. That set cannot drift when the publish step dies before it writes an output.

Stub run, comment POST failing: copy on posts review and AI Review, both failure, both titled review produced but not published; copy off posts review alone; base d477b7d posts review alone in both cases.

[bdchatham]

Both taken, one with a deliberate difference. a4f0cc3.

The input, with the default the other way up. Added publish-ai-review-check, defaulting to true, not off. The owner decided that the copy is safe by default under a ruleset nobody here can read, and that decision stands. A caller that knows its own ruleset sets it false. The description says exactly when: the repository still runs ai-review.yml AND a rule names AI Review.

Your finding is the reason the switch exists. The measurement behind it is in the PR body, and it is what makes the race weaken a gate rather than just reorder it.

The nit, applied with one correction. title="${title:-review}" is in. I could not confirm the 422, so the code does not claim it. output.title requires a present value, and an empty string is present.

What I did measure is the loss of the intended default. On base d477b7d a malformed check file publishes title=[]; with the fix it publishes title=[review]. The summary needs no equivalent line, because // "" and a failed jq both yield the empty string.

[github-actions]

Publishing the review check under both review and AI Review is implemented correctly at both publish sites, with sound error posture (review stays the only publish that can fail the step) and a useful incidental fix to the check title default. One documentation inconsistency in the new input's description; no correctness or security problems found.

Findings: 0 blocking | 5 non-blocking | 1 posted inline

Blockers

• None at the file/PR level.

Non-blocking

• Both publish sites are covered — grep for check-runs finds exactly the two gh api -X POST calls this PR touches, so there is no third site left publishing review alone. Ordering is also right: Publish the review check run runs before Post the verdict, so the fallback's failure supersedes both earlier names rather than sitting beside them.
• publish-ai-review-check defaults to true, and the PR body identifies sei-internal-skills as a repository that runs both tools under the App slug seidroid — i.e. a caller where the default is the wrong setting on day one. The caller lives outside this repo so it cannot be fixed here, but worth landing publish-ai-review-check: false in that caller alongside (or before) this merge rather than after, otherwise the first seidroid run there can overwrite ai-review.yml's AI Review verdict.
• Nice incidental hardening: title is now computed once into a variable with a ${title:-review} fallback, so the two publishes cannot disagree and a failing jq no longer yields an empty check title the way the base branch does. summary correctly needs no equivalent guard.
• The AI Review copy failing is best-effort (warning only), which is the right posture — failing the step would not publish the check either. Worth being aware that in the one scenario the copy exists for (a rule requiring AI Review), a failed copy leaves that rule pending with only a workflow annotation to explain it.
• 1 suggestion(s)/nit(s) flagged inline on specific lines.

[github-actions]

[suggestion] The guidance here is narrower than the actual collision. It says to set this false only where the repository runs ai-review.yml AND a rule names AI Review, but the harm the next sentence describes — "GitHub lists only the check run that completed last, and this copy can hide the incumbent's verdict" — happens whenever both tools publish the name under this App, rule or no rule. With no rule, nothing blocks the merge, but the checks list still shows seidroid's conclusion in place of ai-review.yml's; a green copy landing after a red incumbent silently replaces the visible signal a human reads.

The step comment at the publish site ("A repository that also runs ai-review.yml under this App gets two checks called AI Review on the commit") states it correctly without the rule qualifier. Suggest dropping the AND a rule names \AI Review`` condition here so the input's own documentation matches it.

Superseded: latest AI review found no blocking issues.

[github-actions]

A focused, unusually well-documented change that publishes the review check run under both review and AI Review, gated by a new publish-ai-review-check input, at both the success and verdict-failure publish sites. The shell is correct and the two sites are consistently gated; my two notes are about the true default trading a fail-safe outcome (merge blocked) for an unsafe one (one tool's verdict masking the other's), and about a failed copy publish being reported only as a warning.

Findings: 0 blocking | 7 non-blocking | 2 posted inline

Blockers

• None at the file/PR level.

Non-blocking

• Hoisting title/summary into variables before the two publishes is a genuine improvement over the previous code, which re-ran jq for the trailing echo and so could have printed a title different from the one published. The ${title:-review} guard also correctly covers the empty-string case that jq's // does not.
• The two publish sites are consistently gated on the same inputs.mode == 'review' condition and read the same AI_REVIEW_COPY env, so the set of names published on the success path and on the verdict-failure path does match, as the comment at line 2100 claims.
• No test or dry-run coverage exists for either publish path, so the only verification of the new branch is the workflow running for real. That is the pre-existing posture for this file, not something this PR changed, but the new conditional doubles the number of untested paths through the publish step.
• 2 suggestion(s)/nit(s) flagged inline on specific lines.
• 2 non-blocking pre-existing issue(s) listed below under pre-existing issues.

Pre-existing issues

• [suggestion] .github/workflows/seidroid-review.yml:1427 — summary is read from the driver's check artifact with no length bound before being sent as output[summary]. The GitHub checks API caps output.summary at 65535 characters; an oversized summary makes the publish 422 and, on the review publish, fails the step. The verdict comment path a few hundred lines below does bound its body (see the MaxBodyBytes comment at line ~1896); the check publish does not.
• [suggestion] .github/workflows/seidroid-review.yml:1394 and :1863 — GH_TOKEN falls back to github.token when the App identity step produces nothing. A check run created by github-actions lands in a different check suite than one created by the seidroid App, so on a fallback run the filter=latest supersession reasoning in the new comments (and in the PR description's measurement) does not hold: the copy would sit beside the App's check rather than replacing it.

[github-actions]

[suggestion] Defaulting the copy to true swaps a fail-safe failure mode for an unsafe one.

The two error states are not symmetric:

• Copy off, rule requires AI Review, nothing publishes it → check stays pending → merge is blocked. Noisy, but safe.
• Copy on, repo still runs ai-review.yml, seidroid completes last → seidroid's conclusion is the one filter=latest returns. If ai-review.yml concluded failure and seidroid concludes success, the required check reads green and the merge that the incumbent blocked goes through.

The step comment and the input description both name the race, but the default picks the side where the failure mode is a bad merge rather than a stuck one. Since a caller that has already retired ai-review.yml is the only one that needs the copy, and that caller is by definition making a deliberate migration change, default: false with an explicit opt-in gets the same coverage without ever letting one tool's verdict mask the other's. If the owner has decided on true (PLT-1152), consider at least scoping it to a documented removal date so no repo silently carries the race indefinitely.

[github-actions]

[suggestion] A failed AI Review publish reintroduces exactly the state this input exists to prevent, and does it silently.

If publish "AI Review" fails (transient 5xx, secondary rate limit, token scope), this warns and the step succeeds. A caller that retired ai-review.yml under a rule naming AI Review is then back to the pending-forever case from the input description — a green job, a warning annotation nobody reads, and a merge blocked on a check that will never arrive.

Keeping the copy from failing the review is right; making it indistinguishable from success is not. ::error:: instead of ::warning:: would at least surface it as an annotation on the run without changing the step's exit, and it costs nothing given continue-on-error already governs this step.