Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 105 additions & 33 deletions .github/workflows/seidroid-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,11 @@ run-name: UCI / seidroid review / ${{ (github.event.issue.number || github.event
# TWO PATHS, one review. An AUTOMATIC review runs on `pull_request` when the caller
# wires that trigger and passes `mode: review`. A MANUAL one runs when a person
# comments `@seidroid review`. Both spend model quota and hold a sandbox, so both are
# gated -- see the guard below: the automatic path reviews a pull request once,
# reviews each later push only where the caller sets `re-review-on-push` or a block
# of its own stands, refuses a draft and honours the skip-review label; and the
# manual path additionally checks who is asking.
# gated -- see the guard below. Both paths refuse a fork-originated pull request. The
# automatic path reviews a pull request once. It reviews a later push only where the
# caller sets `re-review-on-push`, or where a block of its own stands. It also refuses
# a draft and honours the skip-review label. The manual path additionally checks who
# is asking.
#
# This file is the automation of record. It REPLACES `ai-review.yml` rather than
# running beside it; a repository that wires the automatic path here should retire
Expand Down Expand Up @@ -329,14 +330,13 @@ on:
that capability. That acceptance covers code from inside the
organisation.

Fork code sits outside it, and one path still reaches it. A review runs
on the repository the pull request is on, and GitHub withholds this
workflow's secrets from an automatic fork run. An explicit
`@seidroid review` arrives as an issue_comment in the base repository,
which does carry the secrets. PLT-1156 is the control that refuses such
a request on a fork-originated pull request. It is not in this file yet,
so a member who asks for one runs this shell over fork code. Weigh that
before you widen or narrow this list.
Fork code sits outside that acceptance, and the guard refuses it. An
explicit `@seidroid review` arrives as an issue_comment in the base
repository, which carries the secrets. That path reaches a fork's code
unless something stops it. The guard's fork check is what stops it, on
that path and on the automatic one. This shell therefore runs only over
code from inside the organisation. Weigh that before you widen or narrow
this list.
required: false
type: string
default: 'Bash,Read'
Expand Down Expand Up @@ -414,21 +414,26 @@ jobs:
# Set at all because the account default is six hours. The guard only reads
# API state, so a minute is generous.
timeout-minutes: 5
# Read-only, and only what the once-per-PR gate below reads. It makes two
# reads: a review, to find a standing block, on a pull-requests endpoint; and a
# comment, to find a verdict, on the ISSUE comments endpoint. GitHub documents
# that second one as taking either permission, so pull-requests alone serves
# it. issues: read is granted beside it so the read does not rest on that
# alias. A reaction is the stricter case and takes issues alone; the review job
# says so where it needs it.
# Read-only, and only what the guard's checks read. Three reads share it. The
# fork check reads the pull request. The once-per-PR gate reads a review, to find
# a standing block, and a comment, to find a verdict.
#
# The gate prefers the App identity and falls back to this, so a caller that
# configures no App still gets one review per pull request rather than one per
# push. That fallback is why a refused read would cost most here: it fails
# open, so a review runs on every push, and only the run log says why.
# That second gate read goes to the ISSUE comments endpoint. GitHub documents it
# as taking either permission, so pull-requests alone serves it. issues: read is
# granted beside it so the read does not rest on that alias. A reaction is the
# stricter case and takes issues alone; the review job says so where it needs it.
#
# Every reader prefers the App identity and falls back to this, so a caller that
# configures no App still gets all three. A refused read costs them differently,
# and both costs are deliberate. The once-per-PR gate fails open: a review runs on
# every push, and only the run log says why. The fork check fails closed: the
# guard refuses the review rather than run it over code it cannot place.
#
# A caller must grant this workflow at least these two, because a reusable
# workflow may only downgrade what its caller granted.
permissions:
pull-requests: read # the reviews the gate reads to find a standing block
issues: read # the comments it reads to find a verdict
pull-requests: read # the pull request the fork check reads, and the gate's reviews
issues: read # the comments the gate reads to find a verdict
# Runs for an automatic pull_request review, and for any comment-triggered
# dispatch, review or close. For a comment it decides whether the commenter may
# command this workflow at all; for an automatic review it decides whether the
Expand All @@ -438,9 +443,8 @@ jobs:
#
# The pull_request branch carries no author-association check, matching the path
# this file replaces: the event is the push itself rather than a person's
# command, and GitHub withholds this workflow's secrets from a fork pull request
# regardless -- such a run fails the machine-client check below and reviews
# nothing, rather than running an agent over unauthorised code.
# command. The fork check in `Admit the request` refuses code from outside the
# organisation, on this path as well as the comment path.
if: >-
${{ (github.event_name == 'pull_request' && inputs.mode == 'review') ||
(github.event_name == 'issue_comment' &&
Expand Down Expand Up @@ -578,11 +582,18 @@ jobs:
IS_DRAFT: ${{ github.event.pull_request.draft }}
ACTION: ${{ github.event.action }}
RE_REVIEW_ON_PUSH: ${{ inputs.re-review-on-push }}
# What the CALLER routed this dispatch as. The fork check below gates on
# this rather than on COMMAND above, and states why.
MODE: ${{ inputs.mode }}
# The pull_request payload's own repository ids, which spare that path an
# API call. Empty on every other event, where the API answers instead.
HEAD_REPO_ID: ${{ github.event.pull_request.head.repo.id }}
BASE_REPO_ID: ${{ github.event.pull_request.base.repo.id }}
# The App identity where a caller configured one, this workflow's own
# token where it did not. Named apart from GH_TOKEN above because the
# team check has no such fallback: reading an organisation's teams
# needs an identity that can see them, and GITHUB_TOKEN cannot, where
# reading the comments below needs no more than pull-requests: read.
# token where it did not. Named apart from GH_TOKEN above, because the
# team check has no such fallback. Reading an organisation's teams needs
# an identity that can see them, and GITHUB_TOKEN cannot. The reads below
# need no more than pull-requests: read.
GATE_TOKEN: ${{ steps.identity.outputs.token || github.token }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Now that GATE_TOKEN exists and the guard job carries pull-requests: read, the skip-label check below (line 535) could use it too. It currently gates on [ -n "${GH_TOKEN:-}" ], so for a caller that configures no App the skip label is silently inert — the label check fails open on "no identity" even though github.token can now read .labels[].name on the same endpoint this new check already calls. Its fail-open semantics would be preserved (2>/dev/null + no match ⇒ admit); only the "no identity to ask with" case changes, which is exactly the case the new token was added to cover.

Minor, and arguably out of scope for a security fix — but worth noting since the two checks now hit the identical repos/$REPO/pulls/$PR endpoint with different tokens.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the observation. I am declining the change here. It deserves its own ticket, rather than a quiet fix inside a security PR.

You are right on the mechanism. github.token with pull-requests: read can now read .labels[].name on the endpoint this check already calls. The [ -n "${GH_TOKEN:-}" ] gate is the only thing left keeping the skip label inert for a no-App caller.

Why not here. That change is not posture-preserving in the way it looks. Today a no-App caller's ai: skip-review label does nothing, and their reviews run. Switching the token makes the label bite, so a pull request carrying it loses its review. The caller cannot tell that apart from the reviewer breaking, and nothing on the pull request says otherwise. It is the right end state. It is also a behaviour change, and it should arrive under its own heading where a caller can read about it. Not inside a fork refusal.

It also touches the same step #86 edits, and #86 lands first. I am keeping this diff to the smallest thing that closes the hole.

Not verified: whether any live caller actually sets skip-review-label without an App. That is what decides whether the change is invisible or disruptive, and I cannot see the caller repos from here.

run: |
set -uo pipefail
Expand Down Expand Up @@ -622,6 +633,62 @@ jobs:
[ "$state" = "active" ] || deny "$ACTOR is not an active member of $ALLOWED_TEAM; denying"
fi

# A fork pull request carries code from outside the organisation. A review
# clones that code into a sandbox. That sandbox holds a live App credential
# and a shell. Both paths refuse it. ai-review.yml refuses the comment path in
# the same words.
#
# The automatic path needs this as much as the comment path. GitHub withholds
# this workflow's secrets from a fork pull_request run by default. The
# machine-client check below then fails such a run. A private or internal
# repository can turn that withholding off, per repository or by organisation
# policy. This check does not rest on a setting nobody here controls.
#
# A pull_request payload carries both repository ids, so that path spends no
# API call. An issue_comment payload carries no head repository, so the API
# answers there. Repository ids, not names, so a rename does not read as a
# fork. A null head repository reads as a fork, which is the safe reading.
#
# This check fails closed, unlike the label check below. Only a definite "same"
# admits, so an unreadable origin refuses. A failed read costs one refused
# review a person can retry. Admitting on a signal nobody could read costs the
# sandbox above. The step runs without -e, so a failed read leaves the variable
# empty. The API's own error goes to the log, where it says why.
#
# It stops a REVIEW, not a teardown, for the reason the label check states. A
# fork pull request must still be able to reclaim its sandbox.
#
# Keyed on the caller's mode, not on the command this guard parsed. Two readers
# derive those two from one comment body, and they can disagree. This guard
# accepts a bare `seidroid review close`. A caller matching the documented
# `@seidroid` form reads that same comment as a review. Mode decides what the
# review job does, so mode is what this gates on.
if [ "$MODE" != "close" ]; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] This gate is unconditional and has no opt-out input, which changes behaviour for callers whose forks are not "outside the organisation". Internal and private-org repositories routinely use member-owned forks as the normal contribution flow; on those, every automatic review silently stops with no way to restore it short of editing this file. The other two gates in this step are both configurable (allowed-team, skip-review-label).

Consider a review-forks (or similarly named) workflow input defaulting to false, so the secure posture stays the default while an org that forks internally can opt back in — rather than making fork-vs-same-repo a proxy for trust that a caller cannot override.

if [ "$EVENT_NAME" = "pull_request" ]; then
# An empty BASE id means the payload did not carry the signal, which is
# the same standing as a read that failed. It must not compare equal to an
# empty HEAD id and admit.
if [ -z "$BASE_REPO_ID" ]; then
origin=unreadable
elif [ "$HEAD_REPO_ID" = "$BASE_REPO_ID" ]; then
origin=same
else
origin=fork
fi
refusal="$REPO#$PR is fork-originated; not reviewing it"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] This deny is not the last word on the automatic path. Require the machine-client secret (line 582) is gated on steps.parse.outputs.should_run == 'true', not on admit, so it still runs after deny sets admit=false and exits 0. A fork pull_request run on a public repository receives no secrets, so SECRET_PRESENT is false, the step emits ::error::OMNIGENT_MACHINE_CLIENT_SECRET is not set on the calling repository and exits 1 — the Guard job goes red on every external contribution, pointing at a caller misconfiguration that isn't there. That failure was the intended mechanism before this change; now that the fork check owns the refusal, the leftover failure contradicts the notice it just wrote. Adding steps.admit.outputs.admit == 'true' to that step's condition keeps the fail-fast for admitted runs and lets a refused fork end as the clean notice this block is written to produce. (The comment path is unaffected — issue_comment runs do receive secrets.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken. a7bc29a. Require the machine-client secret now reads the verdict as well as the parse:

if: steps.parse.outputs.should_run == 'true' && steps.admit.outputs.admit == 'true'

I checked the ids against the file rather than taking them from the report: the step id is admit and its output is admit, so steps.admit.outputs.admit resolves.

Proved by evaluating the condition the way GitHub would for this expression shape, then running the step's own script when it holds. A fork pull_request run on a public repository, refused by the gate, with no secrets:

revision condition step guard
d4780ce should_run == 'true' runs, exit 1 RED, ::error::OMNIGENT_MACHINE_CLIENT_SECRET is not set…
a7bc29a should_run == 'true' && admit == 'true' skipped GREEN

The fail-fast survives where it belongs:

scenario machine-client step guard guard.should_run
fork pull_request, no secrets, refused skipped GREEN false
admitted review, secret missing runs, exit 1 RED true
admitted review, secret present runs, exit 0 GREEN true
comment parsed to nothing skipped GREEN false

The review job skips either way, since guard.should_run is already parse && admit. Only the guard's colour changes.

The other step that keys on the parse alone, and why I left it. Report a half-configured reviewer identity also reads steps.parse.outputs.should_run == 'true'. Gating it on admit would be wrong. #88 made Admit the request deny when a caller sets half an App credential, and that step's warning is what explains the deny. Reading admit there would suppress the diagnostic exactly where a reader needs it. Those are the only two steps in the guard that key on the parse, so nothing else is affected.

One correction to the reasoning, which does not change the fix. The pull_request fork path does not fail today in every configuration. Your earlier finding named the Actions setting: "Send secrets and variables to workflows from fork pull requests", on a private or internal repository. Such a run does receive the secret, passes this step, and reviews fork code. This gate therefore removes a working behaviour there. That removal is the objective rather than a cost. The PR body now records it under its own heading, so nobody later reads it as an accidental over-port.

Not verified: none of this has run in a GitHub runner. My evaluator handles the && of a.b.c == 'literal' terms only, which is the shape of both conditions here. It is not GitHub's expression engine.

else
origin="$(GH_TOKEN="$GATE_TOKEN" gh api "repos/$REPO/pulls/$PR" \
--jq 'if .head.repo.id != null and .head.repo.id == .base.repo.id then "same" else "fork" end' \
|| true)"
refusal="explicit re-reviews are disabled for fork-originated pull requests; not reviewing $REPO#$PR"
fi
case "$origin" in
same) ;;
fork) deny "$refusal" ;;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] This refusal also permanently strands a standing block on any fork pull request this workflow has already reviewed.

The only thing that withdraws a CHANGES_REQUESTED left by this workflow is a later review run (the dismissal loop at line ~1721), and the standing-block exemption that exists specifically to let that run happen (line 744) is evaluated after this check. So once the fork check denies, neither a push nor an explicit @seidroid review can reach the withdrawal.

That matters for two populations that exist today:

  • Repositories with "Send secrets and variables to workflows from fork pull requests" enabled — the exact configuration this PR targets — where automatic fork reviews have been running and leaving blocks.
  • Any repository at all, via the comment path: before this change nothing refused @seidroid review on a fork pull request, so a member could produce a block on one.

After merge those blocks stand forever and need a repository admin to dismiss by hand, which the input doc at line 356 already warns is restricted on a protected branch.

Moving the check later is not the answer — that would run the agent over fork code, defeating the point. Two cheaper options: mention the manual dismissal in the refusal text so the notice tells the reader what to do, and/or note the one-time sweep in the rollout section of this file alongside the divergence-from-ai-review.yml note.

*) deny "could not read where $REPO#$PR comes from, so a fork cannot be ruled out; not reviewing it" ;;
esac
fi

# The label is a convenience rather than a control, so it fails open: it
# stops a review someone did not want, and being unable to read it must
# not stop every review when no identity is configured.
Expand Down Expand Up @@ -740,8 +807,13 @@ jobs:
#
# Only ever tests emptiness -- the value is never echoed, compared against a
# literal, or written to an output.
#
# Reads the verdict as well as the parse. `deny` exits 0, so every step after it
# still runs. A refused request needs no credential, and a fork pull_request run
# holds none: without the verdict here, refusing one would paint a deliberate
# refusal red and name a caller misconfiguration that does not exist.
- name: Require the machine-client secret
if: steps.parse.outputs.should_run == 'true'
if: steps.parse.outputs.should_run == 'true' && steps.admit.outputs.admit == 'true'
env:
SECRET_PRESENT: ${{ secrets.OMNIGENT_MACHINE_CLIENT_SECRET != '' }}
run: |
Expand Down
Loading