Conversation
A build-and-push workflow is a deploy gate: when it fails, no image lands, the deployer has nothing new to sync, and prod just keeps serving whatever image last made it through. That failure is silent, the branch still looks green, and the only trace is a grey cross nobody opens. It happened on angkor-platform-frontend: Trivy started failing before the push step, main went unshipped for days, and prod kept serving an image with known auth-bypass CVEs. It only surfaced because someone's unrelated fix never showed up in prod. Add a reusable workflow_call workflow that checks, on the caller's own schedule, whether a branch's head is covered by a successful run of a named gating workflow. Quiet inside a grace window (default 90 minutes), then it opens one deduplicated labelled issue and exits 1, closing that issue again once the branch ships. It's a watchdog on purpose, not an on-failure hook: a hook only fires when the gate ran and lost, never when it didn't run at all (disabled, path-filtered away, runner outage, schedule stopped), which is exactly what stayed hidden the longest here. Runs on the built-in GITHUB_TOKEN, no Slack webhook or org secret, since the repo that needed this has neither and no admin to add one. Every input is checked against an allowlist before any API call, with distinct exit codes (0 shipped/in grace, 1 stale, 2 bad args). Tests in tests/deploy-gate-watchdog/ extract the watchdog's shell body straight out of the workflow YAML and run it against a stubbed gh, so there's one copy of the logic and the suite is pinned to what actually ships. Logic stays inline instead of in a script file because a reusable workflow runs in the caller's checkout, where a script from this repo isn't on disk. Added repo self-CI (ci.yaml) running the same commands locally and in CI, plus tests/lint_workflows.sh running actionlint with a shrink-only exemption list for five workflows that already had findings. actionlint and PyYAML are pinned as tool directives with matching Dependabot entries so the pins get bumped instead of rotting. Replaying the original incident through the watchdog's own query shows one run, concluded failure, zero successes, so it would've been reported within the grace window instead of days later. Callers need to grant actions: read and issues: write; the reusable workflow's permissions block can only cap what's granted, never add to it. ANG-2720
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I noticed that a failing build-and-push gate is silent, so production keeps serving a stale image. On angkor-platform-frontend that meant days on an image with known auth-bypass CVEs.
The fix is a reusable workflow_call watchdog that checks whether the branch head is covered by a successful run of a named gating workflow, opens one deduplicated issue after a grace window, and closes it once the branch ships again, all on the built-in
GITHUB_TOKEN.It's a watchdog and not an on-failure hook, since a hook only fires when the gate ran and lost, never when it never ran at all. Callers grant
actions: readandissues: writethemselves.