diff --git a/.github/lint/README.md b/.github/lint/README.md new file mode 100644 index 0000000..e162401 --- /dev/null +++ b/.github/lint/README.md @@ -0,0 +1,238 @@ +# lint — org repo literal allowlist + +`check-org-repo-literals.sh` + `org-repo-allowlist.txt` are the lint behind +[`test-org-repo-literals.yml`](../workflows/test-org-repo-literals.yml). They +make AGENTS.md's first convention — *never leak private caller names* — a CI +gate instead of something a reviewer has to remember, the same move +[`test-workflow-pins.yml`](../workflows/test-workflow-pins.yml) made for the +`workflows_ref` default rule. + +## The rule + +Every org-prefixed repo literal in this repo's **tracked** tree must have its +repo-name part on `org-repo-allowlist.txt`. Anything else fails the run, with +`unapproved: file:line: match` for each hit plus a matching `::error` +annotation. The `unapproved: ` prefix is constant and load-bearing: without it +the line starts with the tracked path, and a tracked file named +`::stop-commands::x.md` would emit a *workflow command* at column zero that +suppresses every annotation after it. The annotation's `file=` value is +percent-escaped for the same reason, and `line=` is *validated* rather than +escaped: a line number is always digits, so a non-numeric field means the +`file:line:match` record did not parse and the run refuses it (exit `2`). The +record's **shape** is checked too — after the boundary drop the match must still +begin with the org prefix — because a numeric line field does not prove the split +landed where it appears to. The way both fire is a tracked path containing a +newline on the `grep -r` **fallback** path, which prints paths raw: one hit +arrives as two records, and the leading fragment would otherwise be reported as a +literal cut out of a filename against a path that does not exist. The **git** +path has no split — `git grep` C-quotes newline, tab, `\` and `"` in a path +whatever `core.quotePath` says (that governs only bytes ≥ 0x80) — so there such a +file is one record, a real finding with a C-quoted location. + +The org segment has to **start a token**, so `Not/whatever` — a different +owner whose name happens to end in ours — is not a reference to this org and is +not reported. That is `public-repo-hygiene`'s left-boundary rule verbatim, so +the two checkers agree on what counts as a reference. A token boundary is not an +*owner-name* boundary, though: `-` is legal in a GitHub owner name and is not an +identifier character, so the hyphenated sibling `Not-/whatever` **is** read +as one of ours. That false positive is limitation 8 below, kept rather than fixed +so the boundary stays byte-identical to that checker's. + +**Allowlist, never denylist.** A denylist grep would have to commit the private +names into this public repo in order to match them — the lint would *be* the +leak. Default-deny inverts that: the committed list holds publishable names +only, and publishing a new one is an allowlist edit that shows up in review. + +## Adding a name + +Add one line, lowest-friction spelling first, with a trailing `#` comment saying +why it is safe: + +``` +comfy-cli # public (also on the org-wide PUBLIC_COMFY_ORG_REPOS list) +``` + +Entries are literal names, compared case-insensitively; a trailing `.git` suffix +and trailing periods are stripped from the literal before comparison. A trailing +`-` or `_` is **not** stripped — both are legal at the end of a real slug, so +stripping them would let `-` clear on ``. For a +**real** repo, "why it is safe" means you confirmed it is public. If it is +private, the fix is to remove the reference, not to add the line. + +**Team handles are a separate namespace.** An entry written with a leading `@` +clears only an `@`-prefixed literal — a CODEOWNERS team handle, or an npm / +GitHub Packages scope. A plain entry clears either spelling, because +`@/` is also how a package scope for a repo of that name is written. +Scoping team slugs this way is what stops a CODEOWNERS fixture's team name from +also clearing a literal reference to a *private repo* of the same name; the +org-wide checker splits the two lists for the same reason. + +Three shapes of entry are a configuration error (exit `2`) rather than an +allowlist line, because each reads as an ordinary edit while doing something +other than what its author meant. + +**No globs.** An entry containing `*`, `?`, `[` or `]` is rejected. Note what +this guard is today: entries are compared with *exact equality*, so a `*` entry +already clears nothing but a literal `*` — nothing reaches the comparison as a +pattern any more. The rejection is **defence-in-depth** against that comparison +going back to a `case` glob, kept so a later reader "simplifying" `=` into one +does not silently re-arm every metacharacter on the list. It is also why an +earlier revision's approach — testing entries against two fixed probe names — was +dropped: two probes cannot decide breadth in general (`[!qz]*` and `[a-p]*` match +neither yet clear nearly everything). Enumerate a test-fixture family rather than +globbing it: a `secret-*` line would pre-approve an unbounded set of names on a +list whose entire value is that each name was reviewed once. + +**No `/`.** Entries are bare repo *names* — the comparison runs against the name +alone, so an entry written `/foo` can never match anything. That is exactly +the spelling the tool invites, since a finding quotes the whole literal and the +summary says to add it to the allowlist, so accepting it would yield an inert +line and a rerun still insisting the name is missing from a file that visibly +contains it. Rejected with the spelling that does work. + +**No name ending in the org name.** That is the shape of a *merged* literal, not +a repo name: `grep -o` takes non-overlapping matches, so `/public./private` +scans as one match whose name reads `public.` and the second name is never +examined. The suffix test is complete rather than a sample of separators — the +merged token always ends at the `/` that starts the second name, so `foo.`, +`foo-`, `foo_` (the name class holds `_`) and the bare `` that +`//private` produces are all one rule. It fails closed +(no merged token is on the list, so the line is always a finding), but the +obvious way to silence so confusing a finding is to allowlist the token it +quotes — and that would clear the private reference with the run green. Making +the merged token unallowlistable removes the foot-gun. + +## Why not `public-repo-hygiene`? + +[`.github/public-repo-hygiene/`](../public-repo-hygiene/) is the rigorous, +org-wide implementation of this same default-deny idea, and it is what every +*other* repo should adopt. This repo cannot adopt it as a caller: it is that +checker's own **home**, so its tests and docs deliberately commit fake-private +fixture names, quote internal collaboration-tool hosts, and reference ticket ids +by convention throughout — all three of that checker's categories, all +intentional. This lint is the one category this repo *can* enforce on itself +today. It is deliberately small; it is not a second copy of that checker, and +new detection belongs there rather than here. + +## Known limitations + +Run `bash check-org-repo-literals.sh --help` for the authoritative list — the +script's numbered KNOWN LIMITATIONS block. Every entry there has a bullet below, +in the same order, except limitation 5 (*this lint is one category of +`public-repo-hygiene`*), which is the section above. In short: + +- Only **org-prefixed** literals are caught. A bare repo name cannot be linted + without a denylist that is itself the leak — that half stays with review and + AGENTS.md. +- The name class is **ASCII**, so a name whose tail is non-ASCII (a U+2010 + homoglyph dash, a U+017F long s, CJK text) is read only as far as its ASCII + prefix. The class cannot simply be widened to high bytes: this tree + legitimately writes an ellipsis, an em dash and CJK text flush against a real + name in prose, so widening turns those into findings. The scan runs under + `LC_ALL=C` so this limit is the same on every machine rather than a function + of the runner's locale. Closing it properly needs the offset-aware scan + [`public-repo-hygiene`](../public-repo-hygiene/) already implements. +- **Binary blobs are never read.** Both scan paths pass `-I`, so a UTF-16 file, + a file carrying a stray NUL, or one `.gitattributes` line (`*.md binary`) + takes whole file types out of the scan with no signal in the run output. Text + is the only surface this lint claims. +- **Only a literal written whole is caught.** An org literal assembled at run + time — `printf '%s/%s' '' ''`, `"${ORG}/"`, a name split + across a concatenation — never matches, so it needs no allowlist edit while + the run still reports its scope clean. That is not hypothetical here: this + lint's own smoke tests use exactly that spelling on purpose, which makes it + the house style for org literals in this repo's workflow files. It stays + fully human-readable in the source, so review is what catches it. +- **A team `@` is judged by position, not by grammar.** An `@` glued to the tail + of an identifier (`user@/`, email-shaped) still reads as a scope, so + such a literal can draw on the `@`-scoped team entries. Reaching a private name + through that needs the name spelled exactly like an allowlisted team slug. + `public-repo-hygiene` reads the `@` from the same position and shares the + residual. +- **`_` is identifier-continuation in the left boundary**, so a literal written + directly after an underscore is not read as a reference at all — the + markdown-italic spelling `_/_` matches nothing. The same accepted + trade `public-repo-hygiene` makes, kept identical on purpose so a name cannot + pass one checker and fail the other. +- **A left boundary is not an owner-name boundary.** Only the *unhyphenated* + `Not/x` is excluded; `Not-/x`, an equally real different owner, still + satisfies the boundary and is reported as one of ours. This is a false + positive, not a miss, and it is kept rather than fixed so the boundary class + stays byte-identical to `public-repo-hygiene`'s `REPO_REF_RE` — a name must not + pass one checker and fail the other. Both spellings are pinned by the smoke + tests; widening the class is a change to **both** checkers. +- **Contents only.** The pattern is applied to what a tracked file *contains*, + never to the tracked **path** and never to a **symlink's target string** — so + a name published as a directory component (`docs///placeholder`) or + as a link target is not a finding here. `public-repo-hygiene` does scan a + symlink's target string; neither checker scans the path itself. +- **Two adjacent literals scan as one.** `/public./private` is a single + `grep -o` match whose name reads `public.`; the second name is never + compared and the scan resumes past it. Any separator in the name class does it + — `.`, `-`, `_`, or none at all for `//private`. It fails closed — that token is on no + list, so the line is always a finding — and the remedy that *would* be + dangerous (allowlisting the quoted token) is rejected by the loader, above. +- **The per-hit loop is linear in the hit count**, and the 200-finding print cap + does not bound it: past the cap a hit stops printing but is still normalized + and compared, because the reported count and the exit status have to stay the + true ones. Measured at ~5.4 ms per hit past the cap, so the caller's + `timeout-minutes: 10` is reached at roughly **110,000** hits in one scan. + Documented rather than capped — this tree's largest tracked file is under 600 + lines, and capping the scan would trade an unreachable timeout for a truncated + count, the one number a red run's summary turns on. + +## Running it + +```bash +bash .github/lint/check-org-repo-literals.sh # scan the tracked tree +bash .github/lint/check-org-repo-literals.sh --root DIR +shellcheck -x .github/lint/check-org-repo-literals.sh +``` + +Exit `0` clean, `1` findings, `2` usage/setup error. + +Two bounds keep one hostile tracked line from turning the lint into an +inconclusive run rather than a verdict. A name longer than GitHub's +**100-character** repo-name limit *after* normalization is reported without +being compared (it cannot be a real repo, so it can never be allowlisted — the +bound fails *closed*). The strips run first on purpose: peeling is the only thing +that can bring an over-long name back under the limit, and measuring first turned +`/<98-char-name>.git` into a finding whose stated remedy — the allowlist — is +skipped for exactly that class. Their input is bounded to 8 bytes of headroom +(`.git`, a sentence period, an ellipsis) because the peel is quadratic in the run +it removes: unbounded, a single line of `/` followed by 400 KB of +periods ran for over two minutes. +And past **200** findings the per-hit lines stop printing — the count and the +exit status stay the true ones — so a badly-seeded allowlist floods neither the +public run log nor the annotation list. + +`--root` is resolved robustly rather than trustingly: `CDPATH`, `GIT_DIR`, +`GIT_WORK_TREE` and `GIT_INDEX_FILE` are unset (each can silently move the scan +off the directory the run then reports as its scope), and so are +`GIT_CONFIG_COUNT`, `GIT_CONFIG_GLOBAL`, `GIT_CONFIG_SYSTEM` and +`GIT_CONFIG_PARAMETERS` — those inject arbitrary config, and +`core.attributesFile` pointing at one `*.md binary` line marks a subset of the +tree binary, which `git grep -I` then silently skips. That one fails *open*: the +scannability probe still clears, the scan reads less than the tree, and the OK +line still claims the whole scope. (`GIT_CONFIG_KEY_n`/`_VALUE_n` are inert once +the count is gone. `GIT_CONFIG_PARAMETERS` is the non-obvious one: it is how `-c` +propagates to child git processes and is read unconditionally, so it delivers the +same fail-open with the count already unset.) + +What that does *not* do: unsetting `GIT_CONFIG_GLOBAL`/`GIT_CONFIG_SYSTEM` +restores git's **default** config search rather than disabling it, so a +runner-level `core.attributesFile` still applies. Pointing both at `/dev/null` +plus `GIT_CONFIG_NOSYSTEM=1` would close that and is deliberately not done — it +would also drop a runner's legitimate `safe.directory`, which makes `rev-parse` +fail and silently takes the `grep -r` fallback. The environment is hardened; the +runner's own config is trusted, the same way the tamper boundary trusts the +checkout. + +The git/`grep -r` branch is chosen from `rev-parse --is-inside-work-tree`'s +**output** rather than its exit status (it prints `false` and exits `0` for a +bare repo and for a path under `.git`), and `git grep`'s output-shaping config +(`grep.column`, `grep.fullName`, `color.grep`, `core.quotePath`) is pinned on the +command line. `grep.fullName` is pinned **`true`**, not `false`: with `false` git +prints paths relative to the cwd, which `git -C "$root"` has set to `$root`, so +`--root docs` would report `docs/x.md` as `x.md` and emit an annotation GitHub +resolves against the repo root and drops. diff --git a/.github/lint/check-org-repo-literals.sh b/.github/lint/check-org-repo-literals.sh new file mode 100755 index 0000000..f7d6572 --- /dev/null +++ b/.github/lint/check-org-repo-literals.sh @@ -0,0 +1,809 @@ +#!/usr/bin/env bash +# +# check-org-repo-literals.sh — allowlist lint for org-prefixed repo literals. +# +# This repo is PUBLIC, and AGENTS.md's first convention is "never leak private +# caller names". That rule was convention-enforced only. This makes it +# CI-enforced, the same move the workflow-pins lint made for the +# `workflows_ref` default rule. +# +# ALLOWLIST, NEVER DENYLIST. A denylist grep would have to commit the private +# names into this public repo to match them — the lint would BE the leak. So +# every org-prefixed repo literal in the tracked tree is a finding unless the +# name is on the committed allowlist. Adding a name is an allowlist edit, +# which review sees. +# +# KNOWN LIMITATIONS — read these before trusting a green run: +# 1. Only ORG-PREFIXED literals are caught. A bare repo name written without +# the org prefix cannot be linted without committing a denylist that is +# itself the leak. Bare-name discipline stays with review and AGENTS.md. +# 2. The name class is ASCII and the scan runs in the C locale, so a name +# whose tail is non-ASCII -- a U+2010 homoglyph dash, a U+017F long s, CJK +# text -- is read only as far as its ASCII PREFIX, and a private name +# spelled that way can clear on an allowlisted prefix. The class cannot +# simply be widened to high bytes: this tree legitimately writes an +# ellipsis, an em dash and CJK text flush against a real name in prose, so +# widening turns those into findings. Handling it properly needs the +# offset-aware scan `public-repo-hygiene` does; see limitation 5. +# 3. Both scan paths pass `-I`, so a tracked blob git classifies as BINARY is +# never read: a UTF-16 file, a file carrying a stray NUL, or one +# `.gitattributes` line (`*.md binary`) removes whole file types from the +# scan and a green run says nothing about them. Text is the only surface +# this lint claims. +# 4. Only a literal WRITTEN WHOLE is caught. An org literal assembled at run +# time -- `printf '%s/%s' '' ''`, `"${ORG}/"`, a name +# split across a string concatenation -- never matches PATTERN, so it needs +# no allowlist edit while the run still reports its scope clean. This is +# not hypothetical here: this lint's own smoke tests use exactly that +# spelling on purpose, so it is the house style for org literals in this +# repo's workflow files. It is also fully human-readable in the source, so +# review is what catches it. +# 5. Category 3 only. `.github/public-repo-hygiene/` is the rigorous, +# org-wide implementation of this idea (it also covers ticket ids, +# internal collaboration-tool links and the homoglyph case above). This +# repo cannot adopt that caller as-is: it is that checker's own home, so +# its tests and docs are full of deliberate fake-private fixtures, and it +# references ticket ids by convention throughout. This lint is the subset +# that this repo CAN enforce on itself today. +# 6. A team `@` is judged by POSITION, not by grammar. An `@` glued to the +# tail of an identifier (`user@/`, email-shaped) still reads as +# a package scope, so such a literal may additionally draw on the +# `@`-scoped TEAM entries. Reaching a private name through it needs the +# name spelled exactly like an allowlisted team slug. +# `public-repo-hygiene` reads the `@` from the same position and shares +# this residual. +# 7. `_` counts as identifier-CONTINUATION in the left boundary, so a literal +# written directly after an underscore is not read as a reference at all: +# the markdown-italic spelling `_/_` matches NOTHING. That is +# the same accepted trade `public-repo-hygiene` makes, kept identical on +# purpose so a name cannot pass one checker and fail the other. +# 8. A LEFT boundary is not an owner-name boundary. The org segment must +# start a token, but `-` is not an identifier character and IS legal in a +# GitHub owner name, so a DIFFERENT owner whose name ends in this one -- +# `Not-${ORG}/x` -- satisfies the boundary and is read as one of this +# org's repos, reddening a required lint on a reference that has nothing +# to do with us. Only the unhyphenated `Not${ORG}/x` spelling is excluded. +# This is a FALSE POSITIVE, not a miss, and it is kept rather than fixed +# so the boundary stays byte-identical to `public-repo-hygiene`'s +# `REPO_REF_RE` -- a name must not pass one checker and fail the other. +# Both spellings are pinned by the smoke tests. Widening the class is a +# change to BOTH checkers, not to this one. +# 9. CONTENTS only. PATTERN is applied to what a tracked file CONTAINS, never +# to the tracked PATH itself and never to a symlink's target string +# (neither scan path reads one: `git grep` skips the non-regular worktree +# entry, and `grep -r` does not follow a discovered link). A name +# published as a directory component -- `docs///placeholder` -- +# or as a link target is therefore not a finding here. +# `public-repo-hygiene` does scan a symlink's target string; neither +# checker scans the tracked path itself. +# 10. Two ADJACENT literals scan as ONE. `grep -o` takes non-overlapping +# matches, and the name class holds `.` and `-`, which are also characters +# that can SEPARATE two literals -- so `/public./private` is a +# SINGLE match whose name reads `public.`. The second name is never +# compared, never printed, and the scan resumes at `/private`, which no +# longer matches. It fails CLOSED, because a merged token always ends in +# the org name and nothing on the allowlist does, so such a line is always +# a finding. The residual is the REMEDY, not the miss: the finding quotes a +# token that is not a repo, and the obvious way to silence something so +# confusing is to allowlist the quoted token -- which would clear the +# private reference with the run green. So the loader REJECTS any entry +# ending in the org name. That is the COMPLETE test, not a sample of +# separators: `grep -o` stops the merged token at the `/` that starts the +# second name, so such a token always ends in the org -- `foo.`, +# `foo-`, `foo_` (the name class holds `_`) and the bare `` +# that `//private` produces are all covered by the one suffix. +# 11. The per-hit loop is LINEAR IN THE HIT COUNT, and `MAX_REPORTED` does not +# bound it: past the cap a hit stops printing but is still folded, +# normalized and compared against the list, because the reported count and +# the exit status have to stay the true ones. Measured after the fold above +# was replaced by a glob: ~5.4ms per hit past the cap (2000 hits in one +# tracked file, 11.7s, against 2.1s at 200), so the caller's +# `timeout-minutes: 10` is reached at roughly 110,000 hits in one scan -- +# and `$hits` buffers the whole scan before the loop starts. Documented +# rather than capped: this tree's largest tracked file is under 600 lines, +# and capping the scan would trade an unreachable timeout for a truncated +# count, which is the one number a red run's summary turns on. +# +# TAMPER BOUNDARY: unlike the reusable checkers this repo publishes, this lint +# runs from the PR's own checkout, so a PR here can edit both the script and the +# allowlist. That is the same boundary the workflow-pins lint has and it is the +# point -- widening the allowlist is a reviewed diff. It is NOT a control against +# a hostile committer. +# +# Usage: bash .github/lint/check-org-repo-literals.sh [--root DIR] +# [--allowlist REPO/REL/PATH] +# Exit: 0 clean, 1 findings, 2 usage/setup error. + +set -euo pipefail + +# Byte-wise, deterministic matching everywhere. `grep -i` case-folds according +# to the LOCALE, so in a UTF-8 locale it widens `[A-Za-z]` to reach U+017F and +# U+212A while in the POSIX locale it does not -- and the runner's locale is not +# something this lint should depend on. `LC_ALL=C` pins ASCII-only folding for +# both `grep` and the `tr` below, so a green run here means the same thing it +# means on CI. +export LC_ALL=C + +# Both `cd -- "$root"` calls below take a caller-supplied, possibly RELATIVE +# path. With `CDPATH` exported, `cd` can land in a same-named directory that is +# not the one `[ -d "$root" ]` validated -- and it echoes the resolved absolute +# path on stdout, which lands inside a command substitution and reaches the hit +# loop as a bogus `file:line:match` line. +unset CDPATH + +# `GIT_DIR`, `GIT_WORK_TREE` and `GIT_INDEX_FILE` OVERRIDE `git -C`, so with any +# of them set in the environment the probe and the scan below answer about a +# DIFFERENT repository while the OK line still reports the scope as `$root` +# (measured: `GIT_DIR=b/.git GIT_WORK_TREE=b git -C a grep -l -e ''` lists b's +# files, not a's). Same class of hazard as `CDPATH`, unset in the same place. +unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE + +# The config-INJECTION variables, unset for the same reason but against a +# hazard that fails the other way. `GIT_CONFIG_COUNT` with `GIT_CONFIG_KEY_n`/ +# `GIT_CONFIG_VALUE_n`, and `GIT_CONFIG_GLOBAL`/`GIT_CONFIG_SYSTEM`, can set +# `core.attributesFile` and so mark a subset of the tree `binary` -- which +# `git grep -I` then silently skips. That is limitation 3's hazard delivered +# through the environment rather than through a tracked `.gitattributes`, and +# the `-c grep.*`/`color.grep`/`core.quotePath` pins on the scan cannot cover it +# because the injected key is not one of the pinned ones. +# +# Unlike `GIT_DIR` this one fails OPEN: `first_scannable` still finds a file, so +# the guard clears, the scan reads LESS than the tree, and the OK line still +# claims "the tracked files under '$root'". `GIT_CONFIG_KEY_n`/`_VALUE_n` are +# inert once `GIT_CONFIG_COUNT` is gone (git reads the count first and then +# exactly that many pairs), so they need no unset of their own. +# +# `GIT_CONFIG_PARAMETERS` is the FOURTH and is not obvious: it is the channel +# `-c` uses to propagate to child git processes, and git reads it from the +# environment unconditionally, so `GIT_CONFIG_PARAMETERS="'core.attributesFile=…'"` +# delivers the same fail-open with `GIT_CONFIG_COUNT` already gone (measured). +# +# What this does NOT do: unsetting `GIT_CONFIG_GLOBAL`/`GIT_CONFIG_SYSTEM` +# restores git's DEFAULT search (`$HOME`/XDG, `/etc/gitconfig`) rather than +# disabling config, so a runner-level `core.attributesFile` still applies. +# Pointing both at `/dev/null` plus `GIT_CONFIG_NOSYSTEM=1` would close that, +# and is deliberately not done: it would also drop a runner's legitimate +# `safe.directory`, which makes `rev-parse` fail and silently takes the +# `grep -r` fallback -- trading a hazard nobody can reach from a PR for a +# scope change on every run. The env is hardened; the runner's own config is +# trusted, the same way the TAMPER BOUNDARY block below trusts the checkout. +unset GIT_CONFIG_COUNT GIT_CONFIG_GLOBAL GIT_CONFIG_SYSTEM GIT_CONFIG_PARAMETERS + +ORG='Comfy-Org' +# Deliberately assembled rather than written whole: a literal org-prefixed name +# anywhere in this file would be a finding against this very lint. +# +# Matched case-INSENSITIVELY (`grep -i`), because GitHub resolves owner names +# case-insensitively: a lowercased org spelling reaches the same repository, so +# matching only the canonical spelling would leave a one-keystroke bypass of a +# default-deny control. The name class is already both cases, so `-i` widens +# nothing but the org segment. +# +# The org segment must START A TOKEN. `grep -E` has no lookbehind, so the +# boundary is written as an alternation and the character in front of the org is +# CONSUMED by the match (the loop below drops it again). Without it `grep -o` +# reads straight into the middle of a longer token, so `Not${ORG}/whatever` -- +# a DIFFERENT owner -- is extracted as one of this org's repos and reddens CI on +# a reference that has nothing to do with us. The boundary class is +# `public-repo-hygiene`'s `REPO_REF_RE` lookbehind verbatim, so the two agree on +# what counts as a reference: every real spelling is preceded by a separator +# (`/` in a URL, whitespace, a quote, or the `@` of a team handle), and none of +# those is in the class. The class treats `_` as identifier-continuation, so the +# markdown-italic spelling `_${ORG}/_` is NOT read as a reference -- the +# same accepted trade that checker makes, kept identical on purpose so a name +# cannot pass one checker and fail the other. +# +# The token boundary is NOT an owner-name boundary, and the difference is +# limitation 8: `-` is legal in a GitHub owner name but is not an identifier +# character, so only the UNHYPHENATED `Not${ORG}/x` is excluded here -- +# `Not-${ORG}/x`, an equally real different owner, still satisfies the +# alternation and is read as one of ours. Kept for byte-parity with that +# checker rather than fixed here; widening the class is a change to both. +# +# The `@` of a team handle is that boundary character rather than part of the +# pattern. An `@`-prefixed match is a CODEOWNERS team handle or an npm/GitHub +# Packages scope, and team handles are allowlisted separately from repo names +# (see `@`-entries below); without the split the two share one namespace, so a +# team slug allowlisted for a CODEOWNERS fixture would also clear a literal +# reference to a private REPO of the same name. `public-repo-hygiene` splits +# them the same way and reads the `@` from the same position -- including its +# residual, which this shares: the `@` is judged by POSITION alone, so one glued +# to the tail of an identifier (`user@${ORG}/`, email-shaped) still reads +# as a scope. Reaching a private name through that needs the name to be spelled +# exactly like an allowlisted TEAM slug. +PATTERN="(^|[^A-Za-z0-9_])${ORG}/[A-Za-z0-9_.-]+" + +# Lowercased once for the `case` tests in the hit loop (matching is +# case-insensitive, and bash 3.2 has no `${var,,}`). +org_lc="$(printf '%s' "$ORG" | tr '[:upper:]' '[:lower:]')" + +# The same org name as a case-insensitive GLOB — `Comfy-Org` becomes +# `[Cc][Oo][Mm][Ff][Yy]-[Oo][Rr][Gg]`. Built ONCE here (two forks total) so the +# hit loop can decide the `@`/boundary case with a `case` test instead of +# folding the match's head through `printf | tr`, which cost two forks on every +# hit. Letters become a two-character class; every other byte -- `-` here, which +# is literal outside a bracket expression -- is copied as-is, so the glob +# matches exactly the names the fold did and no others. +org_uc="$(printf '%s' "$ORG" | tr '[:lower:]' '[:upper:]')" +org_glob='' +i=0 +while [ "$i" -lt "${#ORG}" ]; do + lc_char="${org_lc:$i:1}" + uc_char="${org_uc:$i:1}" + if [ "$lc_char" = "$uc_char" ]; then + org_glob="$org_glob$lc_char" + else + org_glob="${org_glob}[$uc_char$lc_char]" + fi + i=$((i + 1)) +done + +root='.' +allowlist_rel='.github/lint/org-repo-allowlist.txt' + +while [ $# -gt 0 ]; do + case "$1" in + --root) + [ $# -ge 2 ] || { echo "error: --root needs a directory" >&2; exit 2; } + root="$2" + shift 2 + ;; + --allowlist) + [ $# -ge 2 ] || { echo "error: --allowlist needs a file" >&2; exit 2; } + allowlist_rel="$2" + shift 2 + ;; + -h|--help) + # The whole leading comment block, minus the shebang, is the help text. + sed -n '2,${/^[^#]/q;p;}' "$0" | sed 's/^# \{0,1\}//' + exit 0 + ;; + *) + echo "error: unknown argument '$1'" >&2 + exit 2 + ;; + esac +done + +[ -d "$root" ] || { echo "error: --root '$root' is not a directory" >&2; exit 2; } + +# The allowlist is resolved against the REPO being scanned when it has one, and +# otherwise against this script's own repo — that is what lets the smoke test +# point --root at a bare fixture directory and still get the real allowlist. +# `|| exit 2`: this is the one setup step that could otherwise exit 1 -- the +# status the header documents as FINDINGS. If the `cd`/`pwd` fails (this +# script's own directory removed or replaced under it, a stale mount, +# `BASH_SOURCE[0]` resolving somewhere unenterable) the substitution returns +# non-zero, the assignment fails, and `set -e` would exit 1 with only bash's +# `cd:` message and no finding lines -- the same hole the `-r` guard below +# closes, on a line that runs ahead of every `-d`/`-f`/`-r` check. +script_repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" || { + echo "error: cannot resolve this script's repo root from '${BASH_SOURCE[0]}'" >&2 + exit 2 +} +# `--root` explicitly supports a directory BELOW a work tree, and for such a +# root `$root/$allowlist_rel` is not where the scanned repo keeps its allowlist +# -- its top level is. Without this step `--root /somewhere/repo/docs` validated +# that repo's literals against THIS repo's list, which is not what the comment +# above promises. `|| true` because a non-repo root is the normal fallback case, +# not an error, and `-d` because `--show-toplevel` prints nothing useful when it +# fails. +root_repo_top='' +if [ "$(git -C "$root" rev-parse --is-inside-work-tree 2>/dev/null)" = true ]; then + root_repo_top="$(git -C "$root" rev-parse --show-toplevel 2>/dev/null || true)" +fi +if [ -f "$root/$allowlist_rel" ]; then + allowlist="$root/$allowlist_rel" +elif [ -n "$root_repo_top" ] && [ -f "$root_repo_top/$allowlist_rel" ]; then + allowlist="$root_repo_top/$allowlist_rel" +elif [ -f "$script_repo_root/$allowlist_rel" ]; then + allowlist="$script_repo_root/$allowlist_rel" +else + echo "error: allowlist '$allowlist_rel' not found under '$root' or '$script_repo_root'" >&2 + exit 2 +fi +# A regular but UNREADABLE allowlist clears the `-f` tests above, and the +# redirection that reads it below would then fail under `set -e` with status 1 -- +# the status documented as FINDINGS, so a caller could not tell an unusable setup +# from a completed scan that found something. Every other setup failure here is +# exit 2; so is this one. +[ -r "$allowlist" ] || { echo "error: allowlist '$allowlist' is not readable" >&2; exit 2; } + +# Three shapes of entry are rejected outright rather than loaded, because each +# of them reads as an ordinary allowlist line while doing something other than +# what its author meant. +# +# A GLOB (`*`, `**`, `?*`, `[a-z]*`, `secret-*`) is the first. Note what this +# guard is and is NOT today: the comparisons below are exact (`[ "$lower" = +# "$known" ]`), so a `*` entry already clears nothing but a literal `*` -- no +# entry reaches the comparison AS A PATTERN any more. The rejection is +# DEFENCE-IN-DEPTH against that comparison going back to a `case` glob, kept so +# that a later reader "simplifying" `=` into one does not silently re-arm every +# metacharacter on the list. It is also why an earlier revision's approach -- +# testing each entry against two fixed nonsense probe names -- was dropped: two +# probes cannot decide breadth in general (`[!qz]*`, `[a-p]*` and `[!q-z]*` +# match NEITHER probe and still clear almost the whole namespace), whereas +# rejecting the metacharacters is deterministic and free. Enumerating a fixture +# family rather than globbing it is already the documented house style. +# +# An entry containing `/` is the second, and unlike the glob it is a live +# foot-gun rather than a latent one. The comparisons below run against the BARE +# name (`${lower#*/}`), so `/foo` can never equal an entry -- yet `/foo` +# is exactly the spelling the tool invites, because the finding line quotes the +# whole literal and the summary says to add it to the allowlist. Accepting it +# would yield an inert line and a rerun still insisting the name is not on a +# file that visibly contains it. Rejected with the spelling that does work. +# +# An entry whose name ENDS in `.` or `-` is the third; see limitation +# 10. `grep -o` takes non-overlapping matches and the name class contains the +# characters that also separate two adjacent literals, so +# `/public-name./private-name` is ONE match whose name reads +# `public-name.` -- and the second name is never compared, never printed, +# and the scan resumes past it. It fails closed today (that merged token is on +# no list), but the obvious way to silence so confusing a finding is to +# allowlist the token it quotes, and THAT would clear the private reference with +# CI green. Making the merged token unallowlistable removes the foot-gun; a real +# repo name in this shape is a namespace collision with the merge artefact and +# has to be spelled some other way regardless. + +# Allowlist entries are LITERAL names compared case-insensitively (GitHub +# resolves owner/repo names case-insensitively, so a case variant of an +# allowlisted PUBLIC name is the same repo — and no private name is on this +# list under any casing, so folding case cannot clear one). +# +# An entry written with a leading `@` is a TEAM/scope entry: it clears only an +# `@`-prefixed literal. A plain entry clears either spelling, because +# `@/` is also how npm and GitHub Packages write a package scope for +# a repo of that name — the same asymmetry `public-repo-hygiene` documents. +entries=() +entry_count=0 +team_entries=() +team_entry_count=0 +# `|| [ -n "$line" ]` so a final entry with no trailing newline is not dropped. +while IFS= read -r line || [ -n "$line" ]; do + line="${line%%#*}" # strip the trailing "why it's safe" comment + line="${line#"${line%%[![:space:]]*}"}" # ltrim + line="${line%"${line##*[![:space:]]}"}" # rtrim + [ -n "$line" ] || continue + entry="$(printf '%s' "$line" | tr '[:upper:]' '[:lower:]')" + is_team=0 + case "$entry" in + @*) is_team=1; entry="${entry#@}" ;; + esac + [ -n "$entry" ] || { echo "error: allowlist '$allowlist' has an entry that is a bare '@'" >&2; exit 2; } + case "$entry" in + *[][*?]*) + echo "error: allowlist '$allowlist' entry '$line' contains a glob metacharacter ([, ], * or ?); entries are literal names, and a glob can allow every name" >&2 + exit 2 + ;; + esac + case "$entry" in + */*) + echo "error: allowlist '$allowlist' entry '$line' contains '/'; entries are bare repo NAMES compared against the name alone, so an entry written as '$ORG/' can never match anything — write just ''" >&2 + exit 2 + ;; + esac + case "$entry" in + *"$org_lc") + echo "error: allowlist '$allowlist' entry '$line' ends in '$ORG'; that is the shape of a MERGED literal (see limitation 10, '/a./b' scans as one match named 'a.') and allowlisting it would clear the second, unexamined name with the run green" >&2 + exit 2 + ;; + esac + if [ "$is_team" -eq 1 ]; then + team_entries+=("$entry") + team_entry_count=$((team_entry_count + 1)) + else + entries+=("$entry") + entry_count=$((entry_count + 1)) + fi +done < "$allowlist" + +# Counted rather than `${#entries[@]}`: bash 3.2 (still /bin/bash on macOS) +# treats an empty array as unset under `set -u`, so the guard would abort with +# `unbound variable` instead of this message. The `for` loops below are only +# reached with a non-empty array for the same reason. +if [ "$((entry_count + team_entry_count))" -eq 0 ]; then + echo "error: allowlist '$allowlist' has no entries" >&2 + exit 2 +fi + +# Tracked files only when there is a work tree (build output and anything +# untracked is out of scope by construction); a plain recursive grep otherwise, +# which is the path the smoke test's fixture directory takes. +hits='' +scan_status=0 +# The OUTPUT is tested, not the exit status: `rev-parse --is-inside-work-tree` +# answers on stdout and exits 0 while printing `false` for a bare repo and for a +# path under `.git`. On the status test those roots took the git branch, where +# `git grep` cannot run, and died with the misleading `no scannable tracked text +# file` instead of using the fallback that exists for exactly them. +if [ "$(git -C "$root" rev-parse --is-inside-work-tree 2>/dev/null)" = true ]; then + # `--is-inside-work-tree` is true for any directory NESTED under a work tree, + # not just its root, and the `.` pathspec resolves relative to `$root`. So a + # `--root` below the top scans only that subtree — and if the subtree holds no + # tracked files, `git grep` matches nothing, exits 1, and that is + # indistinguishable from a clean tree. Fail closed instead, and report the + # scope actually scanned rather than claiming the whole tracked tree. + # + # The probe has to be THE SAME READ the scan makes, not merely a read of the + # same tree. `git ls-files` enumerates the INDEX, which counts blobs the scan + # below skips: a scope whose tracked files are all binary (one `*.md binary` + # attribute line, a UTF-16 subtree, `--root assets/`) or all empty has a + # non-empty index while `git grep -I` reads none of it, so an index probe + # would clear and the scan would then exit 1 -- "no matches" -- and print the + # clean-tree OK line for a tree it never read. `git grep -Il -e ''` lists + # exactly the files the scan can open, which is also what the fallback's + # `grep -rIl -e ''` probe lists, so both paths give a binary-only tree the + # same verdict. `-E` is passed for the same reason the scan passes it: to + # override a `grep.patternType` that would reinterpret the empty pattern. + # + # `|| true` and an emptiness test rather than a status test, because `head` + # closing the pipe early makes `git grep` die of SIGPIPE and `pipefail` would + # read that as a scan error on a perfectly good tree. + scope="the tracked files under '$root'" + first_scannable="$(git -C "$root" grep -IlE -e '' -- . 2>/dev/null | head -n 1 || true)" + if [ -z "$first_scannable" ]; then + echo "error: no scannable tracked text file under '$root' — refusing to report a clean tree" >&2 + exit 2 + fi + # The allowlist file itself is NOT excluded. Its entries are bare names that + # cannot match PATTERN anyway, so an exclusion would buy only one thing: + # letting its free-text rationale comments go unchecked, so that a line like + # `foo # mirrors /` would publish an unapproved name with CI + # green. The non-git path below never excluded it either. + # + # The `-c grep.*` pins are the same move as `LC_ALL=C` above: `git grep` + # honours `grep.column`, `grep.lineNumber` and `grep.fullName` from system or + # global config and from `GIT_CONFIG_*`, and `grep.column=true` alone turns the + # output into `file:line:col:match`, which the right-anchored parse below reads + # as `file:line` + a column-as-line-number. Findings would still fire (the match + # text is authoritative) but every printed location and `::error` annotation + # would point at the wrong place. `-E` is passed explicitly for the same reason + # and overrides `grep.patternType`. + # + # `color.grep` and `core.quotePath` reshape the same output and are pinned + # alongside them: `color.grep=always` injects ANSI bytes into the match text, + # which corrupts the lowercased name so an allowlisted entry stops matching, + # and `core.quotePath=true` (git's DEFAULT) emits a path holding a non-ASCII + # byte quoted and octal-escaped, so the `::error file=` annotation would name + # a path that does not exist -- while the fallback `grep -r` prints it raw. + # + # `grep.fullName=TRUE` is the pinned value, not `false`. Of the two stable + # settings only one keeps a subtree run's locations usable: with `false` git + # prints paths relative to the CWD, which `git -C "$root"` has set to `$root`, + # so `--root docs` reports a hit in `docs/x.md` as `x.md` and emits + # `::error file=x.md` -- an annotation GitHub resolves against the repo root + # and then drops. `true` prints it relative to the repo top, which is the + # frame the annotation is read in. Both settings are equally immune to the + # `file:line:col` parse regression the rest of these pins exist for (neither + # adds a field), so pinning the correct one costs nothing. + hits="$(git -C "$root" -c grep.column=false -c grep.lineNumber=true -c grep.fullName=true -c color.grep=false -c core.quotePath=false grep -oiInE -- "$PATTERN" -- .)" || scan_status=$? +else + # `cd` + a `.` root rather than grepping "$root" and stripping the prefix + # afterwards: the strip would be a `sed` expression built from a path this + # script does not control. + # + # `|| exit 3` rather than `&&`: a failed `cd` would otherwise exit 1, which is + # grep's "no matches" status, and the guard below would wave it through as a + # clean tree. `[ -d "$root" ]` above does not cover it (a directory with no + # execute bit, or a TOCTOU replacement between the two). + scope="'$root'" + # The same "refusing to report a clean tree" guard the git path has, for the + # same reason: `grep` exits 1 both for "no matches" and for a root that holds + # nothing this scan can read, so a stale or mistyped `--root` would otherwise + # print the clean-tree OK line having scanned nothing -- fail-open on exactly + # the path the git branch fails closed on. `-rIl -e ''` enumerates precisely + # what the scan below can read: every file it would open, with the same `-I` + # excluding the binaries it would skip. An empty result therefore means there + # was no scannable file, whether because the tree is empty, because everything + # in it is binary, or because the root could not be entered at all. + # + # `|| true` and an emptiness test rather than a status test: `head` closing the + # pipe early makes `grep` die of SIGPIPE, and under `set -o pipefail` that + # status would read as a scan error on a perfectly good tree. Any real failure + # still lands on empty output here, which is exit 2 either way. + first_scannable="$( + cd -- "$root" 2>/dev/null || exit 0 # unenterable: no scannable file, same verdict + grep -rIl --exclude-dir=.git -e '' . 2>/dev/null | head -n 1 || true + )" + if [ -z "$first_scannable" ]; then + echo "error: no readable text file under '$root' — refusing to report a clean tree" >&2 + exit 2 + fi + hits="$(cd -- "$root" || exit 3; grep -rIoiEn --exclude-dir=.git -- "$PATTERN" .)" || scan_status=$? +fi +# Both tools exit 1 for "no matches" and >1 for a real error. Swallowing the +# latter with `|| true` would report a clean tree because the scan never ran -- +# a guard that fails OPEN is worse than no guard, since the green run reads as +# coverage. +if [ "$scan_status" -gt 1 ]; then + echo "error: the scan failed (exit $scan_status) — refusing to report a clean tree" >&2 + exit 2 +fi + +# Trailing sentence punctuation the name class swallowed, then a `.git` suffix +# (`Foo.git` still references the repo `Foo`), then punctuation again -- +# `/foo.git.` needs both passes, and stripping `.git` first leaves the +# suffix stuck behind the period. +# +# ONLY `.` is stripped. A GitHub slug may not END in a period, so dropping one +# can only narrow the name. `-` and `_` are different: both are LEGAL as a +# slug's last character, so stripping them would WIDEN the match — a literal for +# a private repo named `-` or `_` would normalize onto +# the allowlisted entry and clear this default-deny check. +# +# The peel is quadratic in the length of the run, so it is only safe because +# MAX_NAME below bounds its input first. There is no cheap pure-bash +# alternative: `${value%%"${value##*[!.]}"}`, the obvious two-expansion +# rewrite, is quadratic in bash as well (measured: on a 200 KB run of periods +# both forms run past two minutes). +trim_trailing_dots() { + local value="$1" + while [ -n "$value" ]; do + case "$value" in + *.) value="${value%?}" ;; + *) break ;; + esac + done + printf '%s' "$value" +} + +# GitHub workflow-command escaping. A tracked PATH is untrusted input to the +# runner -- it reaches both output lines below -- and `,`/`:` in it would +# otherwise corrupt the `file=`/`line=` properties so the annotation points +# nowhere, while `%`/CR would corrupt the message. `%` must be replaced FIRST or +# it re-escapes the escapes. +escape_data() { + local value="$1" + value="${value//'%'/%25}" + value="${value//$'\r'/%0D}" + value="${value//$'\n'/%0A}" + printf '%s' "$value" +} +escape_property() { + local value + value="$(escape_data "$1")" + value="${value//':'/%3A}" + value="${value//','/%2C}" + printf '%s' "$value" +} + +# A tracked tree can hold an unbounded number of unapproved literals, and every +# one of them prints twice. Past this many the count keeps rising (the summary +# below still reports the true total, and the exit status is still 1) but the +# per-finding lines stop, so a badly-seeded allowlist floods neither the public +# run log nor the annotation list. `public-repo-hygiene` caps the same two ways +# (`MAX_FINDINGS_PER_FILE`/`MAX_FINDINGS_TOTAL`). +MAX_REPORTED=200 + +# GitHub's own repo-name limit; see the cap in the loop below. +MAX_NAME=100 + +# How many bytes NORMALIZATION is allowed to peel off a name before the length +# test gives up on it. Normalization only ever removes a SUFFIX -- trailing +# periods, then at most one `.git`, then trailing periods again -- so a name +# that is over the limit AS WRITTEN can still normalize onto an allowlisted +# one, and testing the length first is what forced `/<98-char>.git`, or a +# 100-character name followed by a sentence period, into `oversize`. That +# matters because `oversize` skips BOTH allowlist comparisons, so the finding +# named a remedy that could not silence it. +# +# The test therefore runs after the strips, and this bounds their input: 8 bytes +# is `.git` plus a sentence period plus an ellipsis, which covers every way a +# real reference gets punctuated in prose. A longer run of periods is still +# `oversize` -- fail-closed, and it is what keeps `trim_trailing_dots`, whose +# peel is quadratic in the run it removes, off unbounded input. +MAX_STRIP=8 + +findings=0 +while IFS= read -r hit; do + [ -n "$hit" ] || continue + # `file:line:match` — the line number holds no colon, so both come off the + # RIGHT. Cutting the file off the LEFT instead would mangle any path that + # contains a colon. + # + # `${hit%:*}` and then an OFFSET, rather than the equivalent `${hit##*:}`: + # `##` re-matches `*:` against a progressively shorter prefix, so it is + # QUADRATIC in the length of the match text it removes, and the name class is + # unbounded. Measured on one tracked line of `${ORG}/foo` + 400 KB of name + # characters: 62 seconds here, against 65 for the whole run. `%` and + # `${var:offset}` are both linear, and split at exactly the same colon. + where="${hit%:*}" + match="${hit:$((${#where} + 1))}" + # The boundary character PATTERN consumes may ITSELF be a colon: `uses:`, + # `repository:` and prose like `see:${ORG}/x` are all real spellings, and each + # puts a fourth colon on the line, shifting the right-anchored split by one + # field. A line number is never empty, so an empty trailing field is that case + # and nothing else. + if [ -z "${where##*:}" ]; then + match=":$match" + where="${where%:}" + fi + file="${where%:*}" + file="${file#./}" + lineno="${where##*:}" + + # A line number is always DIGITS, so anything else means this record is not a + # `file:line:match` triple and the split above produced three fabrications. + # The way that happens is a tracked path containing a NEWLINE on the `grep -r` + # FALLBACK path, which prints the path raw: one hit arrives as two records and + # the leading fragment yields a file that does not exist, a `line=` that is a + # path, and a "literal" cut out of the filename. Reporting it would also put + # the raw `:`/`,` of a path into the `file=`/`line=` properties that + # `escape_property` exists to protect. + # + # The GIT path does not have the split: `git grep` C-quotes newline, tab, `\` + # and `"` in a path whatever `core.quotePath` says (that setting governs only + # bytes >= 0x80), so such a file arrives as ONE record with a numeric line + # field and a C-quoted pseudo-path. That is a real finding with a quoted + # location, not a fabrication, and it is left as such -- the annotation's + # delimiters (`:` and `,`) are still escaped. + # + # Refused rather than escaped or skipped: escaping would still publish the + # fabricated finding, and skipping would drop a record on a parse this script + # has already lost confidence in. Exit 2 is the same "refusing to report" + # verdict every other unusable-scan branch here returns. + case "$lineno" in + ''|*[!0-9]*) + echo "error: could not parse the scan output — expected 'file:line:match', got a non-numeric line field. A tracked path containing a newline splits one record in two; rename it, or narrow --root past it." >&2 + exit 2 + ;; + esac + + # `grep -E` has no lookbehind, so a match that did not start at column 0 + # carries the boundary character in front of the literal. Drop it -- and read + # the team/scope namespace off it first, because an `@` in that position is + # the `@` of a CODEOWNERS handle or a package scope, which may additionally + # draw on the team entries. The `@` stays on `$match` so the finding quotes + # the literal as written and "add it to the allowlist" stays a copy-paste. + # + # Decided from a BOUNDED head slice (`@` + org + `/` is all it takes), not + # from the whole match folded to lower case: the match text is unbounded, and + # this runs on every hit including the oversize ones the block below skips. + # + # The case-insensitive comparison is a GLOB built once from `$ORG` + # (`[Cc][Oo]...`), not a `printf | tr` fold. The fold cost two forks on EVERY + # hit -- including every hit past `MAX_REPORTED`, which prints nothing and so + # buys only the count -- and it was the larger half of the per-hit cost. + # Measured on a fixture of 2000 unapproved literals in one tracked file: + # 18.4s before, 11.7s after (8.6ms -> 5.4ms per hit past the printing cap). + # The glob is exact (each letter becomes a + # two-character class, every other byte is copied literally), so it decides + # the same three cases the fold did. + match_head="${match:0:$((${#ORG} + 2))}" + is_at=0 + case "$match_head" in + $org_glob/*) ;; # column 0 — no boundary was consumed + "@"$org_glob/*) is_at=1 ;; + *) + match="${match#?}" + # A numeric line field does NOT prove the record parsed, so the shape is + # checked here too: after the boundary drop what remains must still begin + # with the org prefix, because that is all PATTERN can match. A tracked + # path named `a:1:foox.md` splits into the fragment `./a:1:foo`, which + # passes the digit test above (file=`a`, line=`1`) and would otherwise be + # printed as the finding `oo` against a path that does not exist. + # shellcheck disable=SC2254 # $org_glob is a GLOB by construction — its + # `[Cc]` classes are the case-insensitive match, so quoting it would break + # the test. (The two patterns above end in `*`, which is why ShellCheck + # reads those as intentional and only flags this one.) + case "${match:0:$((${#ORG} + 1))}" in + $org_glob/) ;; + *) + echo "error: could not parse the scan output — a record's match text does not begin with the org prefix, so 'file:line:match' did not split where it appears to. A tracked path containing a newline splits one record in two; rename it, or narrow --root past it." >&2 + exit 2 + ;; + esac + ;; + esac + + # COST bound, applied before anything folds or forks on the WHOLE match text. + # A match longer than `org + '/' + MAX_NAME` (plus the `@` a scope keeps) + # cannot spell a name of MAX_NAME characters or fewer, so the exact + # `${#name}` test below would reach the same verdict; this only skips the work + # of getting there. Deliberately coarse — the exact rule stays in one place. + lower='' + oversize=0 + if [ "${#match}" -gt "$((MAX_NAME + MAX_STRIP + ${#ORG} + 2))" ]; then + oversize=1 + fi + + if [ "$oversize" -eq 0 ]; then + lower="$(printf '%s' "$match" | tr '[:upper:]' '[:lower:]')" + name="${lower#*/}" + + # A GitHub repo slug is at most MAX_NAME characters, so a name longer than + # that AFTER normalization is not a reference to a real repository under any + # normalization and therefore cannot be on the allowlist — it is reported + # without being compared at all. The `MAX_STRIP` headroom is what lets the + # strips run first: peeling is the only thing that can bring an over-long + # name back under the limit, and skipping it turned `/<98-char>.git` + # into an `oversize` finding whose stated remedy — the allowlist — is + # skipped for exactly that class. Past the headroom the peel is refused + # rather than run, because `trim_trailing_dots`'s `${value%?}` is quadratic + # in the run it removes: unbounded, one tracked line of `${ORG}/foo` + # followed by a long run of periods ran for over two minutes, which would + # burn the caller's whole `timeout-minutes` and turn a would-be finding into + # an inconclusive run. + # + # Both bounds fail CLOSED — an over-long name is always reported, never + # cleared — so neither can let a name through. For scale: the longest + # literal really in this tree is 29 characters. + if [ "${#name}" -gt "$((MAX_NAME + MAX_STRIP))" ]; then + oversize=1 + else + lower="$(trim_trailing_dots "$name")" + + lower="${lower%.git}" + lower="$(trim_trailing_dots "$lower")" + # A reference left naming nothing (`/.`, `/.git`) is not a + # finding. + [ -n "$lower" ] || continue + # The exact test, now that the strips have run and there is a real name to + # measure. Only a name that is STILL over the limit after normalization is + # unallowlistable. + if [ "${#lower}" -gt "$MAX_NAME" ]; then + oversize=1 + fi + fi + fi + + allowed=0 + if [ "$oversize" -eq 0 ] && [ "$entry_count" -gt 0 ]; then + for known in "${entries[@]}"; do + if [ "$lower" = "$known" ]; then allowed=1; break; fi + done + fi + if [ "$allowed" -eq 0 ] && [ "$oversize" -eq 0 ] && [ "$is_at" -eq 1 ] && [ "$team_entry_count" -gt 0 ]; then + for known in "${team_entries[@]}"; do + if [ "$lower" = "$known" ]; then allowed=1; break; fi + done + fi + + if [ "$allowed" -eq 0 ]; then + findings=$((findings + 1)) + # Bound the literal before it is printed. The name class is unbounded, so + # one tracked line of `/` followed by a long run of name characters is + # a SINGLE match whose text is most of the file -- printed once to stdout + # and once more as an `::error` annotation. `public-repo-hygiene` bounds the + # same token for the same reason (`_bounded`, 200 chars); the cut is + # display-only, so what is compared against the allowlist is still the whole + # name and a truncated finding can never be a cleared one. + if [ "$findings" -le "$MAX_REPORTED" ]; then + shown="$match" + if [ "${#shown}" -gt 200 ]; then + shown="${shown:0:200}... (truncated)" + fi + # `unapproved:` is a CONSTANT prefix, and it is load-bearing: without it + # this line starts with the tracked path, so a tracked file named + # `::stop-commands::x.md` would emit a workflow COMMAND at column zero + # that suppresses every `::error` annotation after it. A path cannot + # reach column zero here. + if [ "$oversize" -eq 1 ]; then + why="is longer than GitHub's ${MAX_NAME}-character repo-name limit, so it cannot be allowlisted" + else + why="is not on $allowlist_rel" + fi + echo "unapproved: $file:$lineno: $shown $why" + echo "::error file=$(escape_property "$file"),line=$lineno::unapproved ${ORG} repo literal: $(escape_data "$shown") $(escape_data "$why")" + fi + fi +done <<< "$hits" + +if [ "$findings" -gt "$MAX_REPORTED" ]; then + echo "note: $((findings - MAX_REPORTED)) further finding(s) were counted but not printed (cap: $MAX_REPORTED)." >&2 +fi + +if [ "$findings" -gt 0 ]; then + cat >&2 <-` clear on ``. +# +# An entry written with a leading `@` is a TEAM/scope entry: it clears only an +# `@`-prefixed literal (a CODEOWNERS team handle, or an npm / GitHub Packages +# scope). A plain entry clears either spelling. Keeping team slugs out of the +# repo namespace is why a CODEOWNERS fixture's team name does not also clear a +# literal reference to a private REPO of that name. +# +# THREE SHAPES ARE REFUSED as configuration errors (exit 2). Each reads as an +# ordinary allowlist edit while doing something other than what its author meant. +# +# NO GLOBS — an entry containing `*`, `?`, `[` or `]`. Note what this is +# today: entries are compared with EXACT EQUALITY, so a `[a-z]*` entry clears +# nothing but a literal `[a-z]*`; no entry reaches the comparison as a +# pattern. The rejection is DEFENCE-IN-DEPTH should that comparison ever go +# back to a `case` glob — do not read it as evidence that entries already +# match as patterns, and do not "simplify" the `=` into one. (It is also why +# no fixed set of probe names is used instead: no two probes can decide +# breadth in general.) Every name here is spelled out; a family of test +# fixtures is enumerated, not globbed. +# +# NO `/` — entries are bare repo NAMES, compared against the name alone, so +# `/foo` can never match anything. That is the spelling the tool invites, +# since a finding quotes the whole literal; write just `foo`. +# +# NO NAME ENDING IN THE ORG NAME — that is the shape of a MERGED literal, not +# of a repo. `/public./private` scans as ONE match named +# `public.`, and allowlisting the token such a finding quotes would clear +# the second, never-examined name with the run green. +# +# Adding a line here is the ratification step: it is what makes "this name is +# safe to publish" a reviewed decision instead of an assumption. Never add a +# name you have not confirmed is publishable — for a real repo that means +# confirming it is PUBLIC. +# +# The org-wide known-public repo list lives in +# .github/public-repo-hygiene/check_public_repo_hygiene.py and is the source of +# truth for other repos; this file is repo-local because this repo additionally +# carries that checker's own fake-private test fixtures. +# +# This file is scanned like every other tracked file — the comments above and +# below are NOT exempt, so a rationale that quotes ANOTHER, unlisted name is a +# finding. The one place the scan structurally cannot look is an entry's own +# line, because the entry clears it: state the DECISION there ("published by +# decision — see "), never a confidentiality attribute of the name, since +# "X is private" is itself the private detail AGENTS.md keeps out of this tree. + +# ── Real repos in this org, verified public ── +github-workflows # this repo, public +.github # org meta repo, public +comfy-cli # public (also on the org-wide PUBLIC_COMFY_ORG_REPOS list) +comfy-desktop # public (org-wide list: Comfy-Desktop) +comfy-typescript-sdk # public (org-wide list) +comfyui # public (org-wide list: ComfyUI) +comfyui_frontend # public (org-wide list: ComfyUI_frontend) + +# ── Documented exceptions ── +unreviewed-merges # published by decision as the SOC-2 tracker destination — see README and docs/callers/detect-unreviewed-merge.md + +# ── CODEOWNERS team handles / package scopes (`@`-prefixed literals only) ── +# Team handles are inherently public on a public repo: GitHub renders CODEOWNERS +# owners to anyone who can see the repo. Mirrors PUBLIC_COMFY_ORG_TEAMS. These +# are `@`-scoped so none of them also clears a bare repo reference. +@core-engine-team # team handle, org-wide PUBLIC_COMFY_ORG_TEAMS +@backend # team handle in the agents-md-integrity suite's CODEOWNERS fixture — a team slug, not a repo +@not-a-public-team # fixture team handle in the public-repo-hygiene suite +@core-engine # ASCII prefix of the public-repo-hygiene suite's `core-engineteam` fixture handle (see limitation 2) + +# ── Team handles that ALSO appear bare, as fixtures ── +# The public-repo-hygiene suite writes these without the `@` on purpose, to +# prove the bare path is checked against the repo LIST. The bare spelling is the +# one that has to clear, so unlike the block above these two stay in the REPO +# namespace — a KNOWN CROSSING of the `@`-split, named here so that split does +# not read as complete. Residual: a private repo named exactly `comfy-cloud-team` +# or `some-team` would clear on these lines. Neither name is a repository in +# this org. +# +# They are safe to publish for different reasons, so they are not "both +# invented": `comfy-cloud-team` is a REAL org team, already on the org-wide +# PUBLIC_COMFY_ORG_TEAMS list (a team handle is inherently public on a public +# repo — GitHub renders CODEOWNERS owners to anyone who can see it), while +# `some-team` is an invented fixture slug. +comfy-cloud-team # REAL org team, on the org-wide PUBLIC_COMFY_ORG_TEAMS list; also written bare in a public-repo-hygiene fixture +some-team # INVENTED fixture team handle, also written bare in a public-repo-hygiene HF-route fixture + +# ── Documentation examples — no such repo exists ── +github-workflows-tools # doc example in the bump-callers docs +github-workflows-actions # doc example in the bump-callers docs +example # generic doc/test placeholder +foo # generic doc/test placeholder +legit # bump-callers doc/test placeholder +x # single-letter placeholder in the public-repo-hygiene docs +notes # path-shaped example (`docs/v1.2//notes`) in the public-repo-hygiene docs +ComfyUI.md # doc example: extension over-flag limitation (public-repo-hygiene README/docs/tests) +ComfyUI.internal # doc example: extension over-flag limitation (public-repo-hygiene README/docs/tests) + +# ── Hugging Face model/dataset namespaces, not GitHub repos ── +# This org publishes weights under a Hugging Face namespace of the same name, so +# these names have no GitHub existence at all. They are product content, not +# references. +# +# Allowlist membership is GLOBAL, not per-host: these lines clear a +# `github.com//` reference just as they clear the model-host one that +# justifies them. A KNOWN CROSSING, named here the way the team block above +# names its own, so the host-specific rationale does not read as a host-specific +# exception. Residual: if a private GitHub repo ever took one of these names, +# the reference would ship without its own ratification step. None of them is a +# repository in this org today. The doc-example and fixture blocks below carry +# the same crossing for the same reason. +qwen-image-edit_comfyui # public HF model, quoted in the public-repo-hygiene docs/tests +qwen-image_comfyui # public HF model, quoted in the public-repo-hygiene tests +ace_step_1.5_comfyui_files # public HF model, quoted in the public-repo-hygiene tests +stable-diffusion-v1-5-archive # public HF model, quoted in the public-repo-hygiene tests +a-model # HF-route fixture in the public-repo-hygiene tests +a-dataset # HF-route fixture in the public-repo-hygiene tests +some-model # HF-route fixture in the public-repo-hygiene tests +some-demo # HF-route fixture in the public-repo-hygiene tests +some-eval-set # HF-route fixture in the public-repo-hygiene tests +some-set-abc123 # HF-route fixture in the public-repo-hygiene tests + +# ── Test fixtures — invented names, no such repo ── +# These are the fake "private" names the bump-callers and public-repo-hygiene +# suites use to prove masking and default-deny. Enumerated rather than globbed +# — and a `secret-*` line is now REJECTED outright, because pre-approving an +# unbounded family on a default-deny list would let a real private repo whose +# name began `secret-` ship without tripping the lint. A NEW fixture name needs +# a line here, on purpose: that edit is where a reviewer confirms it is +# invented. +comfyui2 # public-repo-hygiene fixture: invented near-miss of a real public name +a.b.c # bump-callers fixture (dotted name shape) +not-a-public-repo # public-repo-hygiene fixture +private # public-repo-hygiene fixture +private-thing # public-repo-hygiene fixture +super-secret-repo # public-repo-hygiene fixture +some-internal-thing # public-repo-hygiene fixture +something-else # public-repo-hygiene fixture +definitely-not-public # public-repo-hygiene fixture +nope # public-repo-hygiene fixture +nope-private # public-repo-hygiene fixture +whatever # public-repo-hygiene fixture +some-private-repo # public-repo-hygiene fixture +other-private-repo # public-repo-hygiene fixture +github-workflows-ops # public-repo-hygiene fixture +bare-host-at-line-start # public-repo-hygiene model-host anchor fixture +host-case-and-port # public-repo-hygiene model-host anchor fixture +in-a-markdown-link # public-repo-hygiene model-host anchor fixture +in-angle-brackets # public-repo-hygiene model-host anchor fixture +in-double-quotes # public-repo-hygiene model-host anchor fixture +plain-http # public-repo-hygiene model-host anchor fixture +protocol-relative # public-repo-hygiene model-host anchor fixture +title-case-route # public-repo-hygiene model-host route fixture +upper-case-route # public-repo-hygiene model-host route fixture +secret-a # fixture: bump-callers / public-repo-hygiene +secret-alpha # fixture: bump-callers / public-repo-hygiene +secret-anchor # fixture: bump-callers / public-repo-hygiene +secret-b # fixture: bump-callers / public-repo-hygiene +secret-bad # fixture: bump-callers / public-repo-hygiene +secret-beta # fixture: bump-callers / public-repo-hygiene +secret-both # fixture: bump-callers / public-repo-hygiene +secret-branch # fixture: bump-callers / public-repo-hygiene +secret-c # fixture: bump-callers / public-repo-hygiene +secret-case # fixture: bump-callers / public-repo-hygiene +secret-coloc # fixture: bump-callers / public-repo-hygiene +secret-conv # fixture: bump-callers / public-repo-hygiene +secret-converged # fixture: bump-callers / public-repo-hygiene +secret-converted # fixture: bump-callers / public-repo-hygiene +secret-deriskonly # fixture: bump-callers / public-repo-hygiene +secret-dup # fixture: bump-callers / public-repo-hygiene +secret-dupnopin # fixture: bump-callers / public-repo-hygiene +secret-emptyref # fixture: bump-callers / public-repo-hygiene +secret-expr # fixture: bump-callers / public-repo-hygiene +secret-floating # fixture: bump-callers / public-repo-hygiene +secret-flow # fixture: bump-callers / public-repo-hygiene +secret-gamma # fixture: bump-callers / public-repo-hygiene +secret-good # fixture: bump-callers / public-repo-hygiene +secret-groomed # fixture: bump-callers / public-repo-hygiene +secret-half # fixture: bump-callers / public-repo-hygiene +secret-hashref # fixture: bump-callers / public-repo-hygiene +secret-lbl # fixture: bump-callers / public-repo-hygiene +secret-legacy # fixture: bump-callers / public-repo-hygiene +secret-longkey # fixture: bump-callers / public-repo-hygiene +secret-lowercase # fixture: bump-callers / public-repo-hygiene +secret-mention # fixture: bump-callers / public-repo-hygiene +secret-mono # fixture: bump-callers / public-repo-hygiene +secret-multi # fixture: bump-callers / public-repo-hygiene +secret-nopin # fixture: bump-callers / public-repo-hygiene +secret-nopin-two # fixture: bump-callers / public-repo-hygiene +secret-noscript # fixture: bump-callers / public-repo-hygiene +secret-notuses # fixture: bump-callers / public-repo-hygiene +secret-ok # fixture: bump-callers / public-repo-hygiene +secret-one # fixture: bump-callers / public-repo-hygiene +secret-pinned # fixture: bump-callers / public-repo-hygiene +secret-pkg # fixture: bump-callers / public-repo-hygiene +secret-prose # fixture: bump-callers / public-repo-hygiene +secret-quoted # fixture: bump-callers / public-repo-hygiene +secret-riskonly # fixture: bump-callers / public-repo-hygiene +secret-shortref # fixture: bump-callers / public-repo-hygiene +secret-sibling # fixture: bump-callers / public-repo-hygiene +secret-sibref # fixture: bump-callers / public-repo-hygiene +secret-squad # fixture: bump-callers / public-repo-hygiene +secret-tagref # fixture: bump-callers / public-repo-hygiene +secret-three # fixture: bump-callers / public-repo-hygiene +secret-two # fixture: bump-callers / public-repo-hygiene +secret-unwired # fixture: bump-callers / public-repo-hygiene +secret-wdup # fixture: bump-callers / public-repo-hygiene +secret-wired # fixture: bump-callers / public-repo-hygiene +secret-wref # fixture: bump-callers / public-repo-hygiene +secret-wrongfleet # fixture: bump-callers / public-repo-hygiene +secret_a.b-c # fixture: bump-callers / public-repo-hygiene + +# ── ASCII prefixes of the public-repo-hygiene suite's homoglyph fixtures ── +# That suite deliberately commits names whose tail is non-ASCII, to prove its +# own homoglyph handling. This lint's name class is ASCII (limitation 2 in +# check-org-repo-literals.sh), so it reads only the prefix. Each of these is +# the prefix of an INVENTED fixture name, not of a real repo. +comfy-type # prefix of the `comfy-typecript-sdk` fixture +mod # prefix of the `model` HF-route fixture diff --git a/.github/workflows/test-org-repo-literals.yml b/.github/workflows/test-org-repo-literals.yml new file mode 100644 index 0000000..a074297 --- /dev/null +++ b/.github/workflows/test-org-repo-literals.yml @@ -0,0 +1,845 @@ +name: Test org repo literals + +# Regression lint (BE-8192): every org-prefixed repo literal in this public +# repo's tracked tree must be on `.github/lint/org-repo-allowlist.txt`. +# +# AGENTS.md's first convention is "never leak private caller names", and it was +# convention-enforced only. This is the same move test-workflow-pins.yml made +# for the `workflows_ref` default rule: take a rule a reviewer had to remember +# and make CI remember it instead. +# +# ALLOWLIST, NEVER DENYLIST — a denylist grep would have to commit the private +# names into this public repo to match them, so the lint would BE the leak. +# Adding a name is an allowlist edit, which review sees. +# +# NO `paths:` FILTER, deliberately. The leak surface is the whole tree (past +# leaks sat in `scripts/`, `.github/dependabot.yml` and a component README), so +# gating this on workflow-file paths would exempt exactly the files most likely +# to carry one. + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +# One lint run per ref: a push that supersedes an in-flight run should cancel +# it, not race it for a runner. +concurrency: + group: test-org-repo-literals-${{ github.ref }} + cancel-in-progress: true + +jobs: + test: + name: shellcheck + lint this repo + runs-on: ubuntu-latest + # Matches the sibling lint-test workflows. The lint's per-hit loop is linear + # in the hit count and forks one `printf | tr` per match (the head-slice fold + # is now a glob), so a pathological tree should fail in minutes rather than + # spin to GitHub's 360-minute default. Measured at ~5.4ms per hit past the + # 200-finding print cap, this budget is reached at roughly 110,000 hits in a + # single scan — see KNOWN LIMITATIONS 11 in the script. + timeout-minutes: 10 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: ShellCheck the lint script + run: shellcheck -x .github/lint/check-org-repo-literals.sh + + - name: Lint this repo's org repo literals + run: bash .github/lint/check-org-repo-literals.sh + + # Every fixture below assembles the literal with `printf` instead of + # writing it out. A literal org-prefixed name in this file would be + # scanned by the step above and would have to be allowlisted — which + # would defeat the fixture, since the point is that it is NOT allowlisted. + # (`%s/%s` is not a match: the name class cannot start with `%`.) + # + # Each negative case asserts the EXACT status. "any non-zero" would let a + # lint that died with exit 2 (renamed allowlist, unreadable root, an + # argument rejected after a refactor) read as "correctly rejected the + # name" while nothing was scanned. Output is captured and echoed only on + # failure, because a finding prints a `::error file=…` workflow command + # and a SUCCESSFUL run must not publish annotations pointing at a temp + # file that is not in the repo. + - name: Smoke test — findings exit 1, allowlisted names exit 0 + run: | + fail=0 + expect() { # expect