Skip to content

feat: org standards — drift report, write-free dry-run plan, hostname-scan check - #77

Merged
avrabe merged 5 commits into
mainfrom
feat/standards-dry-run
Oct 30, 2026
Merged

avrabe merged 5 commits into
mainfrom
feat/standards-dry-run

Conversation

@avrabe

@avrabe avrabe commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What

Org standards as one config block, one drift function and one apply path, plus a dry-run planner that runs the real apply code without sending a single write. Also adds the hostname-scan action that temper will distribute as a required check.

Merging this changes nothing in the org by itself. A push to main only triggers self-update. The new behaviour takes effect on the next explicit /sync-all-repos / cli.js sync, or on repository.created. Run cli.js plan first.

The standards (config.yml → standards:)

# Rule Enforced as
1 never merge around red Required checks must be non-empty and bind admins. The contexts a repo already requires are carried into temper's managed ruleset, which has no bypass. Temper never invents check names: a repo with none is reported for the owner, or named in standards.repos.<repo>.required_checks (temper and .github use their verified PR check runs; #76). No bypass: enforce_admins is set through the narrow sub-resource endpoint, and bypass actors are stripped from repo rulesets that apply to the default branch. Org-sourced rulesets are reported, not touched. Rulesets that match no branch are flagged as inert.
2 signed commits required_signatures in the managed ruleset, plus classic protection where it exists. Config validation rejects allow_rebase_merge: true alongside required signatures.
3 update-branch strips signatures No repo setting can block the update-branch API call. So: allow_update_branch: false (removes the standing nudge), and strict up-to-date is switched off only where a merge queue is live. The queue re-tests every entry against the latest main. Switching strict off before the queue exists would merge PRs never tested against main. Required signatures turn a rebase-style update into a blocked PR, never an unsigned commit on main.
4 no private hostnames .github/actions/hostname-scan (composite action, stdlib Python ≥ 3.9). templates/workflows/hostname-scan.yml is the stub temper opens as a PR in each repo. The context hostname-scan becomes required only where the stub already exists, so no repo becomes unmergeable. Distribution stays blocked until required_workflows[].pin is set to a merged temper sha.
5 tag/publish only from green main Two tag rulesets on refs/tags/v* (#63), the shape ordeal runs today. One makes release tags immutable for everyone (no bypass). The other restricts creation, and its bypass list (repo admins) is the allowlist of who may tag. A repo with an equivalent ruleset is left alone. Rulesets limit who can tag. Checking that the tagged commit is green on main stays the release workflow's job.

Also: merge methods and allow_auto_merge are evaluated against config (not a hardcoded rebase-only ideal). Merge-queue presence is audited but never created; that is a per-repo rollout. Private repos on GitHub Free are reported as unenforceable instead of failing the whole repo. Forks keep their pre-standards behaviour.

Fixes to the existing path

  • rulesetsMatch ignored bypass_actors and target, so an admin bypass on the managed ruleset was invisible drift.
  • The legacy fallback PUT .../protection replaced classic protection wholesale with contexts: [], wiping a repo's own required checks. It now keeps them.
  • rulesetsMatch compared the parameters GitHub fills in on its own (allowed_merge_methods, required_reviewers, ...), so every sync re-PUT unchanged rulesets. It now compares only the parameters temper sets.
  • sync and plan share one target selection (selectSyncTargets).

Dry-run

GITHUB_TOKEN=$(gh auth token) node scripts/cli.js plan [--repo a,b] [--cache DIR] [--json FILE] [--fail-on-drift]
node scripts/cli.js audit --snapshots DIR      # offline, from a saved sweep

plan runs configureRepository against src/dry-run.js. GETs pass through, throttled to ≤1 req/s with secondary-rate-limit backoff, and can be cached. Every other method is recorded and never sent. The recorder exposes only request/paginate, so a write cannot leave through octokit.repos.*. The plan therefore is what apply would send, labels and dependabot side effects included.

Tests

  • standards.test.js
    • Positive control: a compliant repo shows no drift.
    • Negative controls, each restored by re-editing:
      • enforce_admins off
      • an admin bypass actor
      • signatures off
      • config floor violations: validateConfig errors
    • Shapes seen in the org: inert ruleset, queue strictness, partial tag protection, a free-plan 403 vs a secondary-rate-limit 403, forks.
  • dry-run.test.js: writes are recorded and the real octokit is never called for them; no side-door surface; cache, throttle, retry.
  • plan.test.js: the planner over a fake read-only GitHub that throws on any write.
  • test_hostname_scan.py:
    • Detections.
    • Negative controls: TEST-NET documentation addresses, versions, config.local.yml, self.local, loopback.
    • Redaction, allowlist, pragma, denylist, history mode.
    • Fixtures are assembled at run time, so this repo's own diff scans clean.
  • CI: a new hostname-scan job runs the self-test, vermin -t=3.9-, and the action on this PR's own diff.
  • npm test 959/959, bazel test //... 2/2, eslint (now including scripts/), shellcheck on the action script, actionlint.

Follow-ups (not in this PR)

  1. After merge: set standards.required_workflows[0].pin to the merge sha, so stub distribution unblocks.
  2. Create the HOSTNAME_SCAN_DENYLIST org secret (real names stay out of every repo).
  3. Owner reviews the cli.js plan output, then approves a sync.

Refs #63, #65, #66, #76

🤖 Generated with Claude Code

https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R

avrabe and others added 3 commits October 30, 2026 02:05
A composite action (pinned by sha from each repo's stub workflow) that
scans the lines a PR / merge-queue entry / push adds for private-network
shapes: mDNS/LAN/tailnet/router names (*.local, *.lan, *.internal,
*.home.arpa, *.ts.net, ...), RFC 1918 / CGNAT / link-local IPv4, and
ULA / link-local IPv6. Real home-network names are never committed: they
come from the HOSTNAME_SCAN_DENYLIST org secret at run time. Findings are
redacted in the log (CI logs of a public repo are public). Per-repo
exceptions live in .github/hostname-scan-allow.txt (path:/value:) or an
inline `hostname-scan: allow` pragma. --history reports counts only, for
a make-public review.

stdlib only, Python 3.9 floor (vermin in CI; tests run under 3.9 locally).
templates/workflows/hostname-scan.yml is the stub temper distributes; its
job name `hostname-scan` is the required context. temper's own CI runs the
self-test and the action on its own diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
config.yml gains `standards:` — one definition of the floor temper
enforces (owner decision 2026-10-30), evaluated by one drift function
(src/standards.js) and applied by the same code the dry-run exercises:

- required checks non-empty and binding admins: contexts a repo already
  requires are carried into temper's managed ruleset (no bypass); temper
  never invents names — repos with none are reported for the owner, or
  set standards.repos.<repo>.required_checks (temper and .github: their
  verified PR check runs; temper#76)
- no bypass: enforce_admins on classic protection via the narrow
  endpoint; bypass actors stripped from repo rulesets on the default
  branch (org rulesets reported, not touched); inert rulesets matching
  no branch are flagged
- signed commits; rebase merges rejected in config alongside them
- rule 3: allow_update_branch off; strict up-to-date off only where a
  merge queue is live (switching it off earlier would merge PRs never
  tested against main)
- release tags (temper#63): an immutable v* ruleset with no bypass plus a
  creation ruleset whose bypass list (admins) is who may tag — the shape
  ordeal runs today; an equivalent existing ruleset is left alone
- required workflows: the hostname-scan stub is opened as a PR only once
  its action pin is a merged sha, and required only where it exists
- private repos on GitHub Free are reported unenforceable, not failed

Fixes on the existing path: rulesetsMatch now compares bypass_actors and
target (an admin bypass was invisible drift); the legacy fallback PUT no
longer wipes a repo's own required contexts; sync and plan share one
target selection. Forks keep their pre-standards behaviour.

`cli.js plan` runs configureRepository against a recording octokit (GETs
throttled to 1 req/s and cacheable, writes recorded, never sent);
`cli.js audit --snapshots DIR` evaluates a saved sweep offline. Tests
include positive and negative controls for both the config floor and the
drift report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
Dogfooding the scanner on this branch flagged exactly its own RFC 1918 /
CGNAT / link-local / ULA prefix definitions — the one place they belong.
Mark those lines with the inline pragma; the rest of the repo and its
whole history scan clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
@temper-pulseengine

Copy link
Copy Markdown
Contributor

Automated review for PR #77

pulseengine/temper:feat/standards-dry-run → pulseengine/temper:main

Verdict: 💬 Comment

Summary: The changes look good and address the issues identified in the previous review. The new standards-schema.js file provides validation for config-level standards, enhancing the robustness of the codebase. Additionally, the applyStandards function ensures that the org standards are applied correctly to repositories, including handling forks and private repos on GitHub Free. Overall, these changes

Findings: 0 mechanical (rivet) · 1 from local AI model.

Findings (1):

  1. __tests__/integration/rulesets.test.js:186
    expect(result.action).toBe('created');
    
    The test case for applyRulesetFromBranchProtection is missing a description or explanation.

Generated by a local AI model and post-validated against a strict JSON contract. Each finding includes the verbatim line being criticised — verify by reading the file at the cited location.

Reviewed at f173c0c

avrabe and others added 2 commits October 30, 2026 02:15
GitHub fills server-side defaults into a stored ruleset's rules
(allowed_merge_methods, required_reviewers, dismissal_restriction, ...).
rulesetsMatch compared them strictly, so every sync re-PUT an unchanged
managed ruleset — the dry-run showed 8 such no-op updates across the org.
Project the stored rules onto the parameter keys temper specifies before
comparing; a changed parameter temper does set is still drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
With standards enabled, forks no longer get the managed branch ruleset
through the legacy path: a required-signatures / non-fast-forward ruleset
on a fork's default branch breaks syncing it from upstream. They get merge
settings and labels only, matching the config comment.

standards.repos.<repo>.report_only: "<reason>" reports a repo's drift but
enforces nothing, for repos whose own automation the floor would break
(first use: a deposit workflow that commits as github-actions[bot] and
pushes straight to main). Test fixtures use generic names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
@avrabe
avrabe merged commit 32ba963 into main Oct 30, 2026
6 checks passed
@avrabe
avrabe deleted the feat/standards-dry-run branch October 30, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant