[Michael McQuade (giraffesyo)]

Summary

• Every Filter, Reachable and Holders reads every grant to the tenant's members and the public on the listed types, of built-in roles too, even for the anonymous and guests, who count none of the members'. (L1, and the first half of X2; a regression from fix: searches read every grant of a custom role in the tenant once planned generically, and as of a time lists read every version of every deny on their types and searches and Grantable every version of every custom role #76)
  The problem: before its statement, a search reads which custom roles the broad principals' grants name on the listed types or path (broadArgs). fix: searches read every grant of a custom role in the tenant once planned generically, and as of a time lists read every version of every deny on their types and searches and Grantable every version of every custom role #76 moved the role LIKE 'custom.%' test outside that read (namedCustom) so a generic plan couldn't pick rbac_grants_custom_role by the tenant alone. As a result the read went through rbac_grants_principal, which holds every grant of the broad principals, built-in roles' too, and it read both broad keys whatever the subject counts. With members:t1 holding built-in cluster.login on 20,000 clusters, the anonymous subject's Filter went from 33 to 4,713 pages of rbac_grants (17 → 28 at 05abbfb), and a member's Reachable from 30 to 9,378. The same happened in tenants with no custom role at all, and for a permission no public role gives, the public's grants were read for nothing.
  The fix: migration 0018_principal_seeks adds rbac_grants_custom_principal, a partial index on (principal, tenant, scope) INCLUDE (role) WHERE role LIKE 'custom.%'. Live, the prefix is tested inside the per-key, per-type read again. The new index leads with the principal, the tenant and the scope, so it is more selective than rbac_grants_custom_role under any plan, and it holds no built-in role's grant. fix: searches read every grant of a custom role in the tenant once planned generically, and as of a time lists read every version of every deny on their types and searches and Grantable every version of every custom role #76's two tests still pass. broadArgs also reads only the broad keys its caller counts:

  • Reachable passes the broad keys among the subject's counted keys.
  • Holders passes those a grant can give p through, so the public's only where a public role can give it.
  • Filter drops the members' key for the anonymous subject. For an identity, the members' key is read only while the subject is a member (not a guest, not suspended), tested in the same statement the way keysSQL counts it. Filter's doc now says the condition also fixes that.
    The index or the key filter alone isn't enough: when the members hold a built-in role on thousands of clusters and others hold a custom role on thousands, either old index reads all of one or the other. A guest or the anonymous beside the members' custom grants reads them all unless the members' key is dropped.

• Filter, its condition and Reachable read the row of every grant their subject holds on the listed types. (X4)
  The problem: filterHeld, filterCounted and Reachable's held read selected and tested principal_kind. No rbac_grants index holds it, so every grant they read was fetched from the heap: 2,064 heap pages for a subject holding a grant on each of 10,000 clusters, against 1 for Authorize.
  The fix: the broad principals are told apart by their key (narrowKey: not public:, not members:%). That is equivalent, because a kind holds no colon, the public's id is empty, and the tables check both. principal_kind is dropped from the columns read, so the read is index-only on rbac_grants_principal. broadKindsSQL is gone.

• As of a time, Filter and Reachable read every version of the members' grants on the listed types for the anonymous and guests, and each version of a group's grants on other types once for each type. (X2's second half, and the as-of read of a subject's own grants)
  The problem: history's index of a principal's versions, rbac_history_principal (tenant, principal, entity, effective_at), holds no scope. Each per-type read of a principal's grants (the subject's in Filter's and Reachable's held reads, the broad principals' in broadArgs) either:

  • read every version of the principal's grants, of every type, once for each listed type; or
  • read every version of every grant on the type's scopes through rbac_history_scope, everyone's.

  Which one depended on statistics. With L1's fix in, the auditor's anonymous Filter as of a time beside the members' grants on clusters still went from 107 to 2,978 history pages. Beside a guest's group holding a role on thousands of volumes, a list of clusters went 382 → 6,363.
  The fix: the same migration replaces rbac_history_principal with an index on (tenant, principal, entity, split_part(scope, ':', 1), scope, effective_at DESC) WHERE principal IS NOT NULL and analyzes history. As of a time, byType tests a type as split_part(g.scope, ':', 1) = t.t, an equality this index seeks, instead of a range of scopes.

  • Why an equality: a range whose bounds come from unnest(types) gets a default estimate of a few rows. In a history of few principals, where the plan estimates a principal's versions as a large share, that made rbac_history_scope look as cheap even with the principal as a constant (cost 52 vs 95 for the same one-row estimate). The equality is one rbac_history_scope can't serve, and ANALYZE keeps statistics for the expression.
  • It covers every reader of the old index (the as-of views by tenant, principal and entity: grants, eligibilities, memberships, members). History, Export and PruneHistory read by txid or key.
  • The audit suggested reading the broad principals' versions once for all listed types instead. That is weaker: it reads every version of the principal on unlisted types too. A member listing clusters beside its members' grants on 500 → 5,000 volumes went 405 → 2,680 that way, against 196 → 259 with the index.

• As of a time, a check, Permissions and Grantable read every version of their subject's grants to find those on the resource's path. (found while measuring the old index's readers)
  The problem: the same index held no scope, so a plan could read a principal's grant versions only all at once. In a history of many tenants, Authorize, Permissions and Grantable for a subject holding 3,000 grants read 965, 965 and 994 history pages.
  The fix: the replacement index holds the scope after its type, and PostgreSQL 18's skip scan seeks it over the few types, with no change to the checks' statements: 68, 68 and 100 pages. Every other as-of reader read the same pages within ±3: Explain, Grants, Eligibilities, Memberships, GroupsOf, Members, Holders.

The migration is numbered 0018, after #79's 0017_transaction_dates, which is on canary and merged into this branch. It builds an index on rbac_grants, which holds writes to it, and drops and rebuilds history's index, which holds reads of history and every change until it commits. It locks rbac_grants and then rbac_history before it builds either, as 0013 does: locked as each statement reached it, history was locked only once the grants' index was built, so a change that started during the build and wrote history before a grant, as DeleteScope does (a scope's eligibilities, then its grants), made Migrate give way after building the index (and a host's own tool without a lock_timeout deadlock with it). Now only a transaction already under way when it starts, that read history or made a change and then writes a grant, can make it give way, before it builds anything. The README's upgrade notes say this.

Testing

RBAC_TEST_DATABASE_URL=… GODEBUG=fips140=only. Each test was run with its fix reverted in a form that compiles:

• TestSearchesReadOnlyTheBroadGrantsTheyCount (live under custom and generic plans, as of a time under AsOf's plans, in the crowded harness): grants on 500 → 5,000 clusters to the members, to others, and to the public, with the subjects that don't count them.
  • With the prefix tested outside again (the new index unused): a member's Filter 154 → 1,183 and 166 → 1,187.
  • With the prefix inside but no new index: 105 → 1,176 and 160 → 1,176.
  • Reading every broad key: the anonymous's and the guest's Filter and Reachable about 110–150 → 1,125; as of a time about 2,230 → 22,060; a member's search of cluster:root beside the public's custom grants about 113–156 → 1,136.
  • Without the membership test: the guest's Filter 109 → 1,122 and 145 → 1,134.
• TestListingsReadTheirGrantsFromAnIndexAlone (custom and generic plans, after VACUUM marks every row visible; skips otherwise): a subject's grants on 1,000 clusters among others'. Without the fix, Filter with its condition and Reachable each read about one heap page per grant (2,972 and 2,963 at 3,000 grants); the bound is 10. TestChecksReadTheirGrantsFromAnIndexAlone now shares its VACUUM loop (grantsVisibleToAll).
• TestAsOfSearchesSeekTheirPrincipalsGrantsOnEachType (one tenant and many, as AsOf plans and generically): the members' grants on 500 → 5,000 clusters and the guest's group's on as many volumes, searched by the anonymous and the guest, plus Holders.
  • Without the fix: the anonymous's Filter 748 → 5,979, the guest's Filter 382 → 6,363 and Reachable 125 → 1,523; all four cases fail.
  • With the index but the range kept: one tenant fails (the anonymous's Filter 605 → 4,526), and many tenants fails generically.
  • With the equality but the old index: the guest's Filter 1,423 → 12,240.
• TestAsOfChecksSeekTheirSubjectsGrantsOnThePath (crowded; AsOf's and generic plans): a subject's grants on 500 → 5,000 other clusters. With the type-only index, Authorize 80 → 537 (Grantable 94 → 551); with the old index, 77 → 534.
• TestMigratingThePrincipalSeeksWaitsForAChangeStartedAsItBuilds: Migrate, tried once, while DeleteScope starts during the grants' index build (100,000 grants, maintenance_work_mem=64kB). With the tables locked as each statement reached them, Migrate fails with 55P03 after building the index, 3 runs of 3; applied without a lock_timeout, DeleteScope fails as a deadlock.
• Existing tests: TestSearchesUnderAGenericPlanReadOnlyTheBroadGrantsOfCustomRoles, TestSearchesReadAsMuchOnceAConnectionPlansThemGenerically, every TestAsOf*, and the rest of the suite pass, with -race, after merging canary. TestThePackageRunsAsTheReadmesDatabaseRoles (from fix: as of a time, a revoke a procedure or DO block makes after it commits counts from before moments it came after, migrating deadlocks with a change reading the tables in another order, the README's role setup fails every call, and AsOf panics on a nil read #79) applies 0018 with Migrate as the README's administrator's role once the roles are hardened.
• Scratch checks, not committed:
  • The auditors' scale repros. With 100 → 20,000 of the members' grants on clusters, the anonymous's Filter reads 28 → 40 pages live (36 → 28 generic). As of a time beside 10,000 it reads 72 → 92 history pages, and the guest's Reachable 77 → 107. The listing query's heap reads are 1 page, against 2,064 before.
  • Auditor 2's randomized comparison of Filter, Reachable, Holders and Grantable with Authorize and Apply, live and as of one or several times: all seven tests pass at 12 seeds.
  • A comparison of every as-of reader of the old index against the new. Beside 20 rounds of the subject's and its group's grants revoked and made again after the time, the principal-wide readers read up to about 20% more history with the new index (Grants of a principal 1,240 → 1,483), as the time is no longer the index's range; in many tenants Authorize, Permissions and Grantable read less (106 → 48).
  • A predicate built while the subject is a member and used once it is a guest, suspended, expired or gone keeps nothing Authorize then refuses. One built in those states and used once it is a member keeps the members' grants of built-in roles but not of custom ones, as Filter's doc says.
  • Ids that look like broad keys (user:members:t1, user:public:, group:public:, declared kinds membersx, public_, members-), and as of a time scope ids holding colons (cluster:a:1, workspace:cluster:a) beside types that start alike (org, organization): Filter and Reachable agree with Authorize.
  • Under the default plan cache on one connection, runs 6 to 8, live and as of a time, beside 500 → 5,000 broad grants: no search grows that doesn't count them. As of a time, a member's Filter call reads every version of the members' grants of built-in roles on the type (2,254 → 21,590), as the prefix isn't tested there; its query counts them anyway (4,510 → 43,181).
  • Auditor 2's comparisons again: round 6's at 12 seeds, and 12 more with ids holding :, '%_, members:t1, group:x and spaces, as of a time too; round 7's at 16 fresh seeds. All pass but TestAuditPlainListingsAsOfATime's MembershipsOf, which reads every tenant now and only the one as of a time, as AsOf documents.

Checklist

• make check passes
• No exported name, stored value or table was renamed (they are contract once released). rbac_history_principal keeps its name with new columns, and the migration is new and unreleased.