diff --git a/CHANGELOG.md b/CHANGELOG.md index 736e4ef..5627636 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,55 @@ the source. ## [Unreleased] +Targets the 2.3.1 patch release. It fixes a JDBI upgrade failure introduced in +2.3.0 for hosts that supply their own users table. Hosts using the documented +`classpath:db/migration` location need no change; the upgrade note below covers +the other configurations. + +### Fixed + +- JDBI hosts with their own identity model can apply pk-auth's migrations + without the example `users` table. Up to 2.3.0 every migration lived flat in + `db/migration`, so a host that excluded the example `V5__example_users.sql` + failed on 2.3.0's `V12__users_username_case_insensitive.sql` with + `relation "users" does not exist`. The migrations are now split into two + classpath locations sharing one version sequence: `db/migration/core` (the + tables pk-auth owns, always required) and `db/migration/example-users` (`V5` + and `V12`, needed only with `JdbiUserLookup`). Every script keeps its file + name and its bytes, so Flyway checksums are unchanged, and `V12` itself is + not modified. `PkAuthJdbiSchema` gains `CORE_MIGRATION_LOCATION`, + `EXAMPLE_USERS_MIGRATION_LOCATION`, `MIGRATION_LOCATION`, + `CORE_SCHEMA_VERSION` (`11`), and `migrateCoreForDevelopment(DataSource)`. + See [ADR 0021](./docs/adr/0021-split-example-users-migrations.md) and the + [operator guide](./docs/operator-guide.md#jdbi--postgres). + + Upgrade note, by host configuration: + + - Flyway location `classpath:db/migration`, with `JdbiUserLookup` or the + example table: no change. Flyway scans the location recursively and resolves + the same twelve migrations with the same checksums, so an existing database + validates and nothing new runs. + - Own identity model, new database (or one that never applied `V5`): list + `classpath:db/migration/core` and target + `PkAuthJdbiSchema.CORE_SCHEMA_VERSION`. Flyway refuses a `target` that no + configured location resolves, so `CURRENT_SCHEMA_VERSION` (`12`) fails + against the core location alone. + - Own identity model, scripts copied out of the jar by pattern with `V5` + excluded: change the pattern from `db/migration/V*.sql` to + `db/migration/core/V*.sql` and drop the `V5` exclusion. Neither `V5` nor + `V12` needs excluding any more. The old pattern matches nothing in 2.3.1, + and Flyway validation then fails on the applied migrations it can no longer + resolve. A database left by the failed 2.3.0 upgrade sits at `V11` (Postgres + rolled the failed `V12` back) and validates against the core set without + further action. + - Spring Boot with the default `spring.flyway.locations` + (`classpath:db/migration`): unchanged, and the default still applies both + sets from the jar. A new host that wants only the core set sets the property + explicitly to its own location plus `classpath:db/migration/core`. + - A database that applied `V5` and `V12` must keep the example location + configured. Dropping it fails Flyway validation, because the applied example + migrations no longer resolve. + ## [2.3.0] - 2026-10-08 A security and maintainability release. The Security entries close findings diff --git a/CLAUDE.md b/CLAUDE.md index 60abc0b..4a6956d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -94,9 +94,13 @@ account/credential/backup-code/email/phone admin operations mounted by all three uses conditional `UPDATE ... WHERE consumed_at IS NULL`. DynamoDB uses `ConditionExpression` (a failed condition raises `ConditionalCheckFailedException`, mapped to `Conflict` / `Expired`). - DynamoDB is single-table (`DynamoDbTable` per item type; refresh tokens write 3 items per - token for the jti/user/family indexes). JDBI uses Flyway migrations at - `pk-auth-persistence-jdbi/src/main/resources/db/migration/`, and adding one requires bumping - `PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION`. + token for the jti/user/family indexes). JDBI uses Flyway migrations under + `pk-auth-persistence-jdbi/src/main/resources/db/migration/`, split into `core/` (pk-auth's + tables) and `example-users/` (the example `users` table; opt-in for hosts). Both share one version + sequence. A `core/` migration bumps `PkAuthJdbiSchema.CORE_SCHEMA_VERSION` and + `CURRENT_SCHEMA_VERSION`; an `example-users/` one bumps only `CURRENT_SCHEMA_VERSION`. No `core/` + migration may reference `users`, and released scripts are never edited or renamed (Flyway + checksums). See ADR 0021. ## Enforced conventions diff --git a/DESIGN.md b/DESIGN.md index e7997e2..95d1a7a 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -111,7 +111,7 @@ host. | `pk-auth-otp` | Alt flow: 6-digit OTPs over the host's SMS dispatcher; hashed with HMAC-SHA256 (server-side pepper) and atomic-claim. | | `pk-auth-refresh-tokens` | Rotating refresh tokens with family-based replay defence. `RefreshTokenService` + `RefreshTokenRepository` SPI; `RefreshHandler` is the framework-neutral `POST /auth/refresh` composer ([ADR 0013](./docs/adr/0013-refresh-tokens-family-rotation.md)). | | `pk-auth-admin-api` | `AdminService` exposes account/credential/backup-code/email/phone operations. Result-typed (`AdminResult` sealed sum). | -| `pk-auth-persistence-jdbi` | SPI impls on JDBI + Postgres + Flyway. Migrations at `src/main/resources/db/migration/`. | +| `pk-auth-persistence-jdbi` | SPI impls on JDBI + Postgres + Flyway. Migrations at `src/main/resources/db/migration/{core,example-users}/`. | | `pk-auth-persistence-dynamodb` | SPI impls on AWS SDK v2 DynamoDB Enhanced. Single table, schema per item-type ([ADR 0008](./docs/adr/0008-dynamodb-single-table-design.md)). | | `pk-auth-testkit` | `FakeAuthenticator` for ceremony-driving tests + `InMemoryX` for every SPI. | | `pk-auth-spring-boot-starter` | Spring Boot 4 / Spring Security 7 autoconfigure + controller. | @@ -299,8 +299,13 @@ Two real-backend modules ship in-tree, both implementing the same SPIs. - Migrations live under `pk-auth-persistence-jdbi/src/main/resources/db/migration/` (Flyway). The schema is hand-tuned for the SPI access patterns; there is no JPA / Hibernate. -- Tables: `users`, `credentials`, `challenges`, `backup_codes`, `otp_codes` (V1–V5, no `pkauth_` - prefix), plus the append-only `pkauth_audit_events` table from V6. +- The migrations are split across two classpath locations sharing one version sequence + ([ADR 0021](./docs/adr/0021-split-example-users-migrations.md)). `db/migration/core` holds the + tables pk-auth owns and is always applied. `db/migration/example-users` holds the example `users` + table (V5, V12) read by `JdbiUserLookup`; a host with its own identity model leaves it out. The + parent `db/migration` resolves both. +- Tables: `credentials`, `challenges`, `backup_codes`, `otp_codes` (V1–V4, no `pkauth_` prefix), + the example `users` table (V5), plus the append-only `pkauth_audit_events` table from V6. - `V8__create_access_tokens.sql` and `V9__create_refresh_tokens.sql` add the stateful-access-token and refresh-token tables for the 1.1.0 SPIs. `V10__refresh_tokens_amr.sql` adds the `amr` (RFC 8176 authentication-method-reference) column to `refresh_tokens`. @@ -308,7 +313,8 @@ Two real-backend modules ship in-tree, both implementing the same SPIs. requirement. `V12__users_username_case_insensitive.sql` makes username uniqueness case-insensitive (unique index on `lower(username)`) so the JDBI and DynamoDB backends share one identity model. It refuses to run, naming the offending rows, if the database already holds - usernames differing only by case. `PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION` is `"12"`. + usernames differing only by case. `PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION` is `"12"`, and + `PkAuthJdbiSchema.CORE_SCHEMA_VERSION` (the highest core migration) is `"11"`. - Magic-link tokens are not persisted: the JWT itself is the credential. Consumed JTIs live in a `ConsumedJtiStore` (in-memory by default; a shared backend replaces it for multi-replica deployments). diff --git a/README.md b/README.md index 3e457c3..cef7bc3 100644 --- a/README.md +++ b/README.md @@ -46,7 +46,10 @@ All modules share one version and the `com.codeheadsystems` group id. - A framework-neutral admin service that every adapter mounts at the same paths, so the same TypeScript SDK drives all three. - Persistence options: in-memory (testkit), JDBI + Postgres with Flyway migrations, or DynamoDB - single-table. + single-table. The JDBI migrations ship as a required core set (`classpath:db/migration/core`) + and an opt-in example `users` set (`classpath:db/migration/example-users`) used only with + `JdbiUserLookup`; the [operator guide](./docs/operator-guide.md#jdbi--postgres) gives the Flyway + configuration for each. - A zero-dependency browser SDK (`@pk-auth/passkeys-browser`) covering both ceremony and admin operations, published on npm (`npm install @pk-auth/passkeys-browser`). Its version tracks the pk-auth server release it speaks to. diff --git a/docs/adr/0021-split-example-users-migrations.md b/docs/adr/0021-split-example-users-migrations.md new file mode 100644 index 0000000..64cf9f8 --- /dev/null +++ b/docs/adr/0021-split-example-users-migrations.md @@ -0,0 +1,71 @@ +# 21. Separate classpath locations for core and example-users migrations + +Date: 2026-10-16 + +## Status + +Accepted. + +## Context + +`pk-auth-persistence-jdbi` ships its Flyway migrations as classpath resources. Up to 2.3.0 every +migration lived flat in `db/migration`, and the documented host configuration was to add +`classpath:db/migration` to the host's Flyway locations. + +Two of those migrations are not pk-auth data. `V5__example_users.sql` creates a `users` table that +pk-auth does not own ([ADR 0006](./0006-userlookup-spi-not-owned.md)); it exists for the reference +`JdbiUserLookup`, the demos, and the tests. `V12__users_username_case_insensitive.sql` (2.3.0) +queries and indexes that table. A host with its own identity model has no use for either, and had +no supported way to leave them out. Hosts that extracted the scripts from the jar and excluded `V5` +failed on `V12` in 2.3.0 with `relation "users" does not exist`, and any later migration on `users` +would fail the same way. + +The released migrations constrain the fix. Flyway validates each applied migration's version, +description, and checksum against the resolved script, so a released script's file name and +contents cannot change without failing validation on every database that applied it. Its location +on the classpath is not validated. Flyway also scans a location recursively, in a directory and in +a jar alike, and resolves a script reached through two overlapping locations once. + +Three layouts were considered. + +1. Example migrations moved to a sibling location (`db/migration-example-users`), core left in + `db/migration`. A host with its own identity model needs no change. A host using + `JdbiUserLookup` and the documented `classpath:db/migration` stops resolving `V5` and `V12`, and + Flyway validation then fails on the applied-but-missing migrations until the host adds the + second location. +2. Both sets moved into subdirectories of `db/migration` (`core`, `example-users`). The documented + `classpath:db/migration` still resolves the full set, so every host using it is unaffected. A + host with its own identity model lists `classpath:db/migration/core`. A host that extracted + scripts with a non-recursive `db/migration/V*.sql` pattern finds none. +3. A guard (`to_regclass('users')`) added to `V12`. This changes a released checksum, and it leaves + every future `users` migration to repeat the guard. + +## Decision + +The migrations are split into two subdirectories of `db/migration` (option 2): + +- `db/migration/core` holds every table pk-auth owns. Every JDBI host applies it. +- `db/migration/example-users` holds `V5__example_users.sql`, + `V12__users_username_case_insensitive.sql`, and every future migration on the example `users` + table. It is opt-in. + +Both sets keep one version sequence, so their numbers never collide and either set applies alone. +Every script keeps its file name and its bytes. `PkAuthJdbiSchema` publishes the three locations +as `MIGRATION_LOCATION`, `CORE_MIGRATION_LOCATION`, and `EXAMPLE_USERS_MIGRATION_LOCATION`, and adds +`CORE_SCHEMA_VERSION` beside `CURRENT_SCHEMA_VERSION`, because Flyway refuses a `target` that no +configured location resolves. + +## Consequences + +- The documented `classpath:db/migration` configuration resolves the same twelve migrations with the + same checksums, so the change ships as the 2.3.1 patch release. +- A host with its own identity model applies `classpath:db/migration/core` with + `CORE_SCHEMA_VERSION` as its target, and needs no knowledge of the example set. +- A host that copied scripts out of the jar by file pattern updates the pattern to + `db/migration/core/V*.sql`. That integration relied on the resource layout inside the jar, which + was never a documented surface. +- A new migration goes into exactly one of the two directories. A migration in `core` bumps both + `CORE_SCHEMA_VERSION` and `CURRENT_SCHEMA_VERSION`; one in `example-users` bumps only + `CURRENT_SCHEMA_VERSION`. No core migration may reference the `users` table. +- `JdbiMigrationLocationsIntegrationTest` pins the split: core alone on an empty database, core + with the example set, the parent location, and databases migrated under the 2.3.0 flat layout. diff --git a/docs/adr/README.md b/docs/adr/README.md index ac19b3d..5e63c87 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -28,3 +28,4 @@ ADR is added and the prior one is marked `Superseded by NNNN`. See | [0018](./0018-remove-sonarqube-cloud.md) | Accepted | 2026-06-11 | Native JaCoCo coverage gates replacing SonarQube Cloud | | [0019](./0019-crypto-agility-post-quantum-readiness.md) | Accepted | 2026-06-17 | Crypto-agility and post-quantum readiness for passkey algorithms | | [0020](./0020-browser-sdk-testing-strategy.md) | Accepted | 2026-06-18 | Browser SDK testing strategy | +| [0021](./0021-split-example-users-migrations.md) | Accepted | 2026-10-16 | Separate classpath locations for core and example-users migrations | diff --git a/docs/operator-guide.md b/docs/operator-guide.md index d94e47e..1d14a3b 100644 --- a/docs/operator-guide.md +++ b/docs/operator-guide.md @@ -66,20 +66,81 @@ available to select yet; see the "Post-quantum readiness" section of `docs/threa ### JDBI / Postgres -- Flyway resources live in `pk-auth-persistence-jdbi/src/main/resources/db/migration`. -- No adapter runs these migrations. The host runs them by adding `classpath:db/migration` from this - artefact to its own Flyway locations, targeting `PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION`. - `PkAuthJdbiSchema.migrateForDevelopment(DataSource)` exists for demos and integration tests only. -- The shipped baseline is split across `V1__credentials.sql`, `V2__challenges.sql`, - `V3__backup_codes.sql`, `V4__otp_codes.sql`, and `V5__example_users.sql`: five tables - (`credentials`, `challenges`, `backup_codes`, `otp_codes`, `users`) with no `pkauth_` prefix. - `V6__audit_soft_delete.sql` adds the append-only `pkauth_audit_events` table. +- No adapter runs these migrations. The host runs them with its own Flyway setup. + `PkAuthJdbiSchema.migrateForDevelopment(DataSource)` and + `PkAuthJdbiSchema.migrateCoreForDevelopment(DataSource)` exist for demos and integration tests + only. +- The migrations ship in two classpath locations that share one version sequence. Version numbers + never collide across the two, so either set can be applied alone or both together. + +| Location | Constant | Contents | Required | +|---|---|---|---| +| `classpath:db/migration/core` | `CORE_MIGRATION_LOCATION` | Every table pk-auth owns. | Always. | +| `classpath:db/migration/example-users` | `EXAMPLE_USERS_MIGRATION_LOCATION` | The example `users` table (`V5`, `V12`). | Only with `JdbiUserLookup`. | +| `classpath:db/migration` | `MIGRATION_LOCATION` | Both of the above. | Alternative to listing both. | + +- The example `users` table is host-app data that pk-auth does not own (see + [ADR 0006](./adr/0006-userlookup-spi-not-owned.md)). It exists for the reference + `JdbiUserLookup`, the demos, and the tests. A host with its own identity model and its own + `UserLookup` applies only the core location, and needs no knowledge of the example set. +- Flyway scans a location recursively, so `classpath:db/migration` resolves both subdirectories. + It is the location documented before 2.3.1, and hosts that already list it keep working + unchanged. Listing a subdirectory alongside its parent is harmless: Flyway resolves each + migration once. +- Spring Boot's default `spring.flyway.locations` is `classpath:db/migration`. Because classpath + scanning spans every jar, that default picks up both pk-auth sets from this artefact. A Spring + Boot host with its own identity model sets `spring.flyway.locations` explicitly, to its own + location plus `classpath:db/migration/core`, and keeps its own migrations out of + `db/migration` itself. +- The Flyway `target` depends on the locations applied. `PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION` is + the highest version across both sets, and `PkAuthJdbiSchema.CORE_SCHEMA_VERSION` is the highest + core version. Flyway refuses a `target` that no configured location resolves, so a core-only host + targets `CORE_SCHEMA_VERSION`. +- See [ADR 0021](./adr/0021-split-example-users-migrations.md) for the split. + +#### Flyway configuration with `JdbiUserLookup` + +```java +Flyway.configure() + .dataSource(dataSource) + .locations( + "classpath:db/migration/host", // the host's own migrations, if any + PkAuthJdbiSchema.CORE_MIGRATION_LOCATION, + PkAuthJdbiSchema.EXAMPLE_USERS_MIGRATION_LOCATION) + .target(PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION) + .load() + .migrate(); +``` + +#### Flyway configuration with a host-owned users table + +```java +Flyway.configure() + .dataSource(dataSource) + .locations("classpath:db/migration/host", PkAuthJdbiSchema.CORE_MIGRATION_LOCATION) + .target(PkAuthJdbiSchema.CORE_SCHEMA_VERSION) + .load() + .migrate(); +``` + +A host whose own migrations share pk-auth's version sequence (one `flyway_schema_history` table) +does not pin `target` to a pk-auth constant; it orders its own versions around pk-auth's instead. + +#### Shipped migrations + +- Core: `V1__credentials.sql`, `V2__challenges.sql`, `V3__backup_codes.sql`, and + `V4__otp_codes.sql` create `credentials`, `challenges`, `backup_codes`, and `otp_codes`, with no + `pkauth_` prefix. `V6__audit_soft_delete.sql` adds the append-only `pkauth_audit_events` table. `V7__credentials_hard_delete.sql` drops the `revoked_at` / `revoked_reason` columns on `credentials`: credential delete is a hard delete, with the audit record captured as a structured log event (`pkauth.credential.deleted`). `V8__create_access_tokens.sql` and `V9__create_refresh_tokens.sql` add the 1.1.0 `access_tokens` and `refresh_tokens` tables; `V10__refresh_tokens_amr.sql` adds the `amr` (RFC 8176 authentication-method-reference) column to - `refresh_tokens`. `V11` and `V12` follow; the migration directory lists the current set. + `refresh_tokens`. `V11__challenges_user_verification.sql` follows; the `core` directory lists the + current set. +- Example users: `V5__example_users.sql` creates `users`, and + `V12__users_username_case_insensitive.sql` adds the case-insensitive unique index + `users_username_lower_key`. - Magic-link tokens are not persisted: the JWT is the credential, and the consumed-JTI store is in-memory by default (see the `ConsumedJtiStore` SPI for a multi-replica override). - The unique key on credential ID is byte-array shaped. Introducing a string-encoded column for it diff --git a/gradle.properties b/gradle.properties index 6dac3d7..3c7dc11 100644 --- a/gradle.properties +++ b/gradle.properties @@ -21,4 +21,4 @@ org.gradle.jvmargs=-Xmx2g \ # Project identity group=com.codeheadsystems -version=2.4.0-SNAPSHOT +version=2.3.1-SNAPSHOT diff --git a/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiUserLookup.java b/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiUserLookup.java index bcfa97b..16d16a1 100644 --- a/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiUserLookup.java +++ b/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiUserLookup.java @@ -13,7 +13,9 @@ /** * {@link UserLookup} backed by the demo {@code users} table (see {@code V5__example_users.sql}). * The brief notes that the users table is host-app data; this implementation is the reference - * provided alongside the demo schema. + * provided alongside the demo schema. The table's migrations live in {@link + * PkAuthJdbiSchema#EXAMPLE_USERS_MIGRATION_LOCATION}, which a host using this class applies + * alongside {@link PkAuthJdbiSchema#CORE_MIGRATION_LOCATION}. */ public final class JdbiUserLookup implements UserLookup { diff --git a/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/PkAuthJdbiSchema.java b/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/PkAuthJdbiSchema.java index 53367dc..7318fc0 100644 --- a/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/PkAuthJdbiSchema.java +++ b/pk-auth-persistence-jdbi/src/main/java/com/codeheadsystems/pkauth/persistence/jdbi/PkAuthJdbiSchema.java @@ -5,15 +5,29 @@ import org.flywaydb.core.Flyway; /** - * Convenience helper to run pk-auth's Flyway migrations against a {@link DataSource}. + * Convenience helper to run pk-auth's Flyway migrations against a {@link DataSource}, and the + * canonical names of the classpath locations those migrations ship in. * - *

For development and test use only. The {@link #migrateForDevelopment(DataSource)} helper is - * scoped to demos and integration tests. Production hosts must drive Flyway themselves so they can - * control the target schema version, baseline settings, out-of-order handling, and other - * operational parameters, by adding {@code classpath:db/migration} from this artifact to the host's - * existing Flyway location list. + *

The migrations are split across two locations that share one version sequence: * - *

The current schema version shipped by this library is {@value #CURRENT_SCHEMA_VERSION}. + *

+ * + *

{@link #MIGRATION_LOCATION} is the parent of both. Flyway scans locations recursively, so it + * resolves the full set, exactly as it did before the split. + * + *

For development and test use only. The {@code migrate*ForDevelopment} helpers are scoped to + * demos and integration tests. Production hosts must drive Flyway themselves so they can control + * the target schema version, baseline settings, out-of-order handling, and other operational + * parameters, by adding the relevant locations from this artifact to the host's existing Flyway + * location list. + * + *

The current schema version shipped by this library is {@value #CURRENT_SCHEMA_VERSION}, and + * the current core schema version is {@value #CORE_SCHEMA_VERSION}. */ public final class PkAuthJdbiSchema { @@ -24,21 +38,69 @@ public final class PkAuthJdbiSchema { */ public static final String CURRENT_SCHEMA_VERSION = "12"; + /** + * The highest Flyway migration version in {@link #CORE_MIGRATION_LOCATION}. A host that applies + * only the core location targets this value: Flyway refuses a {@code target} that no configured + * location resolves, and {@link #CURRENT_SCHEMA_VERSION} can belong to an example migration. + * + * @since 2.3.1 + */ + public static final String CORE_SCHEMA_VERSION = "11"; + + /** + * The parent location: the core and the example {@code users} migrations together. + * + * @since 2.3.1 + */ + public static final String MIGRATION_LOCATION = "classpath:db/migration"; + + /** + * The migrations for the tables pk-auth owns. Required by every JDBI host. + * + * @since 2.3.1 + */ + public static final String CORE_MIGRATION_LOCATION = "classpath:db/migration/core"; + + /** + * The migrations for the example {@code users} table read by {@link JdbiUserLookup}. Opt-in. + * + * @since 2.3.1 + */ + public static final String EXAMPLE_USERS_MIGRATION_LOCATION = + "classpath:db/migration/example-users"; + private PkAuthJdbiSchema() {} /** - * Runs pk-auth's Flyway migrations up to {@link #CURRENT_SCHEMA_VERSION} on the supplied {@code - * dataSource}. Idempotent. + * Runs the core and the example {@code users} migrations up to {@link #CURRENT_SCHEMA_VERSION} on + * the supplied {@code dataSource}. Idempotent. * *

For development and integration testing only. Production code must not call this method; * production hosts should configure Flyway independently. */ public static void migrateForDevelopment(DataSource dataSource) { - Flyway.configure() - .dataSource(dataSource) - .locations("classpath:db/migration") - .target(CURRENT_SCHEMA_VERSION) - .load() - .migrate(); + migrate( + dataSource, + CURRENT_SCHEMA_VERSION, + CORE_MIGRATION_LOCATION, + EXAMPLE_USERS_MIGRATION_LOCATION); + } + + /** + * Runs only the core migrations up to {@link #CORE_SCHEMA_VERSION} on the supplied {@code + * dataSource}, leaving out the example {@code users} table. Idempotent. Intended for hosts that + * supply their own {@code UserLookup}. + * + *

For development and integration testing only. Production code must not call this method; + * production hosts should configure Flyway independently. + * + * @since 2.3.1 + */ + public static void migrateCoreForDevelopment(DataSource dataSource) { + migrate(dataSource, CORE_SCHEMA_VERSION, CORE_MIGRATION_LOCATION); + } + + private static void migrate(DataSource dataSource, String target, String... locations) { + Flyway.configure().dataSource(dataSource).locations(locations).target(target).load().migrate(); } } diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V10__refresh_tokens_amr.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V10__refresh_tokens_amr.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V10__refresh_tokens_amr.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V10__refresh_tokens_amr.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V11__challenges_user_verification.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V11__challenges_user_verification.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V11__challenges_user_verification.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V11__challenges_user_verification.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V1__credentials.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V1__credentials.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V1__credentials.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V1__credentials.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V2__challenges.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V2__challenges.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V2__challenges.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V2__challenges.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V3__backup_codes.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V3__backup_codes.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V3__backup_codes.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V3__backup_codes.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V4__otp_codes.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V4__otp_codes.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V4__otp_codes.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V4__otp_codes.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V6__audit_soft_delete.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V6__audit_soft_delete.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V6__audit_soft_delete.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V6__audit_soft_delete.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V7__credentials_hard_delete.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V7__credentials_hard_delete.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V7__credentials_hard_delete.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V7__credentials_hard_delete.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V8__create_access_tokens.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V8__create_access_tokens.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V8__create_access_tokens.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V8__create_access_tokens.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V9__create_refresh_tokens.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V9__create_refresh_tokens.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V9__create_refresh_tokens.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/core/V9__create_refresh_tokens.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V12__users_username_case_insensitive.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/example-users/V12__users_username_case_insensitive.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V12__users_username_case_insensitive.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/example-users/V12__users_username_case_insensitive.sql diff --git a/pk-auth-persistence-jdbi/src/main/resources/db/migration/V5__example_users.sql b/pk-auth-persistence-jdbi/src/main/resources/db/migration/example-users/V5__example_users.sql similarity index 100% rename from pk-auth-persistence-jdbi/src/main/resources/db/migration/V5__example_users.sql rename to pk-auth-persistence-jdbi/src/main/resources/db/migration/example-users/V5__example_users.sql diff --git a/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiMigrationLocationsIntegrationTest.java b/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiMigrationLocationsIntegrationTest.java new file mode 100644 index 0000000..87ce93e --- /dev/null +++ b/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/JdbiMigrationLocationsIntegrationTest.java @@ -0,0 +1,301 @@ +// SPDX-License-Identifier: MIT +package com.codeheadsystems.pkauth.persistence.jdbi; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatCode; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +import com.zaxxer.hikari.HikariConfig; +import com.zaxxer.hikari.HikariDataSource; +import java.io.IOException; +import java.net.URISyntaxException; +import java.net.URL; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.stream.Stream; +import org.flywaydb.core.Flyway; +import org.flywaydb.core.api.output.MigrateResult; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.DisabledIfEnvironmentVariable; +import org.junit.jupiter.api.io.TempDir; +import org.testcontainers.containers.PostgreSQLContainer; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +/** + * Pins the split between the core migrations and the example {@code users} migrations. + * + *

Up to 2.3.0 every migration lived flat in {@code db/migration}, so a host that excluded the + * example {@code V5__example_users.sql} failed on {@code V12__users_username_case_insensitive.sql} + * with {@code relation "users" does not exist}. These tests run each location set against a fresh, + * empty database, and replay the 2.3.0 flat layout from a filesystem location to check that + * databases migrated under it still validate against the split layout. + */ +@Testcontainers +@DisabledIfEnvironmentVariable(named = "PKAUTH_SKIP_TESTCONTAINERS", matches = "1") +class JdbiMigrationLocationsIntegrationTest { + + private static final Set EXAMPLE_FILES = + Set.of("V5__example_users.sql", "V12__users_username_case_insensitive.sql"); + + private static final List CORE_TABLES = + List.of( + "credentials", + "challenges", + "backup_codes", + "otp_codes", + "pkauth_audit_events", + "access_tokens", + "refresh_tokens"); + + private static final AtomicInteger DATABASE_COUNTER = new AtomicInteger(); + + private static PostgreSQLContainer container; + + @TempDir Path tempDir; + + @BeforeAll + static void startContainer() { + container = + new PostgreSQLContainer<>(DockerImageName.parse("postgres:16-alpine")) + .withDatabaseName("pkauth_locations") + .withUsername("pkauth") + .withPassword("pkauth-test"); + container.start(); + } + + @AfterAll + static void stopContainer() { + if (container != null) { + container.stop(); + } + } + + // -- fresh databases --------------------------------------------------------------------------- + + /** The regression test for the 2.3.0 bug: a host with its own identity model. */ + @Test + void coreLocationAloneMigratesAnEmptyDatabaseWithoutAUsersTable() { + try (HikariDataSource ds = freshDatabase()) { + MigrateResult result = flyway(ds, PkAuthJdbiSchema.CORE_MIGRATION_LOCATION).migrate(); + + assertThat(result.success).isTrue(); + assertThat(result.targetSchemaVersion).isEqualTo(PkAuthJdbiSchema.CORE_SCHEMA_VERSION); + assertThat(tableExists(ds, "users")).isFalse(); + assertThat(indexExists(ds, "users_username_lower_key")).isFalse(); + CORE_TABLES.forEach(table -> assertThat(tableExists(ds, table)).as(table).isTrue()); + assertThatCode(() -> flyway(ds, PkAuthJdbiSchema.CORE_MIGRATION_LOCATION).validate()) + .doesNotThrowAnyException(); + } + } + + /** Guards the constant against drift: it must name the highest core migration on disk. */ + @Test + void coreSchemaVersionIsTheHighestCoreMigration() throws IOException { + try (Stream files = Files.list(classpathDirectory("db/migration/core"))) { + int highest = + files + .map(f -> f.getFileName().toString()) + .mapToInt(n -> Integer.parseInt(n.substring(1, n.indexOf("__")))) + .max() + .orElseThrow(); + assertThat(String.valueOf(highest)).isEqualTo(PkAuthJdbiSchema.CORE_SCHEMA_VERSION); + } + } + + @Test + void coreDevelopmentHelperLeavesOutTheUsersTable() { + try (HikariDataSource ds = freshDatabase()) { + PkAuthJdbiSchema.migrateCoreForDevelopment(ds); + + assertThat(tableExists(ds, "users")).isFalse(); + assertThat(tableExists(ds, "credentials")).isTrue(); + } + } + + @Test + void coreAndExampleLocationsCreateTheUsersTableAndCaseInsensitiveIndex() { + try (HikariDataSource ds = freshDatabase()) { + MigrateResult result = + flyway( + ds, + PkAuthJdbiSchema.CORE_MIGRATION_LOCATION, + PkAuthJdbiSchema.EXAMPLE_USERS_MIGRATION_LOCATION) + .migrate(); + + assertThat(result.success).isTrue(); + assertThat(result.targetSchemaVersion).isEqualTo(PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION); + assertThat(tableExists(ds, "users")).isTrue(); + assertThat(indexExists(ds, "users_username_lower_key")).isTrue(); + CORE_TABLES.forEach(table -> assertThat(tableExists(ds, table)).as(table).isTrue()); + } + } + + /** The location documented before 2.3.1 still resolves all twelve migrations. */ + @Test + void parentLocationStillResolvesTheFullSet() { + try (HikariDataSource ds = freshDatabase()) { + MigrateResult result = flyway(ds, PkAuthJdbiSchema.MIGRATION_LOCATION).migrate(); + + assertThat(result.migrationsExecuted).isEqualTo(12); + assertThat(tableExists(ds, "users")).isTrue(); + assertThat(indexExists(ds, "users_username_lower_key")).isTrue(); + } + } + + /** Documented in the operator guide: overlapping locations resolve each migration once. */ + @Test + void parentLocationTogetherWithASubdirectoryResolvesEachMigrationOnce() { + try (HikariDataSource ds = freshDatabase()) { + MigrateResult result = + flyway(ds, PkAuthJdbiSchema.MIGRATION_LOCATION, PkAuthJdbiSchema.CORE_MIGRATION_LOCATION) + .migrate(); + + assertThat(result.migrationsExecuted).isEqualTo(12); + } + } + + // -- databases migrated under the 2.3.0 flat layout ------------------------------------------- + + @Test + void databaseWithTheExampleTableFromTheFlatLayoutValidatesAgainstTheSplitLayout() + throws IOException { + Path legacy = flatLegacyLayout(true); + try (HikariDataSource ds = freshDatabase()) { + flyway(ds, "filesystem:" + legacy).migrate(); + assertThat(indexExists(ds, "users_username_lower_key")).isTrue(); + + assertThatCode( + () -> + flyway( + ds, + PkAuthJdbiSchema.CORE_MIGRATION_LOCATION, + PkAuthJdbiSchema.EXAMPLE_USERS_MIGRATION_LOCATION) + .validate()) + .doesNotThrowAnyException(); + assertThatCode(() -> flyway(ds, PkAuthJdbiSchema.MIGRATION_LOCATION).validate()) + .doesNotThrowAnyException(); + assertThat(flyway(ds, PkAuthJdbiSchema.MIGRATION_LOCATION).migrate().migrationsExecuted) + .isZero(); + + // Dropping the example location on such a database is refused rather than silently + // accepted: Flyway reports the applied example migrations as missing. + assertThatThrownBy(() -> flyway(ds, PkAuthJdbiSchema.CORE_MIGRATION_LOCATION).validate()) + .hasMessageContaining("not resolved locally"); + } + } + + /** The reporting host's shape: V5 excluded, so 2.3.0 stopped at V11 and failed on V12. */ + @Test + void databaseThatExcludedTheExampleTableUpgradesWithTheCoreLocationAlone() throws IOException { + Path legacy = flatLegacyLayout(false); + try (HikariDataSource ds = freshDatabase()) { + // On Postgres the failed V12 rolls back with its transaction and leaves no history row, so + // such a database sits at V11. + Flyway.configure() + .dataSource(ds) + .locations("filesystem:" + legacy) + .target("11") + .load() + .migrate(); + assertThat(tableExists(ds, "users")).isFalse(); + + MigrateResult result = flyway(ds, PkAuthJdbiSchema.CORE_MIGRATION_LOCATION).migrate(); + + assertThat(result.success).isTrue(); + assertThat(result.migrationsExecuted).isZero(); + assertThatCode(() -> flyway(ds, PkAuthJdbiSchema.CORE_MIGRATION_LOCATION).validate()) + .doesNotThrowAnyException(); + assertThat(tableExists(ds, "users")).isFalse(); + } + } + + // -- helpers --------------------------------------------------------------------------------- + + /** Configures Flyway as a host would: core-only hosts target the core schema version. */ + private static Flyway flyway(HikariDataSource ds, String... locations) { + String target = + List.of(locations).equals(List.of(PkAuthJdbiSchema.CORE_MIGRATION_LOCATION)) + ? PkAuthJdbiSchema.CORE_SCHEMA_VERSION + : PkAuthJdbiSchema.CURRENT_SCHEMA_VERSION; + return Flyway.configure().dataSource(ds).locations(locations).target(target).load(); + } + + /** + * Rebuilds the 2.3.0 layout, every migration flat in one directory, from the shipped files. + * {@code includeExample = false} reproduces a host that excluded the example migrations. + */ + private Path flatLegacyLayout(boolean includeExample) throws IOException { + Path flat = Files.createDirectories(tempDir.resolve("flat-2.3.0")); + for (String location : List.of("db/migration/core", "db/migration/example-users")) { + try (Stream files = Files.list(classpathDirectory(location))) { + for (Path file : files.toList()) { + String name = file.getFileName().toString(); + if (includeExample || !EXAMPLE_FILES.contains(name)) { + Files.copy(file, flat.resolve(name)); + } + } + } + } + return flat; + } + + private static Path classpathDirectory(String location) { + URL url = PkAuthJdbiSchema.class.getClassLoader().getResource(location); + assertThat(url).as(location).isNotNull(); + try { + return Path.of(url.toURI()); + } catch (URISyntaxException e) { + throw new IllegalStateException(e); + } + } + + private static HikariDataSource freshDatabase() { + String name = "pkauth_loc_" + DATABASE_COUNTER.incrementAndGet(); + HikariConfig admin = config(container.getJdbcUrl()); + try (HikariDataSource adminDs = new HikariDataSource(admin); + var connection = adminDs.getConnection(); + var statement = connection.createStatement()) { + statement.execute("CREATE DATABASE " + name); + } catch (Exception e) { + throw new IllegalStateException(e); + } + String url = container.getJdbcUrl().replace("/pkauth_locations", "/" + name); + return new HikariDataSource(config(url)); + } + + private static HikariConfig config(String jdbcUrl) { + HikariConfig cfg = new HikariConfig(); + cfg.setJdbcUrl(jdbcUrl); + cfg.setUsername(container.getUsername()); + cfg.setPassword(container.getPassword()); + cfg.setMaximumPoolSize(2); + return cfg; + } + + private static boolean tableExists(HikariDataSource ds, String table) { + return queryReturnsRow(ds, "SELECT to_regclass('public." + table + "') IS NOT NULL", true); + } + + private static boolean indexExists(HikariDataSource ds, String index) { + return queryReturnsRow(ds, "SELECT 1 FROM pg_indexes WHERE indexname = '" + index + "'", false); + } + + private static boolean queryReturnsRow(HikariDataSource ds, String sql, boolean readBoolean) { + try (var connection = ds.getConnection(); + var statement = connection.createStatement(); + var rs = statement.executeQuery(sql)) { + if (!rs.next()) { + return false; + } + return !readBoolean || rs.getBoolean(1); + } catch (Exception e) { + throw new IllegalStateException(e); + } + } +} diff --git a/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/V12UsernameCaseMigrationGuardIntegrationTest.java b/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/V12UsernameCaseMigrationGuardIntegrationTest.java index 78d445f..67214d6 100644 --- a/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/V12UsernameCaseMigrationGuardIntegrationTest.java +++ b/pk-auth-persistence-jdbi/src/test/java/com/codeheadsystems/pkauth/persistence/jdbi/V12UsernameCaseMigrationGuardIntegrationTest.java @@ -115,7 +115,7 @@ private static void withPostgres(DataSourceConsumer body) { private static void migrateTo(HikariDataSource ds, String target) { Flyway.configure() .dataSource(ds) - .locations("classpath:db/migration") + .locations(PkAuthJdbiSchema.MIGRATION_LOCATION) .target(target) .load() .migrate();