Repository navigation
Conversation
dbmate sends a transaction:false section to Postgres as one query, which runs in an implicit transaction block if it has more than one statement. Model that, skip the CONCURRENTLY fixes in function bodies, on partitioned tables and on system catalogs, count statements that don't parse, and make the fix unsafe where the transaction mode is unknown. banConcurrentIndexCreationInTransaction now also covers DROP INDEX, REINDEX and DETACH PARTITION with CONCURRENTLY, and CONCURRENTLY in function bodies.
Token helpers skip comments and keep them in replaced ranges, the analyser drops a fix whose statement no longer parses, the type rules only match built-in types, and inserted keywords and swapped types follow the case of the statement. The spec harness checks that applying the fixes converges, and fix_file warns when it hits its pass limit.
Diagnostics with a fix are tagged FIXABLE, the summary says how many have a fix or how many unsafe fixes --fix skipped, and only says 'No fixes applied.' when the command writes files. check gets --only and --skip, which also limit the fixes. check with --stdin-file-path now lints the input and prints it with the fixes of --fix applied.
- banCharField checks ALTER COLUMN TYPE like the other column type rules. - preferBigInt, preferIdentity and changingColumnType only match built-in type names, not a schema or type of the user with the same name. - requireConcurrentIndexCreation and disallowUniqueConstraint only skip tables created earlier in the transaction, in the same schema. - lockTimeoutWarning and multipleAlterTable resolve unqualified tables through the search path instead of assuming public.
# Conflicts: # crates/pgls_analyser/src/fix.rs # crates/pgls_analyser/src/lib.rs # crates/pgls_analyser/src/lint/safety/require_concurrent_index_creation.rs # crates/pgls_analyser/src/linter_context.rs # crates/pgls_analyser/tests/rules_tests.rs # crates/pgls_cli/src/commands/mod.rs # crates/pgls_cli/src/execute/stdin.rs # crates/pgls_workspace/src/workspace/server.rs # crates/pgls_workspace/src/workspace/server.tests.rs # crates/pgls_workspace/src/workspace/server/migration.rs # docs/features/linting.md # docs/guides/checking_migrations.md # docs/reference/rules.md
There was a problem hiding this comment.
replace_keeping_comments copies -- comments without preserving their terminating newline. This can silently comment out constraints following the replaced type.
For example, --fix --unsafe --only=preferTextField transforms:
CREATE TABLE t (name varchar -- length
(10) NOT NULL
);into:
CREATE TABLE t (name text -- length NOT NULL
);The result still parses, so keeps_statement_valid accepts it, but the column becomes nullable. The same issue affects preferTimestamptz.
replace_keeping_comments copied a -- comment from the replaced type without the line break that ends it, so the comment swallowed the rest of the line. preferTextField turned name varchar -- length (10) NOT NULL into name text -- length NOT NULL, which still parses but drops NOT NULL; preferTimestamptz did the same with WITHOUT TIME ZONE.
| if ctx | ||
| .session() | ||
| .has_created_object(&relation.schemaname, &relation.relname) | ||
| { |
There was a problem hiding this comment.
Switching these rules to Session::has_created_object introduces case-insensitive comparisons through eq_ignore_ascii_case. PostgreSQL already folds unquoted identifiers during parsing, so the resulting names must be compared exactly to preserve quoted identifiers.
Assuming public.users already exists:
CREATE TABLE "Users" (id bigint);
CREATE INDEX users_id_idx ON users (id);
ALTER TABLE users ADD CONSTRAINT users_id_key UNIQUE (id);requireConcurrentIndexCreation and disallowUniqueConstraint now emit no diagnostics, although the operations target the existing table users, not the newly created "Users"
There was a problem hiding this comment.
I rechecked the latest changes, and this issue is still reproducible. Both rules still suppress the expected diagnostics when "Users" is created before an operation targeting the distinct table users. The same problem occurs with quoted schema names such as "App" versus app.
| let result = guard.pull_file_diagnostics( | ||
| RuleCategoriesBuilder::default().all().build(), | ||
| config.max_diagnostics(), | ||
| only, | ||
| skip, | ||
| )?; |
There was a problem hiding this comment.
The new stdin check bypasses check_database_connection, which the file-based path calls before running checks.
With an explicitly configured but unreachable database, checking this input through --stdin-file-path exits with code 0 and no diagnostic:
SELECT nonexistent_column FROM users;Checking the same SQL from a file reports database/connection and exits with code 1.
The stdin path silently skips database-dependent checks, making an incomplete check appear successful.
| console.append(markup! {{content}}); | ||
|
|
||
| let file_name = payload.path.display().to_string(); | ||
| let mut has_errors = false; |
There was a problem hiding this comment.
The new stdin path determines failure solely from diagnostics with severity Error and does not apply cli_options.error_on_warnings.
For example:
printf 'CREATE TABLE t (data json);\n' |
postgres-language-server check \
--disable-db \
--only=preferJsonb \
--error-on-warnings \
--stdin-file-path=virtual.sqlThis prints a preferJsonb warning but exits with code 0. Checking the same SQL from a file with the same options exits with code 1.
The parser already folds unquoted names to lower case, so the session's created objects and NOT VALID constraints must be compared exactly. With a case-insensitive comparison, CREATE TABLE "Users" made an index or a unique constraint on the existing table users look like one on a new table, and requireConcurrentIndexCreation and disallowUniqueConstraint reported nothing. The same happened with quoted schemas like "App".
The parser already folds unquoted names to lower case, so the session's created relations and NOT VALID constraints must be compared exactly. With a case-insensitive comparison, CREATE TABLE "Users" made the existing table users look new, and the same with quoted schemas like "App". #970 fixes the same comparison on its branch.
…mment A fix that keeps a -- comment inside the replaced text ended it with \n, so a fix in a file with CRLF line endings left a mixed line ending.
…in-file-path (#1082) `check --stdin-file-path` (added in #916) handled two things differently from checking files: - It never called `check_database_connection`. With a configured database that can't be reached, the checks that need it were skipped silently and the command exited with 0. Checking the same SQL from a file reports `database/connection` and exits with 1. - It failed only on diagnostics with severity `Error` and ignored `--error-on-warnings`. `printf 'CREATE TABLE t (data json);\n' | postgres-language-server check --disable-db --only=preferJsonb --error-on-warnings --stdin-file-path=virtual.sql` printed the `preferJsonb` warning and exited with 0. Both were reported in the review of #970, whose branch contained #916 at the time. ### What changed - `check` checks the database connection before it branches into stdin or the traversal. With stdin, the `database/connection` diagnostic is printed to stderr next to the configuration errors, and the command fails after printing the checked code. - `ExecutionConfig` carries `error_on_warnings`, set from the CLI option by `check`. The stdin check fails on warnings when it is set. The traversal still applies the option to its report in `enforce_exit_codes`, as before. ### Tests - `check_stdin_fails_when_database_is_unreachable`: nothing listens on port 1, so stdin fails with `database/connection`. Before: `status: success`. - `check_stdin_fails_on_warnings_with_error_on_warnings`: the `preferJsonb` warning succeeds without the flag and fails with it. Before: `status: success` with the flag. - `cargo test -p pgls_cli`, `cargo clippy --workspace --all-targets -- -D warnings`. ### Not covered With stdin, the errors of nested configuration files are still only printed, while checking files adds them to the report and its exit code. #971 moves the exit-code handling of the commands into one place, and this PR will conflict with it in `commands/check.rs`.
Stacked on #916.
A handful of lint rules matched the AST too loosely or resolved names on their own, which gave false positives and false negatives on ordinary migrations.
Column types
banCharFieldonly checkedADD COLUMN, soalter table users alter column code type char(10);went unreported. It now checksALTER COLUMN ... TYPElikepreferJsonb,preferTextFieldandpreferTimestamptz, including thevarcharfix.preferBigIntcompared every part of the type name, lowercased, somyschema.int4,int4.counterand the quoted"integer"were flagged. It now uses thebuiltin_type_namehelper that the other column type rules already use: the last name, unqualified or inpg_catalog, compared exactly (the parser already lowercases unquoted names and turnsinteger/smallintintoint4/int2).preferIdentityflaggedmyschema.serialandserial.counter. Like Postgres'transformColumnDefinition, only an unqualified name (and not%TYPE) is a serial type now.changingColumnTypetreated any type whose last name wastextas a safe change, e.g.type myschema.text, and alsotype text[]. Both are flagged now.Tables created in the migration
requireConcurrentIndexCreationskipped an index when aCREATE TABLEwith the same name was anywhere in the file, also after the index and in another schema:disallowUniqueConstraintonly looked at earlier statements, but also ignored the schema. Both now ask the session (Session::has_created_object), likelockTimeoutWarningdoes, so only tables created earlier in the transaction count, in the schema the search path puts them in.CREATE TABLE AScounts too now.Search path
lockTimeoutWarningandmultipleAlterTabletreated an unqualified table aspublic.<name>. Withset search_path = app,create table users (...); alter table users ...reported a missing lock timeout for a table the migration had just created, andalter table authors ...; alter table app.authors ...wasn't reported as twoALTER TABLEs on the same table. The session now resolves the schema (Session::object_schema), and it records the tables thatALTER TABLEchanged with the search path of that statement (Session::has_altered_table), soalter table authors ...; set search_path = app; alter table authors ...is no longer reported. Messages name the resolved schema, e.g.app.orders.Dropped
requireConcurrentReindexalready usesis_reindex_concurrentfor the check on this base, so(concurrently false)is reported. The remaining check of the option in the fix is on purpose: an explicitCONCURRENTLYoption in parentheses is left alone.Tests
New spec files for each case above (
banCharField/alter_column_type,preferBigInt/type_forms,preferIdentity/qualified_type_valid,changingColumnType/user_type,requireConcurrentIndexCreation/{table_created_later,table_in_other_schema,new_table_on_search_path}, a newdisallowUniqueConstraintspec directory,lockTimeoutWarning/search_path,multipleAlterTable/{search_path,search_path_changed_valid}) and a session unit test. No existing snapshot changed.Risks
requireConcurrentIndexCreationanddisallowUniqueConstraintnow report a table created before aCOMMITin the same file, since the session forgets created tables at the end of a transaction. This matcheslockTimeoutWarningand Squawk. Unqualified names resolve to the first schema of the search path, where Postgres creates them; anALTER TABLEof a table that exists in a later schema of the path is still attributed to the first one.Update: #916 is merged, so main is merged in (merge commit, no rebase) and the diff now only contains this PR's own changes. Nothing here duplicated the catalog lookups that #1000 moves into
pgls_catalog: the checks for tables created or altered earlier are transaction state, so they stay in the session. #1000 leavesrequireConcurrentIndexCreation,disallowUniqueConstraint,lockTimeoutWarningandmultipleAlterTableto this PR. Both PRs touchrequireConcurrentIndexCreation, in different hunks (this one changes the created-table check, #1000 changes the partitioned-table check in the fix).replace_keeping_comments(used by the type fixes ofpreferTextField,preferTimestamptzandbanCharField) now keeps the line break after a--comment from the replaced type. Before,name varchar -- length+ newline +(10) NOT NULLbecamename text -- length NOT NULL, which parses but dropsNOT NULL. New specsline_comment_in_type.sqlfor the three rules.Update:
Session::has_created_objectandhas_not_valid_constraintcompare names exactly. The parser already folds unquoted names, so with the case-insensitive comparisonCREATE TABLE "Users"madeCREATE INDEX ... ON usersandALTER TABLE users ADD ... UNIQUElook like they target a new table, and the same with quoted schemas ("App"vsapp). New specsquoted_table_name.sqlforrequireConcurrentIndexCreationanddisallowUniqueConstraint, and the session testnames_are_compared_exactly. #1042 gets the same fix. The--stdin-file-pathexit-code issues from the review are in code from #916 on main and are fixed in #1082.