Conversation
PR #33 added memory_limit, cpu_weight, max_active_queries, max_queued_queries, queue_timeout, mapping, priority, resource and unset as tokens without adding them to IDENTIFIER_KEYWORD_NAMES, so they could no longer be used as column names. memory_limit is a column of query_activity(), which the Web Console queries. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…th 50 - Indent the query one level after AS in CREATE statements and after the INSERT target. - Open a block for a parenthesis that follows FROM, JOIN, a comma in the FROM list, or AS in a WITH item or CREATE, so implicit-select subqueries break into lines; a single plain item stays inline. - Drop gaps at the start of a line in the printer, which keeps blocks that begin with a comment or unknown text idempotent. - DECLARE with two or more variables always breaks, one per line. - ATTACH/DETACH/DROP/CONVERT PARTITION LIST keep LIST in the clause head. - Default maxLineWidth is 50; fixtures pin width 80 unless they set it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two changes that both come from the same idea: the grammar is the source
of truth, so the formatter should read it rather than repeat it.
Grammar drift test. The phrase table that drives layout is hand-written
while QuestDB syntax arrives through the parser, so the two drift apart
silently and an unknown clause stays inline instead of starting a line.
tests/formatter/grammar-drift.test.ts reads the grammar through
Chevrotain and fails when a statement rule can start a part with a
keyword sequence the formatter neither breaks on nor lists as inline by
design, and when a phrase in the table no longer exists in the grammar.
It found six gaps, fixed here with fixtures:
- LATEST BY, the legacy form of LATEST ON, was not a clause
- UPDATE ... FROM t JOIN u did not break its joins
- ALTER TABLE SUSPEND WAL, REBASE WAL and the storage policy actions
stayed inline while ADD COLUMN started a line
- ALTER MATERIALIZED VIEW DROP EXPIRE stayed inline
- EXPIRE ROWS was not a CREATE option
- UNPIVOT was in the table but never in the grammar
Capitalize option. format(sql, { capitalize: true }) uppercases the
keywords. Casing cannot be decided from the word alone, since QuestDB
has about 60 reserved keywords and 350 non-reserved ones, and the
non-reserved group holds most type and option words, which are also the
most common column names. So it parses the statement and reads the
answer off the tree: a word the grammar consumed through its identifier
rule names a table, view or column and keeps its case, every other
keyword is raised, and SQL that does not parse keeps the case it was
written in. Layout stays the same either way and needs no lists.
The corpus test formats every parseable query this way and requires that
the words the grammar read as names come back with their exact case.
maxLineWidth is renamed to maxWidth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 5,000-line budget failed on CI at 98 ms against 80 ms, while the same script measures 22 ms locally and 23 ms on the commit before the formatter changes, so nothing had slowed down. The budget was four times a local median taken before the corpus grew, and this file runs beside suites that saturate a shared runner. Assert how the cost grows instead, which no machine speed can move: ten times the script must cost less than twenty five times the work, and four times the nesting less than twelve times. Measured today those are 7.5 to 9.4 times and 2.4 times. A loose ceiling still catches a hang. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Formatter
Adds
format(sql, options)— a token-based SQL formatter, exported from the root and from@questdb/sql-parser/formatter(lexer only, no parser in the bundle).SELECT,INSERT,UPDATE,CREATE TABLE/MATERIALIZED VIEW/LIVE VIEW,ALTER TABLE/MATERIALIZED VIEW). Unknown statements get normalized spacing and no line breaks. One width rule (maxLineWidth, default 80): lists, predicates,CASE,IN/VALUESlists,OVER (...), named windows, join sub-clauses andPIVOT (...)stay inline when they fit, otherwise break one item per line.indent(default two spaces),maxLineWidth.Playground: https://questdb-sql-formatter-playground.netlify.app/
Parser
Gaps found by running the current documentation through the parser and the round-trip test:
ALTER USER | GROUP | SERVICE ACCOUNT … SET MEMORY LIMIT <size> | UNLIMITEDRIGHT [OUTER] JOIN,FULL [OUTER] JOINUNNEST(...)after a join, e.g.CROSS JOIN UNNEST(t.asks[2]) u(vol)FILL(PREV(other_column), PREV)CONVERT PARTITION TO PARQUET … WITH (bloom_filter_columns = …, bloom_filter_fpp = …)CREATE LIVE VIEW … OWNED BY ownerFROM t alias TIMESTAMP(col)(alias before the designation, as the server parses it)COPY … TO … WITH … BLOOM_FILTER_COLUMNS … BLOOM_FILTER_FPP …ALTER MATERIALIZED VIEW … ADD INDEX TYPE POSTING [DELTA | EF] [INCLUDE (…)] [CAPACITY n]Serializer:
CREATE LIVE VIEW … AS SELECTno longer gains parentheses on output;CREATE TABLEwritesFORMATafterWAL.Corpus
docs-queries.jsongrows from 1,797 to 2,637 queries: everyquestdb-sqlblock in the documentation that was not yet covered (syntax templates, fragments and intentionally invalid examples excluded), plus the queries from the 0.1.16 verification set. All pass parse, round-trip, autocomplete walkthrough and the formatter oracles.No version bump.