parser: record scanned comments and stamp table/column positions

[kyleconroy]

This gives sqlc's MySQL engine what its ParseFile needs to power sqlc fmt: statements and comments from one lexer pass, with enough positions to anchor comments and the author's line breaks.

Comment recording

The lexer now records every comment it skips — -- and # line comments (span excludes the terminating newline) and /* */ block comments (span includes the closing */) — as byte-offset spans on the Scanner, exposed through Scanner.Comments() and a new Parser.Comments() that returns the comments from the most recent parse.

• Executable comments (/*! ... */, recognized /*T[...] ... */) and optimizer hints (/*+ ... */) are lexed as SQL, so they are not recorded; a misplaced hint, which MySQL treats as a plain comment, is.
• The scanner's lookaheads (getNextToken and friends) save and restore the reader, so the same comment can be scanned more than once; recording is gated on strictly increasing positions to keep one entry per comment.
• The slice is reset per parse and reused, so parses without comments allocate nothing.

Position stamping

OriginTextPosition is now stamped on FROM-clause table factors, INSERT/REPLACE table references, and INSERT column lists, which goyacc left unstamped (it stamped only <expr> symbols). Consumers placing comments or preserving author line breaks need these positions, and 0 (the unstamped value) is indistinguishable from the start of the input.

Only Accept-reachable nodes are stamped, so the restore round-trip tests can keep resetting positions; parseTableName and parseColumnNameList themselves stay unstamped because other call sites hang their nodes off structures Accept skips (ALTER…UNION options, FOR UPDATE OF lists, procedure names).

Both changes are additive: no existing node, field, or Restore output changes, and stamping respects SkipPositionRecording.

Testing

New parser/comments_test.go covers every comment syntax, the lookahead-dedup case, the not-a-comment cases (--x, executable comments, hints), and per-parse reset. go test ./... -count=1 -timeout 120s is green, and the sqlc side (sqlc-dev/sqlc branch claude/parsefile-sqlc-fmt-marino-x6v056) passes sqlc's full --tags=examples suite against this commit.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C48orUYrL7xE989UGLTVqi

---

Generated by Claude Code