[psteinroe]

Stacked on #929.

#913 made each document keep the parse result and tree of every one of its statements, so that walking a big file doesn't parse it again. Nothing bounded that across documents. A statement's parse tree takes about 10 KB, so an editor with 20 migration files of 5,000 statements each open kept 2.7 GB, until the files were closed.

What the caches are

Cache	What it stores	Key	Grows with	Dropped when
PgQueryStore::ast_db (#913)	parse result of each statement and SQL function body	statement text	statements of the document	statement edited away; document closed; document evicted by the budget
PgQueryStore::plpgsql_db (#913)	PL/pgSQL parse result of each CREATE FUNCTION ... plpgsql	statement text	PL/pgSQL functions of the document	same as above
TreeSitterStore (#913)	tree-sitter tree of each statement	statement text	statements of the document	same as above
AnnotationStore (#913)	whether a statement ends with ;	statement text	statements of the document (unused today)	same as above
Document::analysis (#929)	the lint/typecheck analysis of the document	settings version + database snapshot	diagnostics and statements of the document	content changed; document closed; document evicted by the budget
snapshot_before (#913)	one database snapshot with the file's DDL before the hovered or completed statement	text before the statement + database snapshot	one entry	settings changed; any document closed
catalog_base (main)	the indexed database snapshot	database snapshot	one entry	database snapshot replaced

Measured with the new test (release build), the statement caches are almost all of it: about 21 KB per statement with its parse result and tree, against 0.5 MB for the whole analysis of a 5,000-statement file.

Bounds

• A budget across documents. DOCUMENT_CACHE_ENTRIES_MAX = 20_000 cached parse results and trees, about 200 MB at the measured 10 KB per entry. That is enough to keep a few files of several thousand statements fully cached, so switching between them doesn't parse them again. When the open documents keep more, the caches of the least recently used documents are dropped. They are recomputed on the next request for that document, so results stay the same. Dropping is logged at debug level. The document of the current request is never evicted, so a single file larger than the budget stays fully cached while it is in use.
• Closing a document drops its document with all its caches, removes it from the budget, and drops the hover snapshot. That snapshot is a single entry and may have been built for the closed document; dropping it is simpler than tracking which document it belongs to.
• The CLI (check, format) opens each file through a FileGuard, which closes it once the file is processed. At any time it holds at most one file per rayon worker, and nothing is retained after a file. The test checks this path.

The mechanism, for the other caches (lane G's walk cache)

server/cache_budget.rs has a small generic CacheBudget<K>, kept separate from the documents:

let budget = CacheBudget::new(MAX);                        // in a unit of your choice
for key in budget.record(&key, usage_after_this_request) { // after using the cache of `key`
    drop_the_caches_of(key);                               // least recently used first
}
budget.remove(&key);                                       // when `key` goes away (didClose)

• record updates key's usage and last use. It returns the other keys to evict, oldest first, until the total is within max, and stops tracking them.
• The caller owns the caches and drops them. For documents that is Document::clear_caches(), under the documents read lock that the request already holds. The stores use interior mutability, and a request that is iterating an evicted document keeps its Arcs and simply recomputes on a miss.
• The workspace calls it once per request (charge_document_caches) at the end of diagnostics, hover, inlay hints, code actions, completions and formatting.

A cache with checkpoints (Catalog/Session) can count checkpoints or approximate bytes as its usage and use its own CacheBudget with its own *_MAX.

Measurements

tests/memory_bounds.rs counts every allocation with a global allocator. It opens N documents (migration files: a table, an index, a SQL function and a query per group), and on each runs diagnostics, hover and code actions at the end of the file and inlay hints over the last 4 KB. Then it closes them all, and runs the CLI path (open, diagnostics, close) on the same files. Release build, 20 documents × 5,000 statements (9.6 MiB of SQL):

	before (#929)	after
peak with all documents open	2,800 MiB	426 MiB
retained with all documents open	2,668 MiB	162 MiB
retained after closing every document	2.2 MiB	0.0 MiB
CLI: peak (one file at a time)	265 MiB	265 MiB
CLI: retained afterwards	2.2 MiB	0.0 MiB

The test runs in CI with 20 × 1,000 statements (debug build, 35 s), so the budget evicts there too. It asserts:

• what is retained with every document open stays within what the budget can keep (the documents that fit in DOCUMENT_CACHE_ENTRIES_MAX, plus one, plus the text of the others);
• the peak stays within that plus one document's peak;
• closing every document leaves less than 4 MiB;
• the CLI's peak is at most twice that of one document;
• the CLI retains less than 4 MiB.

MEMORY_DOCUMENTS and MEMORY_GROUPS change the workload.

The CLI's peak is what analysing one 5,000-statement file takes (its parse trees plus the analyser's copies of them). It is unchanged here. check runs one file per rayon worker, so its peak is that times the number of workers.

Tests

• cache_budget::tests::evicts_the_least_recently_used_keys_first: LRU order, the key in use is never evicted, remove.
• document_caches_stay_within_the_budget: with a small budget, using a second document drops the caches (statements and analysis) of the first. Using the first again recomputes them with the same diagnostics, and closing it removes it from the budget.
• open_documents_stay_within_the_budget: the measurement above.

Not covered

• The analyser deep-clones each statement's AST while it runs, which is most of the 265 MiB transient peak per 5,000-statement file. Borrowing them (R2) would lower it.
• The usage is counted in entries, not bytes. A file of very large statements uses more memory per entry. Counting bytes would need an estimate of the parse tree's size.