Skip to content

perf(workspace): one memory budget for the caches of the open documents - #1006

Open
psteinroe wants to merge 3 commits into
perf/cache-statement-walkfrom
perf/document-cache-budget
Open

psteinroe wants to merge 3 commits into
perf/cache-statement-walkfrom
perf/document-cache-budget

Conversation

@psteinroe

@psteinroe psteinroe commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #1005. Part of the rebuild described in #997 (point 5: one memory budget). Replaces #951, #953's budget, #956 and #957.

Each open document keeps the parse result and tree of every statement (#913), its analysis (#1004), and what the walk resolved (#1005). Nothing bounded that across documents: a statement's parse tree takes about 10 KB, so 20 open migration files of 5,000 statements kept 2.7 GB until they were closed. #951 and #953 bounded the document caches and the walk cache with two budgets of their own; here there is one.

What changed

  • document_cache/budget.rs: CacheBudget<K> (from perf(workspace): bound the memory of the document caches #951). record(key, usage) updates the key's usage and last use, and returns the other keys to evict, least recently used first, until the total is within its maximum. The key just used is never evicted, so a single file larger than the budget stays cached while it is used. remove(key) stops tracking it.
  • DocumentCache::usage counts what a document keeps in one unit: its parse results, trees and resolved statements, and CHECKPOINT_ENTRIES per walk checkpoint (a checkpoint copies the maps of the objects the file changed before it). DocumentCache::clear drops all of it.
  • The server charges the document's cache after diagnostics, code actions, completions, hover, inlay hints, semantic tokens and formatting (charge_document_caches), and clears the caches it is told to evict. DOCUMENT_CACHE_ENTRIES_MAX = 20_000 entries, about 200 MB at the measured 10 KB per entry: enough to keep a few files of several thousand statements fully cached. Eviction is logged at debug level, and an evicted document recomputes on its next request, with the same results.
  • Closing a document drops it with its caches and removes it from the budget. The CLI (check, format) closes each file after it is processed, so it holds at most one file per worker.
  • #[doc(hidden)] WorkspaceServer::document_cache_stats reports what a document's cache holds, for the memory tests. Counting doesn't fill the cache.

Tests

  • evicts_the_least_recently_used_keys_first (unit): LRU order, the key in use is never evicted, remove.
  • document_caches_stay_within_the_budget: with a small budget, using a second document drops the caches (parse results and analysis) of the first; using the first again recomputes them with the same diagnostics; closing removes it from the budget.
  • tests/memory_bounds.rs (from perf(workspace): bound the memory of the document caches #951) counts every allocation with a global allocator. It opens N migration files, runs diagnostics, hover, inlay hints and code actions on each, closes them, then runs the CLI path (open, diagnostics, close) on the same files. It asserts that what is retained with every document open stays within what the budget can keep plus the text of the others, that the peak stays within that plus one document, that closing every document leaves less than 4 MiB, and that the CLI's peak is at most twice that of one document. MEMORY_DOCUMENTS and MEMORY_GROUPS change the workload.
  • tests/edit_churn.rs (from test(workspace): the document caches follow a long editing session #956 and test(workspace): check that editing a document doesn't grow the walk cache #957) applies 2,000 seeded random edits to a 1,000-statement document in 500 changes, with hover and inlay hints every 5 changes and diagnostics, code actions and full semantic tokens every 50. After every change the walk keeps at most 4 checkpoints and no more resolved statements than the document has; at the end every statement has exactly one parse result, tree and set of references, and the memory retained after the edits is at most 1.5 times that after the first pass.
  • pgls_workspace passes, and clippy on the whole workspace.

Debug build, the CI defaults:

memory_bounds, 20 documents × 1,000 statements: one document peak 26.6 MiB, retained 26.2 MiB
all documents open peak 260.9 MiB, retained 239.2 MiB
after closing all / CLI retained 0.0 MiB / 0.0 MiB
CLI peak (one file at a time) 22.1 MiB
edit_churn, 1,000 statements, 2,000 edits: retained after the first pass / after the edits 36.0 MiB / 39.1 MiB
at the end 1,205 statements, 1,205 parse results, 1,205 trees, 1,205 resolved statements, 4 checkpoints

The tests take about 30 and 40 seconds in a debug build.

Release build, 20 documents × 5,000 statements (9.6 MiB of SQL), with the whole stack up to #1008. "Before" is #951's measurement of the same test without a budget:

before after
peak with all documents open 2,800 MiB 312 MiB
retained with all documents open 2,668 MiB 176 MiB
retained after closing every document 2.2 MiB 0.0 MiB
one document: peak / retained 148 / 146 MiB
CLI: peak (one file at a time) / retained 265 / 2.2 MiB 138 / 0.0 MiB

Not covered

  • Usage is counted in entries, not bytes. A file of very large statements, or a schema of very wide tables (references copy the objects they resolve to), uses more per entry.

A CacheBudget across documents counts what each document's cache keeps
(parse results, trees, resolved statements and walk checkpoints) after
each request, and drops the caches of the least recently used documents
when all of them keep more than DOCUMENT_CACHE_ENTRIES_MAX. Closing a
document drops its caches. Two tests count the memory: open documents
stay within the budget and nothing is left after closing them, and a
long editing session doesn't grow the caches.
# Conflicts:
#	crates/pgls_workspace/src/workspace/server/document.rs
#	crates/pgls_workspace/src/workspace/server/document_cache.rs
Comment on lines +200 to +209
for evicted in self.document_caches.record(path, doc.cache().usage()) {
if let Some(doc) = documents.get(&evicted) {
tracing::debug!(
"Dropping the caches of {} to keep at most {} cache entries",
evicted.display(),
self.document_caches.max()
);
doc.cache().clear();
}
}

@edjubert edjubert Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also charge the document immediately after parser.find(..., ExecuteStatementMapper) in execute_statement? The lookup caches preceding statements, but this workspace method never records their usage. The standard LSP command normally catches up via update_all_diagnostics; direct workspace callers and errors that return before that refresh do not. Charging here would enforce the budget independently of the caller and cover the early-return paths.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants