Skip to content

perf(workspace): bound the memory of the editor walk cache - #953

Open
psteinroe wants to merge 1 commit into
perf/editor-requests-off-runtimefrom
perf/bound-walk-cache-memory
Open

psteinroe wants to merge 1 commit into
perf/editor-requests-off-runtimefrom
perf/bound-walk-cache-memory

Conversation

@psteinroe

Copy link
Copy Markdown
Collaborator

Stacked on #931.

The walk cache from #930 had no memory bound. Every checkpoint deep-copied the catalog's objects, a document could keep up to 32 checkpoints, and nothing limited the cached references across documents. With 20 open migration files of 5,000 statements each, hover, inlay hints and semantic tokens kept 734 MiB.

Caches this stack owns

Cache Stores Key Grows Dropped
Walk checkpoints (#930) Catalog (maps of the objects the file changed; the objects are now shared through Arc), Session, section state top-level statement index + hash of the statements before it per document, every max(64, statements / 16) top-level statements on close, schema invalidation, search path change, or when the document is evicted; stale ones are pruned. ≤ 16 per document, ≤ 256 across documents
Walk references (#930) each requested statement's references, offsets, function body ids hash of the statements before it + statement id per requested statement same as above; stale entries are pruned past 2 × statements + 1024. ≤ 50,000 statements across documents
Walk snapshot (#930) Arc<Snapshot> the walk started from – one per document with the walk
Document snapshots (#931) Arc<Document> held by a running request – one per in-flight request; an edit during a request clones the document (text, ranges; statement caches are shared) when the request ends; an outdated walk stops within 64 statements

The CLI doesn't use any of these. check and format never walk for editor features, and FileGuard closes each file after it is processed.

What changed

  • Catalog's Entry::Defined holds an Arc. Clones of the catalog (checkpoints, and savepoints, which clone it too) share the objects, and a change copies only the changed object.
  • CHECKPOINTS_PER_DOCUMENT_MAX = 16: the checkpoint interval grows with the file. When the interval changes and there are too many checkpoints, the ones that no longer fall on it are dropped first, then the earliest.
  • CACHED_STATEMENTS_MAX = 50_000 and CHECKPOINTS_MAX = 256 across documents (in EditorWalks). After each walk, the caches of the least recently used other documents are dropped until all caches are within budget. Caches that another request is using are skipped. A single document over the statement budget drops its own references. Each eviction is logged at debug level, and the next request recomputes the walk, so results don't change.
  • Closing a document and invalidating the schema drop the walks as before, now through EditorWalks::remove/clear.

Measurements

crates/pgls_workspace/tests/memory_tests.rs uses a counting #[global_allocator]. It opens N documents of M statements (migration-style: table, index, SQL function with body, query), and on each runs diagnostics, hover at the end, inlay hints and semantic tokens for the whole file, and code actions. It reports the peak and the memory with all documents open over a baseline taken after a warmup, plus the editor features' share (what hover, hints, tokens and code actions keep on top of what diagnostics keep), and the memory left after closing them all. It asserts editor features < 128 MiB, peak < 1 GiB and retained < 16 MiB. The default is 20 × 1,000, which takes about 30 s in debug; MEMORY_DOCUMENTS/MEMORY_STATEMENTS change it.

20 × 1,000 (debug) before after 20 × 5,000 (release) before after
editor features 71.9 MiB 41.0 MiB 734.0 MiB 107.0 MiB
all documents open 499.3 MiB 468.3 MiB 1,166.0 MiB 538.9 MiB
peak 520.2 MiB 489.2 MiB 1,273.0 MiB 645.6 MiB
after closing all 3.8 MiB 3.8 MiB 5.9 MiB 5.9 MiB

The rest of the memory with all documents open is the per-document statement caches (AST, CST, annotations): about 21 MiB per 1,000-statement document on this branch. Lane A bounds those across documents on main (#913/#929). What remains after closing is the one-entry hover snapshot cache (snapshot_before), which #913 also replaces. When the stacks meet, the walk cache and those caches should use one budget mechanism (one LRU of documents, one notion of cost) instead of two.

Tests

  • test_editor_walk_cache_budget: three 3,000-statement documents keep at most 16 checkpoints each; a budget a bit over one document keeps only the most recently used cache; evicted documents give the same tokens on the next request; closing drops everything.
  • memory_tests as above. Catalog tests, the Postgres regression suite and the analyser specs pass with the shared entries.

Not covered

  • The budget counts statements and checkpoints, not bytes. A statement's references copy the objects they resolve to, columns included, so a schema with very wide tables costs more per statement than the 1.75 KiB measured here.

Checkpoints share the objects of the catalog instead of copying them, a
document keeps at most 16 checkpoints, and the walks of all documents
together keep at most 50,000 statements and 256 checkpoints: beyond
that, the caches of the least recently used documents are dropped and
recomputed on the next request. A test with a counting allocator checks
the memory of the editor features on many large documents.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant