Skip to content

test(workspace): check that editing a document doesn't grow the walk cache - #957

Open
psteinroe wants to merge 1 commit into
perf/cancel-editor-requestsfrom
test/editor-walk-edit-churn
Open

psteinroe wants to merge 1 commit into
perf/cancel-editor-requestsfrom
test/editor-walk-edit-churn

Conversation

@psteinroe

Copy link
Copy Markdown
Collaborator

Stacked on #954.

The memory test from #953 covers open → requests → close, but not a long editing session. That is where a leak in the walk cache would show: checkpoints and references of earlier versions, or document snapshots kept alive by outdated requests.

The test (crates/pgls_workspace/tests/memory_churn_tests.rs)

  • Opens a document of 1,000 statements: tables, SQL functions with their bodies, and queries.
  • Applies 1,000 change_file edits by default (MEMORY_EDITS changes this). The edits come from a seeded random generator, at random positions:
    • rewrite a query
    • insert a table with its function and query
    • delete statements
    • alter table ... add column
    • drop table
    • rewrite a function body
      Deletes keep the document at about 1,000 top-level statements.
  • Every 10 edits: semantic tokens and inlay hints for a random range. Every 100: also full semantic tokens and a hover at a random position. Every 100th edit runs a full token request on another thread while the edit is applied, so the request ends up working on an outdated version.
  • After every request round, the walk cache must stay within its bounds: at most 16 checkpoints, and cached statements at most 2 × the document's statements + 1,024, after which stale ones are pruned.
  • At the end, after a full walk, the cached statements of the current document must equal its statements.
  • Retained memory (over a baseline after a warmup) after the edits must stay below 3× the retained memory after the first pass.

It uses the counting allocator of #953, which moves to tests/common/counting_allocator.rs. Each memory test has its own test binary, because the counter is global per binary. WorkspaceServer::editor_walk_size (#[doc(hidden)]) reports a document's cached statements, the ones that belong to its current version, and its checkpoints.

Results (debug build)

edits document after the edits retained after the first pass retained after the edits walk cache, largest during the edits walk cache after the edits time
1,000 1,000 → 1,288 statements 27.1 MiB 24.5 MiB 2,949 statements, 15 checkpoints 1,525 statements (all 1,288 current cached), 15 checkpoints 59 s
2,000 1,000 → 1,296 statements 27.1 MiB 23.7 MiB 2,970 statements, 15 checkpoints 1,296 statements (all current), 15 checkpoints 118 s

No leak found, so there is no fix in this PR. Stale references are pruned once there are about twice as many as the document's statements, checkpoints stay at the per-document maximum, and outdated requests don't keep document snapshots alive. Retained memory even drops slightly. It is dominated by the document's statement caches (AST/CST), which depend on the statement texts. The default of 1,000 edits keeps the test at about a minute in debug: each change_file re-splits the whole document, about 30 ms per edit in a debug build.

…cache

A thousand edits of a 1,000-statement document that rewrite, insert and
delete statements at random positions, alter and drop tables and change
function bodies, with semantic tokens, inlay hints and hover in between
and requests that edits make outdated while they run. The walk cache
stays within its bounds and holds exactly the current statements after
a full walk, and the retained memory doesn't grow with the edits. The
counting allocator moves to a module that both memory tests share.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant