Repository navigation
Conversation
After a configuration changes or a project is removed, drop the pools, failures and schema snapshots of the connections no settings use any more. Close pools idle for five minutes (cache hits now count as a use), let sqlx close idle connections after the same time, and drop snapshots unused for 30 minutes. Add a test that measures snapshot sizes.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #936 → #921 → … → #881. Must merge after #881.
With nested configurations every database a configuration points at gets a connection pool and a schema snapshot, and nothing ever dropped them. Editing a nested config to point at another database, renaming a folder, or removing a workspace folder left the old pool (with its open connections) and its snapshot in memory until the server stopped. The existing idle cleanup of pools only ran when a new pool was created, and a cache hit didn't count as a use, so a pool in constant use could be evicted while an idle one stayed.
How big is a snapshot
crates/pgls_workspace/tests/snapshot_memory.rs(ignored, needs the test DB) counts the bytes a loadedSnapshotkeeps allocated, with a counting global allocator:postgres(mostly the system catalog)So each database costs about 3 MiB before it has any user objects, plus a few KiB per table. That's fine for a handful of databases, but it adds up when stale ones are never dropped.
Caches and their bounds
ConnectionManager.poolsupdate_settings,update_nested_settings,unregister_project_folder); unused for 5 min (fixed: hits count, the check runs at most once a minute on any use); settings fingerprint changesidle_timeout, was sqlx's 10 min default)ConnectionManager.failuresSnapshotManager.db_schemasinvalidate_schema_cachecatalog_base,snapshot_beforeroot_files,configuration_files,real_foldersEvictions are logged at debug level; a dropped pool or snapshot is created again on the next use, so results don't change.
LastUsed(an atomic timestamp) lets a cache hit under the read lock record a use.Tests
connection_manager:unused_connections_are_dropped,idle_pools_are_closed(incl. the once-a-minute check).snapshot_manager:unused_snapshots_are_dropped,idle_snapshots_are_dropped.pgls_workspace:the_connection_of_a_removed_configuration_is_dropped, against the test DB. A nested config that connects keeps one pool and one snapshot; moving it to another folder with the same database keeps them; removing it drops both.Not covered
retaincall indrop_unused_connections.