Skip to content

Vacuum the write-loop timing test's tables without parallel workers, so a pooled session's shared memory doesn't starve a later vector-index build - #436

Merged
jasperblues merged 1 commit into
mainfrom
fix-shm-main
Nov 6, 2026
Merged

jasperblues merged 1 commit into
mainfrom
fix-shm-main

Conversation

@jasperblues

Copy link
Copy Markdown
Member

This fixes main, which is red on Python 3.14 since #428's merge (b33e2fd), in CI run 42118394771. The sync vector-index tests fail there with DiskFull: could not resize shared memory segment … to 63999616 bytes: No space left on device.

Cause

tests/*/sql/test_compiled_write_loop_time.py leaves pooled Postgres sessions holding /dev/shm. The first time a backend runs anything in parallel, it keeps a dynamic shared memory segment of about 1.2 MB until it disconnects, and the pool keeps backends connected. A parallel HNSW build asks for 63,999,616 bytes, which leaves about 3.1 MB of the 64 MB free. So the build fits while one session holds a segment, and fails once two do.

The test made two sessions hold one:

  1. The VACUUM before the MERGE case. The async run's 20k MERGE grows the shared edge indexes past min_parallel_index_scan_size, and its final DELETE leaves 20,000 dead edges. The sync twin's VACUUM ANALYZE drivine_edge then vacuums those indexes in parallel. Across that statement, /dev/shm went from 1,075,552 to 2,320,864 bytes, and the pooled backend that ran it held the segment through the sync schema tests.
  2. The emptied node table. The test left drivine_node with about 8 MB of dead rows, which is over min_parallel_table_scan_size. A later range-index build on it ran in parallel, and that build's backend held a second segment. The same table size is what makes the HNSW build parallel at all.

These were found by listing /dev/shm around each test with pg_ls_dir and pg_stat_file, and mapping each segment to its backend through /proc/<pid>/maps.

Whether the failure shows depends on autovacuum clearing the dead rows in time, which is why only some runs fail.

Fix

The fix is in the tests only:

With both, no backend held an extra segment at any point in a full run. Before every sync test_schema test, /dev/shm held only Postgres's own two segments. No shm size was raised, and nothing was skipped or marked xfail.

Follow-ups (next train)

Gate

  • ruff check, ruff format --check, pyright and unasync --check.
  • The suite on Python 3.14, through the test lock.
  • One review pass, which corrected one comment.

…so a pooled session's shared memory doesn't starve a later vector-index build

test_compiled_write_loop_time ran VACUUM ANALYZE drivine_edge over 20k
dead edges, which Postgres did in parallel, and left drivine_node at some
8 MB of dead rows, so a later index on it was built in parallel. Each
pooled session that ran something in parallel kept a dynamic shared memory
segment until it ended, and with two of them a parallel HNSW build couldn't
get its 64 MB of a 64 MB /dev/shm: main failed on Python 3.14. The VACUUM
now runs with PARALLEL 0, and the test vacuums its emptied tables after it.
@jasperblues
jasperblues merged commit 86e4724 into main Nov 6, 2026
7 checks passed
@jasperblues
jasperblues deleted the fix-shm-main branch November 6, 2026 00:29
@jasperblues jasperblues mentioned this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant