Don't close relation files when the visibility map is extended
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t254404psql -h localhost -U postgresBuilt from patchset v1 (message #1), October 26, 2026 at 10:17 PM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t254404_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t254404_1 && git checkout t254404_1Patchset v1 (message #1) is on t254404_1
Greetings,
When vm_extend() grows the visibility map it sends an smgr
invalidation. Every backend that has the relation open then closes
all of its files, and reopens each segment the next time it touches
the table. VACUUM extends the map about once per 255MB of heap. On a
multi-TB table with a few hundred backends inserting, that is a lot of
opens, and as far as I can tell none of them are needed. The other
backends only have to learn that the map got bigger.
This assumes open() is cheap, and it isn't always. I ran into it on
17 with a ~4TB partition, on a system where open() goes through a
filesystem encryption layer. Each time VACUUM extended the map, every
inserting backend reopened about 4,000 segments, and inserts stalled
for seconds at a time. The same invalidation is in every branch from
14 to master, and on master on-access pruning can extend the map as
well, so it isn't only VACUUM.
The attached patch adds an invalidation message that just resets the
cached fork sizes, and vm_extend() sends that instead. Readers see
new map pages when they do now. I also tried dropping the
invalidation entirely, but then a backend that doesn't prune the
pages itself keeps a stale map size until VACUUM finishes.
On eCryptfs, with about 2,700 segments and 300 inserting clients,
VACUUM went from about 10 million opens to under 1 million, the extra
system CPU it caused mostly went away, and p99 insert latency during
VACUUM dropped from ~350ms to under 20ms. check-world passes.
Is there a reason the full smgr invalidation is needed here that I'm
missing?
--
Bryan Green
EDB: https://www.enterprisedb.com