GIN page deletion and page recycling bugs
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
This thread has been committed, so CI has stopped here. Anything below is the last result it produced.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253497psql -h localhost -U postgresBuilt from patchset v3 (message #3), August 22, 2026 at 10:48 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t253497_3 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t253497_3 && git checkout t253497_3Patchset v3 (message #3) is on t253497_3
I had Opus 5 look for more bugs in GIN. It found and diagnosed another
2. I attach 2 patches, each adding a test case that demonstrates a
distinct bug in GIN.
The first bug is an oversight in posting tree page deletion. The issue
is that ginScanPostingTreeToDelete fails to consider whether a target
page (or its left sibling page) is GinPageIsIncompleteSplit-marked.
Page deletion must not proceed if either page is marked, because the
right half of an unfinished split has no downlink yet, which deletion
is fundamentally incapable of handling. Currently, unsafe page
deletions are allowed, which leaves the posting tree in an
inconsistent state. This index corruption can lead to wrong answers,
which the test case added by 0001 demonstrates directly.
nbtree page deletion deals with the same incomplete split problem by
backing out of deleting the page. Fixing this bug will likely involve
teaching GIN to do much the same thing. ginScanPostingTreeToDelete
already avoids deleting both the leftmost and rightmost page; it
likely needs some additional deletion-is-unsafe tests. Incomplete
splits are rare, so we can reasonably assume an inserter will complete
them eventually (again, like nbtree).
The second bug is in page recycling. Apparently, GIN doesn't write a
WAL record to create a recovery conflict on standbys when a deleted
page is subsequently recycled on the primary. Both nbtree and GiST
already handle this correctly. It's intuitively obvious that GIN
should do the same thing, but the tap test added by 0002 actually
proves it: it demonstrates a query running on a standby that gets a
wrong answer (one that disagrees with the answer the same query gets
when a sequential scan is forced).
--
Peter Geoghegan
Attachments:
t253497_10001-Add-repro-GIN-VACUUM-unlinks-a-live-page-past-an-unf.patchapplication/octet-stream; name=0001-Add-repro-GIN-VACUUM-unlinks-a-live-page-past-an-unf.patchDownload+218-3
0002-Add-repro-GIN-page-reuse-raises-no-recovery-conflict.patchapplication/octet-stream; name=0002-Add-repro-GIN-page-reuse-raises-no-recovery-conflict.patchDownload+197-2
On Wed, Aug 19, 2026 at 6:59 PM Peter Geoghegan <pg@bowt.ie> wrote:
nbtree page deletion deals with the same incomplete split problem by
backing out of deleting the page. Fixing this bug will likely involve
teaching GIN to do much the same thing. ginScanPostingTreeToDelete
already avoids deleting both the leftmost and rightmost page; it
likely needs some additional deletion-is-unsafe tests. Incomplete
splits are rare, so we can reasonably assume an inserter will complete
them eventually (again, like nbtree).
Attached patch fixes the bug as outlined. It would be good to get a
GIN expert to review this.
This fix will need some work to be backpatchable; we'll need to work
around refactoring commit fa6f2f624c, which is only on 19 and master.
I simplified the repro that Claude code devised for this, and included
it in this patch. I think that the test case is worth committing.
The second bug is in page recycling. Apparently, GIN doesn't write a
WAL record to create a recovery conflict on standbys when a deleted
page is subsequently recycled on the primary.
I don't have a fix for this one yet. It's likely less complicated than
the incomplete split bug fix, but it is nevertheless a bit more
awkward.
The standard way of fixing this issue is to invent a new GIN record
type that works exactly like XLOG_BTREE_REUSE_PAGE and
XLOG_GIST_PAGE_REUSE. But that seems silly to me: the underlying
structs for both of those existing record types (gistxlogPageReuse and
xl_btree_reuse_page) are already identical. That's because nothing
specific to either index AM is involved, and no buffer is registered
-- we only need to generate a recovery conflict using a given
snapshotConflictHorizon XID. That suggests there should be one generic
WAL record that all 3 index AMs (nbtree, GiST, and now GIN) use,
superseding the existing record types. But that still leaves the back
branches.
On the back branches, it makes sense to avoid inventing a new WAL
record for this (doing so is theoretically allowed, but it needlessly
breaks standbys on older point releases when the primary is on the
latest point release). We could fix the bug on the back branches by
having GIN call a new GiST extern helper function when it recycles a
deleted page on the primary. That helper would create the necessary
conflict based on a snapshotConflictHorizon passed by GIN -- it'd just
write a XLOG_GIST_PAGE_REUSE record to represent GIN's page recycling
action.
--
Peter Geoghegan
On 20 Aug 2026, at 20:52, Peter Geoghegan <pg@bowt.ie> wrote:
Attached patch fixes the bug as outlined. It would be good to get a
GIN expert to review this.
Hi Peter,
I don't think I qualify as a GIN expert, but both bugs and the proposed
fixes seem fairly clear to me. PFA a three-patch version. The first
two patches are intended for backpatching; the third one is for master
only.
Your patch looks correct to me. It might be worth covering the
incomplete target page separately: the test currently covers deletion
after an incomplete left sibling. I would also avoid the
expected count of 39879. Comparing the index result with a sequential
scan would express the correctness property more directly and tie the
test less closely to the physical page layout. This already failed in
Linux Autoconf CI, which produced 36449 instead of 39879 [0]https://github.com/x4m/postgres_g/actions/runs/32563364088/job/97008437447.
In the longer term, a heap-all-indexed check for GIN in amcheck would
be a better oracle for bugs like this.
The second patch follows your back-branch proposal and emits
XLOG_GIST_PAGE_REUSE when GIN reuses a deleted page. The reproducer now
expects replay to cancel the standby scan. Without the fix I get 7860
rows from the GIN scan instead of 112338 from the sequential scan; with
the fix the recovery conflict is raised as expected.
The third patch addresses one related old issue on master. GIN still
stores its deletion horizon as a 32-bit XID. After about 2 billion XIDs
a deleted page can be considered non-recyclable for the following
2-billion-XID window. GiST and nbtree were changed to store full XIDs in
6655a7299d8 and e5d8a999030. I mentioned the same problem in GIN in 2019
[1]: /messages/by-id/AFDDB5CC-CC41-4E73-AD92-09AF4BDE926F@yandex-team.ru
page flag, so pg_upgraded indexes keep the old conservative behavior for
their old deleted pages. I do not propose this part for backpatching.
Thanks!
Best regards, Andrey Borodin.
[0]: https://github.com/x4m/postgres_g/actions/runs/32563364088/job/97008437447
[1]: /messages/by-id/AFDDB5CC-CC41-4E73-AD92-09AF4BDE926F@yandex-team.ru