Re-read conindid under the referenced table's lock in the RI fast path

Started by Mihail Nikalayeu20 days ago6 messageshackers
Beta feature

Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.

needs rebasesuccessCI history

You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:

docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253325
psql -h localhost -U postgres

Built from patchset v4 (message #4), August 17, 2026 at 01:20 PM.

Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:

git clone --branch t253325_4 https://github.com/hackorum-dev/postgres.git

In a checkout you already have, add the fork once:

git remote add hackorum https://github.com/hackorum-dev/postgres.git

then, for this patchset and every later one:

git fetch hackorum t253325_4 && git checkout t253325_4

Patchset v4 (message #4) is on t253325_4

Jump to latest
#1Mihail Nikalayeu
mihailnikalayeu@gmail.com

Hello, everyone!

As was asked in [0]/messages/by-id/CA+HiwqGXcHnrYD1BoXRwNat3QdtbsYZEZrexnPET_QGtMY=TZQ@mail.gmail.com I have created a separate thread for [1]/messages/by-id/CADzfLwUJiVuv69uwuF5z4TrMhNkVwQUXW03q+uVNwmYFLtjEhw@mail.gmail.com.
Also, I realized that [1]/messages/by-id/CADzfLwUJiVuv69uwuF5z4TrMhNkVwQUXW03q+uVNwmYFLtjEhw@mail.gmail.com and [2]/messages/by-id/CADzfLwXj_9rz3XGSBVzJ=HdWXt6VMp8h2tAC1e_49yDBESUg-A@mail.gmail.com is the same issue (sorry, I lost a
bit in them all).

Short quote from commit message:

The RI fast path looks up the constraint, takes RowShareLock on the
referenced table, and opens the index conindid names. Reading conindid
before that lock is not safe. REINDEX CONCURRENTLY repoints the
constraint at a new index and then drops the old one, and it waits only
for backends holding a lock on the referenced table; a backend that has
read the constraint but not yet taken that lock is not one of them. It
then opens an index that is already gone, and the write fails with
"could not open relation with OID"

Main patch\reproducer is v1-0001, nocfbox-X series also provide a
reproducer for the data corruption (though VACUUM is also involved).

Best regards,
Mikhail.

[0]: /messages/by-id/CA+HiwqGXcHnrYD1BoXRwNat3QdtbsYZEZrexnPET_QGtMY=TZQ@mail.gmail.com
[1]: /messages/by-id/CADzfLwUJiVuv69uwuF5z4TrMhNkVwQUXW03q+uVNwmYFLtjEhw@mail.gmail.com
[2]: /messages/by-id/CADzfLwXj_9rz3XGSBVzJ=HdWXt6VMp8h2tAC1e_49yDBESUg-A@mail.gmail.com

Attachments:

t253325_1
v1-0001-Re-read-the-FK-constraint-after-locking-the-refer.patchtext/x-patch; charset=US-ASCII; name=v1-0001-Re-read-the-FK-constraint-after-locking-the-refer.patchDownload+380-1
nocfbot-X-0002-Reproducer-RI-fast-path-accepts-a-row-whose-refer.patchtext/x-patch; charset=US-ASCII; name=nocfbot-X-0002-Reproducer-RI-fast-path-accepts-a-row-whose-refer.patchDownload+195-2
nocfbot-X-0003-Re-read-conindid-under-the-referenced-table-s-loc.patchtext/x-patch; charset=US-ASCII; name=nocfbot-X-0003-Re-read-conindid-under-the-referenced-table-s-loc.patchDownload+19-1
nocfbot-X-0001-Reproducer-RI-fast-path-opens-an-index-a-concurre.patchtext/x-patch; charset=US-ASCII; name=nocfbot-X-0001-Reproducer-RI-fast-path-opens-an-index-a-concurre.patchDownload+162-1
#2Melanie Plageman
melanieplageman@gmail.com
In reply to: Mihail Nikalayeu (#1)
Re: Re-read conindid under the referenced table's lock in the RI fast path

On Wed, Aug 5, 2026 at 9:56 AM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:

Hello, everyone!

As was asked in [0] I have created a separate thread for [1].
Also, I realized that [1] and [2] is the same issue (sorry, I lost a
bit in them all).

[RMT hat]

I've added an open item to track this [1]https://wiki.postgresql.org/wiki/PostgreSQL_19_Open_Items#Open_Issues.

- Melanie

[1]: https://wiki.postgresql.org/wiki/PostgreSQL_19_Open_Items#Open_Issues

#3Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Melanie Plageman (#2)
Re: Re-read conindid under the referenced table's lock in the RI fast path

On Thu, Aug 13, 2026 at 6:24 Melanie Plageman <melanieplageman@gmail.com>
wrote:

On Wed, Aug 5, 2026 at 9:56 AM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:

Hello, everyone!

As was asked in [0] I have created a separate thread for [1].
Also, I realized that [1] and [2] is the same issue (sorry, I lost a
bit in them all).

[RMT hat]

I've added an open item to track this [1].

- Melanie

[1] https://wiki.postgresql.org/wiki/PostgreSQL_19_Open_Items#Open_Issues

Thank you Melanie. I’m going to look at this next week.

- Amit

Show quoted text

<https://wiki.postgresql.org/wiki/PostgreSQL_19_Open_Items#Open_Issues&gt;

#4Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Mihail Nikalayeu (#1)
Re: Re-read conindid under the referenced table's lock in the RI fast path

Hi Mihail,

On Wed, Aug 5, 2026 at 10:56 PM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:

Hello, everyone!

As was asked in [0] I have created a separate thread for [1].
Also, I realized that [1] and [2] is the same issue (sorry, I lost a
bit in them all).

Short quote from commit message:

The RI fast path looks up the constraint, takes RowShareLock on the
referenced table, and opens the index conindid names. Reading conindid
before that lock is not safe. REINDEX CONCURRENTLY repoints the
constraint at a new index and then drops the old one, and it waits only
for backends holding a lock on the referenced table; a backend that has
read the constraint but not yet taken that lock is not one of them. It
then opens an index that is already gone, and the write fails with
"could not open relation with OID"

Main patch\reproducer is v1-0001, nocfbox-X series also provide a
reproducer for the data corruption (though VACUUM is also involved).

Thanks. Attached is v2 of the patch posted here. I adjusted the commit
message and comments, kept you credited as Author, and replaced the
Opus co-author trailer with a disclosure sentence, which I can remove
if you so prefer. I have noticed only one such disclosure in the
commit log so far.

I dropped the subtransaction permutation from the test suite because
the pending subtransaction-batching patch [1]/messages/by-id/CA+HiwqGHa3tc6MZFSLyLrvwySdrmpkb1TqkH2jGd3HtKGGZ6cQ@mail.gmail.com will duplicate the
batched case. I briefly considered using ALTER TABLE validation to
retain per-row coverage, but since it already holds the
referenced-table lock, it cannot exercise the race the test case was
meant to manifest.

I will push this tomorrow.

[1]: /messages/by-id/CA+HiwqGHa3tc6MZFSLyLrvwySdrmpkb1TqkH2jGd3HtKGGZ6cQ@mail.gmail.com

--
Thanks, Amit Langote

Attachments:

t253325_4
v2-0001-Fix-RI-fast-path-race-with-REINDEX-CONCURRENTLY.patchapplication/octet-stream; name=v2-0001-Fix-RI-fast-path-race-with-REINDEX-CONCURRENTLY.patchDownload+302-1
#5Mihail Nikalayeu
mihailnikalayeu@gmail.com
In reply to: Amit Langote (#4)
Re: Re-read conindid under the referenced table's lock in the RI fast path

Hi!

On Mon, Aug 17, 2026 at 3:07 PM Amit Langote <amitlangote09@gmail.com> wrote:

Thanks. Attached is v2 of the patch posted here. I adjusted the commit
message and comments, kept you credited as Author, and replaced the
Opus co-author trailer with a disclosure sentence, which I can remove
if you so prefer.

I'm not sure it's required, I think using some AI tool is the new
default now, at least for some phases.

I will push this tomorrow.

Thanks!

Best regards,
Mikhail.

#6Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: Mihail Nikalayeu (#5)
Re: Re-read conindid under the referenced table's lock in the RI fast path

Hi,

On Mon, Aug 17, 2026 at 10:17 PM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:

On Mon, Aug 17, 2026 at 3:07 PM Amit Langote <amitlangote09@gmail.com> wrote:

Thanks. Attached is v2 of the patch posted here. I adjusted the commit
message and comments, kept you credited as Author, and replaced the
Opus co-author trailer with a disclosure sentence, which I can remove
if you so prefer.

I'm not sure it's required, I think using some AI tool is the new
default now, at least for some phases.

Ok, removed.

I will push this tomorrow.

Done.

--
Thanks, Amit Langote