pgsql: Set HEAP_XMAX_INVALID for tuples updated by aborted subtransacti

Started by Michael Paquier28 days ago5 messageshackerscommitters
Jump to latest
#1Michael Paquier
michael@paquier.xyz
hackerscommitters

Set HEAP_XMAX_INVALID for tuples updated by aborted subtransaction

HeapTupleSatisfiesVacuumHorizon() did not set HEAP_XMAX_INVALID on a
tuple inserted, then updated by a subtransaction that rolled back.
rewriteheap.c relies on this hint to tell if such tuples should be
treated as updated and would drop them, causing rewrites to lose some
tuples.

HeapTupleSatisfiesVacuumHorizon() is updated to set the hint
HEAP_XMAX_INVALID, so as heap AM is able to redirect such a tuple to its
correct route, and does not lose it.

This issue impacted both CLUSTER (down to v14) and REPACK [USING] (down
to v19).

Reported-by: Alperen Keles <alpkeles99@gmail.com>
Author: Shihao Zhong <zhong950419@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Reported-by: Zhijie Hou <houzhijie22@gmail.com>
Reported-by: Takeshi Ideriha <iderihatakeshi@gmail.com>
Reported-by: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Discussion: /messages/by-id/19760-8f9a0eba17bc8c92@postgresql.org
Backpatch-through: 14

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e7f4463e869cc006b7d3210c264d92b1016dcad2

Modified Files
--------------
src/backend/access/heap/heapam_visibility.c | 2 ++
src/test/regress/expected/cluster.out | 28 ++++++++++++++++++++++++++++
src/test/regress/sql/cluster.sql | 19 +++++++++++++++++++
3 files changed, 49 insertions(+)

#2Robert Haas
robertmhaas@gmail.com
In reply to: Michael Paquier (#1)
hackerscommitters
Re: pgsql: Set HEAP_XMAX_INVALID for tuples updated by aborted subtransacti

On Sun, Oct 25, 2026 at 8:56 PM Michael Paquier <michael@paquier.xyz> wrote:

Set HEAP_XMAX_INVALID for tuples updated by aborted subtransaction

HeapTupleSatisfiesVacuumHorizon() did not set HEAP_XMAX_INVALID on a
tuple inserted, then updated by a subtransaction that rolled back.
rewriteheap.c relies on this hint to tell if such tuples should be
treated as updated and would drop them, causing rewrites to lose some
tuples.

HeapTupleSatisfiesVacuumHorizon() is updated to set the hint
HEAP_XMAX_INVALID, so as heap AM is able to redirect such a tuple to its
correct route, and does not lose it.

Uggh. I think that this is the right fix given that
rewrite_heap_tuple() is documented to rely on hint bits being set for
correctness, but it seems to me to be pretty questionable that we rely
on hint bits being set for correctness. It's not going to be obvious
to someone modifying HeapTupleSatisfiesVacuum() or
HeapTupleSatisfiesVacuumHorizon() that rewrite_heap_tuple() critically
depends on that, and even if we added a comment about that, somebody
reading that comment might justifiably complain about our decision to
create such a dependency.

--
Robert Haas
Databricks

#3Michael Paquier
michael@paquier.xyz
In reply to: Robert Haas (#2)
hackerscommitters
Re: pgsql: Set HEAP_XMAX_INVALID for tuples updated by aborted subtransacti

On Wed, Oct 28, 2026 at 03:32:44PM -0400, Robert Haas wrote:

Uggh. I think that this is the right fix given that
rewrite_heap_tuple() is documented to rely on hint bits being set for
correctness, but it seems to me to be pretty questionable that we rely
on hint bits being set for correctness. It's not going to be obvious
to someone modifying HeapTupleSatisfiesVacuum() or
HeapTupleSatisfiesVacuumHorizon() that rewrite_heap_tuple() critically
depends on that, and even if we added a comment about that, somebody
reading that comment might justifiably complain about our decision to
create such a dependency.

As far as I know this is implied since 7b78474da356. So it's been
around for some time... It's really hard to miss, as the other thread
has proved.
--
Michael

#4Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#3)
hackerscommitters
Re: pgsql: Set HEAP_XMAX_INVALID for tuples updated by aborted subtransacti

On Thu, Oct 29, 2026 at 05:11:26PM +0900, Michael Paquier wrote:

As far as I know this is implied since 7b78474da356. So it's been
around for some time... It's really hard to miss, as the other thread
has proved.

I meant the opposite here, as of "It's really *easy* to miss, as the
other thread has proved". Sorry for the confusion. :)
--
Michael

#5Robert Haas
robertmhaas@gmail.com
In reply to: Michael Paquier (#4)
hackerscommitters
Re: pgsql: Set HEAP_XMAX_INVALID for tuples updated by aborted subtransacti

On Fri, Oct 30, 2026 at 8:34 PM Michael Paquier <michael@paquier.xyz> wrote:

I meant the opposite here, as of "It's really *easy* to miss, as the
other thread has proved". Sorry for the confusion. :)

That makes more sense. Not that I'm volunteering you, or myself, but
maybe somebody should put some effort into redesigning this so that
there isn't so much action-at-a-distance.

--
Robert Haas
Databricks