Avoid using raw xmin of frozen tuples in logicalrep conflict detection
Hi,
While working on another logical replication feature, I noticed a bug: the
apply worker uses a raw xmin that may have been frozen when fetching tuple info
for conflict detection and reporting.
The reason is that, GetTupleTransactionInfo() used slot_getsysattr() with
MinTransactionIdAttributeNumber to fetch the local tuple's xmin.
However, heap_getsysattr() calls HeapTupleHeaderGetRawXmin(), which
returns the raw 32-bit t_xmin from the tuple header even when the tuple
is marked frozen (HEAP_XMIN_FROZEN).
Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.
Attaching a patch fixing this by fetching the tuple via
ExecFetchSlotHeapTuple() and calling HeapTupleHeaderGetXmin(), returning
InvalidTransactionId and skipping the commit timestamp lookup when the tuple's
xmin is not a normal transaction ID. Also update errdetail_apply_conflict() to
omit the transaction ID from unique-conflict detail messages when localxmin is
invalid.
--
Best Regards,
Zhijie Hou
Hi everyone,
On Wed, 4 Nov, 2026, 6:35 pm Zhijie Hou, <houzhijie22@gmail.com> wrote:
Hi,
While working on another logical replication feature, I noticed a bug: the
apply worker uses a raw xmin that may have been frozen when fetching tuple
info
for conflict detection and reporting.The reason is that, GetTupleTransactionInfo() used slot_getsysattr() with
MinTransactionIdAttributeNumber to fetch the local tuple's xmin.
However, heap_getsysattr() calls HeapTupleHeaderGetRawXmin(), which
returns the raw 32-bit t_xmin from the tuple header even when the tuple
is marked frozen (HEAP_XMIN_FROZEN).Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.Attaching a patch fixing this by fetching the tuple via
ExecFetchSlotHeapTuple() and calling HeapTupleHeaderGetXmin(), returning
InvalidTransactionId and skipping the commit timestamp lookup when the
tuple's
xmin is not a normal transaction ID. Also update
errdetail_apply_conflict() to
omit the transaction ID from unique-conflict detail messages when
localxmin is
invalid.--
Best Regards,
Zhijie Hou
I am ready to contribute, but I've some doubts running with regards to and
design decision.
1. Why did GetTupleTransactionInfo() use slot_getsysattr() originally? Was
it chosen to avoid full tuple conversion/detoasting overhead via
ExecFetchSlotHeapTuple(), and what is the performance trade-off of changing
it?
2. How is a "normal transaction ID" strictly defined here? Does the check
use TransactionIdIsNormal() to filter out frozen, bootstrap, and
bootstrap/subtransaction placeholder XIDs?
3. How can this bug be reliably reproduced in an automated regression test?
Is it possible to artificially freeze a tuple via vacuum or pg_freeze/SQL
commands and trigger a logical replication conflict deterministically, or
does it require specific timing with XID wraparound?
I'm kind of busy as my university exams are approaching but I'll try to
find a perfect time to review the patch.
With Regards,
Osama Abdul Qader.
Show quoted text
Hi,
On Wed, Nov 4, 2026 at 9:17 PM Osama Abdul Qader
<osamaabdulqaderpostgres@gmail.com> wrote:
1. Why did GetTupleTransactionInfo() use slot_getsysattr() originally? Was it chosen to avoid full tuple conversion/detoasting overhead via ExecFetchSlotHeapTuple(), and what is the performance trade-off of changing it?
There is no performance difference here. The tuple in the slot is already a
buffer heap tuple in this path, so passing it directly adds no conversion cost.
I missed to handle the frozen case when writing the original codes.
2. How is a "normal transaction ID" strictly defined here? Does the check use TransactionIdIsNormal() to filter out frozen, bootstrap, and bootstrap/subtransaction placeholder XIDs?
Yes, TransactionIdIsNormal() conveniently filters out the frozen XID here. The
other cases you mention shouldn't occur in this path anyway, since we're
detecting conflicts on user table changes.
3. How can this bug be reliably reproduced in an automated regression test? Is it possible to artificially freeze a tuple via vacuum or pg_freeze/SQL commands and trigger a logical replication conflict deterministically, or does it require specific timing with XID wraparound?
Please refer to the tests added in the patch for the reproduction steps.
--
Best Regards,
Zhijie Hou
Hi,
On 2026-11-04 21:04:42 +0800, Zhijie Hou wrote:
The reason is that, GetTupleTransactionInfo() used slot_getsysattr() with
MinTransactionIdAttributeNumber to fetch the local tuple's xmin.
However, heap_getsysattr() calls HeapTupleHeaderGetRawXmin(), which
returns the raw 32-bit t_xmin from the tuple header even when the tuple
is marked frozen (HEAP_XMIN_FROZEN).Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.Attaching a patch fixing this by fetching the tuple via
ExecFetchSlotHeapTuple() and calling HeapTupleHeaderGetXmin(), returning
InvalidTransactionId and skipping the commit timestamp lookup when the tuple's
xmin is not a normal transaction ID. Also update errdetail_apply_conflict() to
omit the transaction ID from unique-conflict detail messages when localxmin is
invalid.
That surely is not the right fix, it breaks any sense of abstraction across
table AMs. There's afaict no guarantee that the target table is heapam. And
even if there currently is, we shouldn't bake that in further.
Greetings,
Andres Freund
Hi Zhijie,
On Wed, 4 Nov, 2026, 7:00 pm Zhijie Hou, <houzhijie22@gmail.com> wrote:
Hi,
On Wed, Nov 4, 2026 at 9:17 PM Osama Abdul Qader
<osamaabdulqaderpostgres@gmail.com> wrote:1. Why did GetTupleTransactionInfo() use slot_getsysattr() originally?
Was it chosen to avoid full tuple conversion/detoasting overhead via
ExecFetchSlotHeapTuple(), and what is the performance trade-off of changing
it?There is no performance difference here. The tuple in the slot is already a
buffer heap tuple in this path, so passing it directly adds no conversion
cost.
I missed to handle the frozen case when writing the original codes.2. How is a "normal transaction ID" strictly defined here? Does the
check use TransactionIdIsNormal() to filter out frozen, bootstrap, and
bootstrap/subtransaction placeholder XIDs?Yes, TransactionIdIsNormal() conveniently filters out the frozen XID here.
The
other cases you mention shouldn't occur in this path anyway, since we're
detecting conflicts on user table changes.3. How can this bug be reliably reproduced in an automated regression
test? Is it possible to artificially freeze a tuple via vacuum or
pg_freeze/SQL commands and trigger a logical replication conflict
deterministically, or does it require specific timing with XID wraparound?Please refer to the tests added in the patch for the reproduction steps.
--
Best Regards,
Zhijie Hou
Thank you for clearing my concerns.
Please refer to the tests added in the patch for the reproduction steps.
Sure, I'll refer to the test added in the patch, and also I'll review it,
though my schedule is a bit tight, but sure I'll do it.
With Regards,
Osama Abdul Qader.
Show quoted text
Hi Andres,
On Wed, Nov 4, 2026 at 09:33 PM Andres Freund wrote:
On 2026-11-04 21:04:42 +0800, Zhijie Hou wrote:
The reason is that, GetTupleTransactionInfo() used slot_getsysattr() with
MinTransactionIdAttributeNumber to fetch the local tuple's xmin.
However, heap_getsysattr() calls HeapTupleHeaderGetRawXmin(), which
returns the raw 32-bit t_xmin from the tuple header even when the tuple
is marked frozen (HEAP_XMIN_FROZEN).Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.Attaching a patch fixing this by fetching the tuple via
ExecFetchSlotHeapTuple() and calling HeapTupleHeaderGetXmin(), returning
InvalidTransactionId and skipping the commit timestamp lookup when the tuple's
xmin is not a normal transaction ID. Also update errdetail_apply_conflict() to
omit the transaction ID from unique-conflict detail messages when localxmin is
invalid.That surely is not the right fix, it breaks any sense of abstraction across
table AMs. There's afaict no guarantee that the target table is heapam. And
even if there currently is, we shouldn't bake that in further.
Thanks for pointing it out. I agree that conflict.c shouldn't be
reaching into BufferHeapTupleTableSlot or HeapTupleHeaderGetXmin() directly.
To avoid that, on master, it seems cleanest to handle this at the
TupleTableSlotOps level (alongside getsysattr and is_current_xact_tuple
in execTuples.c), for example, via an optional callback that returns the
effective xmin (HeapTupleHeaderGetXmin() for heap/buffer-heap slots, and
InvalidTransactionId when NULL for virtual/minimal slots or AMs that
don't store tuple xmin).
TransactionId (*get_effective_xmin) (TupleTableSlot *slot);
I also considered adding a callback to TableAmRoutine instead, but
wasn't sure if it's worth adding a TableAmRoutine entry when the tuple
is already in the slot and getsysattr / is_current_xact_tuple are on
TupleTableSlotOps.
For back-patching to 18/19 where we can't change TupleTableSlotOps
without breaking ABI, GetTupleTransactionInfo() can branch on
TTS_IS_BUFFERTUPLE(localslot) || TTS_IS_HEAPTUPLE(localslot) to use
HeapTupleHeaderGetXmin() and fall back to slot_getsysattr() otherwise,
while still filtering out !TransactionIdIsNormal(xmin).
What do you think ?
--
Best Regards,
Zhijie Hou
Hello Zhijie and Everyone
On Fri, 6 Nov 2026 at 09:43, Zhijie Hou <houzhijie22@gmail.com> wrote:
Hi Andres,
On Wed, Nov 4, 2026 at 09:33 PM Andres Freund wrote:
On 2026-11-04 21:04:42 +0800, Zhijie Hou wrote:
The reason is that, GetTupleTransactionInfo() used slot_getsysattr()
with
MinTransactionIdAttributeNumber to fetch the local tuple's xmin.
However, heap_getsysattr() calls HeapTupleHeaderGetRawXmin(), which
returns the raw 32-bit t_xmin from the tuple header even when the tuple
is marked frozen (HEAP_XMIN_FROZEN).Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.Attaching a patch fixing this by fetching the tuple via
ExecFetchSlotHeapTuple() and calling HeapTupleHeaderGetXmin(), returning
InvalidTransactionId and skipping the commit timestamp lookup when thetuple's
xmin is not a normal transaction ID. Also update
errdetail_apply_conflict() to
omit the transaction ID from unique-conflict detail messages when
localxmin is
invalid.
That surely is not the right fix, it breaks any sense of abstraction
across
table AMs. There's afaict no guarantee that the target table is heapam.
And
even if there currently is, we shouldn't bake that in further.
Thanks for pointing it out. I agree that conflict.c shouldn't be
reaching into BufferHeapTupleTableSlot or HeapTupleHeaderGetXmin()
directly.To avoid that, on master, it seems cleanest to handle this at the
TupleTableSlotOps level (alongside getsysattr and is_current_xact_tuple
in execTuples.c), for example, via an optional callback that returns the
effective xmin (HeapTupleHeaderGetXmin() for heap/buffer-heap slots, and
InvalidTransactionId when NULL for virtual/minimal slots or AMs that
don't store tuple xmin).TransactionId (*get_effective_xmin) (TupleTableSlot *slot);
I also considered adding a callback to TableAmRoutine instead, but
wasn't sure if it's worth adding a TableAmRoutine entry when the tuple
is already in the slot and getsysattr / is_current_xact_tuple are on
TupleTableSlotOps.For back-patching to 18/19 where we can't change TupleTableSlotOps
without breaking ABI, GetTupleTransactionInfo() can branch on
TTS_IS_BUFFERTUPLE(localslot) || TTS_IS_HEAPTUPLE(localslot) to use
HeapTupleHeaderGetXmin() and fall back to slot_getsysattr() otherwise,
while still filtering out !TransactionIdIsNormal(xmin).What do you think ?
--
Best Regards,
Zhijie HouThanks for the explanation and for providing the updated design direction.
I've reviewed the v1 patch and the relevant code paths, including
GetTupleTransactionInfo(), TupleTableSlotOps, and its implementations in
execTuples.c.
I agree that the patch addresses a real issue: using the raw xmin of a
frozen tuple can expose an obsolete transaction ID. This can lead to
misleading conflict detection and reporting, particularly when commit
timestamp information is no longer available or the transaction ID has been
reused.
However, I also agree with Andres's concern that directly accessing
BufferHeapTupleTableSlot or using HeapTupleHeaderGetXmin() inside conflict.c
introduces a table access method abstraction problem. The conflict-handling
code should not need to know the underlying tuple storage implementation.
One possible approach for master would be to extend TupleTableSlotOps with
a callback such as:
`TransactionId (*get_effective_xmin) (TupleTableSlot *slot);`
Each slot implementation could then provide the appropriate behavior:
- Heap tuple and buffer heap tuple slots would return the effective xmin
from the underlying tuple header.
-
Virtual and minimal tuple slots, or implementations that cannot provide
a meaningful xmin, would return InvalidTransactionId.
GetTupleTransactionInfo() could then obtain the effective xmin through this
callback, check TransactionIdIsNormal(), and proceed with commit timestamp
and replication origin lookup only when a valid transaction ID is available.
This would keep the conflict-handling code independent of heap-specific
details while making the semantics of retrieving an effective xmin explicit
at the slot abstraction level.
For backpatch branches, I understand that modifying TupleTableSlotOps may
not be appropriate because of ABI compatibility concerns. We may therefore
need a separate, version-specific solution that avoids introducing an ABI
change while still handling frozen tuples correctly.
I think this separates the underlying design issue from the immediate bug
fix and provides a cleaner direction for master.
What do you think about introducing get_effective_xmin as a slot operation
for master?
Best regards,
Osama Abdul Qader
Hi,
On Sun, Nov 8, 2026 at 12:29 AM Osama Abdul Qader wrote:
One possible approach for master would be to extend TupleTableSlotOps with
a callback such as:`TransactionId (*get_effective_xmin) (TupleTableSlot *slot);`
...
What do you think about introducing get_effective_xmin as a slot operation
for master?
I'm afraid I find your reply quite confusing, as it is repeating the proposal
I already made in my reply to Andres, without any difference I can see.
Could you please read the discussion in this thread carefully and
share your thoughts after considering it? If you're using AI to help
draft replies, please review and understand the output fully before sending it.
--
Best Regards,
Zhijie Hou
Hi
On Sun, 8 Nov, 2026, 11:41 am Zhijie Hou, <houzhijie22@gmail.com> wrote:
Hi,
On Sun, Nov 8, 2026 at 12:29 AM Osama Abdul Qader wrote:
One possible approach for master would be to extend TupleTableSlotOps
with
a callback such as:
`TransactionId (*get_effective_xmin) (TupleTableSlot *slot);`
...
What do you think about introducing get_effective_xmin as a slot
operation
for master?
I'm afraid I find your reply quite confusing, as it is repeating the
proposal
I already made in my reply to Andres, without any difference I can see.Could you please read the discussion in this thread carefully and
share your thoughts after considering it? If you're using AI to help
draft replies, please review and understand the output fully before
sending it.--
Best Regards,
Zhijie Hou
Thank you for pointing this out, I repeat the proposal of
get_effective_xmin callback approach, I apologize for the confusion I
caused here.
I'd be interested in helping with the implementation and further
investigation of this approach. In particular, I can examine the callback's
behavior across the different TupleTableSlotOps implementations, identify
any edge cases around frozen tuples and invalid transaction IDs, and work
on regression tests.
I'd also like to understand the compatibility constraints for the backpatch
branches and how the proposed master implementation should differ from them.
If useful, I'd be happy to take on this implementation work and submit a
patch for review.
Best regards,
Osama Abdul Qader.
Hi,
On Fri, Nov 6, 2026 at 12:13 PM Zhijie Hou wrote:
Hi Andres,
On Wed, Nov 4, 2026 at 09:33 PM Andres Freund wrote:
That surely is not the right fix, it breaks any sense of abstraction
across table AMs. There's afaict no guarantee that the target table
is heapam. And even if there currently is, we shouldn't bake that in
further.Thanks for pointing it out. I agree that conflict.c shouldn't be
reaching into BufferHeapTupleTableSlot or HeapTupleHeaderGetXmin()
directly.To avoid that, on master, it seems cleanest to handle this at the
TupleTableSlotOps level (alongside getsysattr and
is_current_xact_tuple in execTuples.c), for example, via an optional
callback that returns the effective xmin (HeapTupleHeaderGetXmin() for
heap/buffer-heap slots, and InvalidTransactionId when NULL for
virtual/minimal slots or AMs that don't store tuple xmin).TransactionId (*get_effective_xmin) (TupleTableSlot *slot);
Following this, I am attaching two patches implementing the approaches
discussed above for reference:
v2-0001-Fix-logical-xx.patch adds an optional get_effective_xmin
callback to TupleTableSlotOps (alongside getsysattr and
is_current_xact_tuple). Heap and buffer-heap slots implement it via
HeapTupleHeaderGetXmin(), while slots that do not provide this
information leave the callback unset so slot_get_effective_xmin()
returns InvalidTransactionId.
nocfbot-v2-0001-Fix-logical-xx.patch avoids adding a new
TupleTableSlotOps callback (which is also needed for back-patching to
18) by checking TTS_IS_BUFFERTUPLE(localslot) ||
TTS_IS_HEAPTUPLE(localslot) in GetTupleTransactionInfo() to use
HeapTupleHeaderGetXmin(), and falling back to slot_getsysattr() for
other slot types.
What do you think ?
--
Best Regards,
Zhijie Hou
Attachments:
t254559_10nocfbot-v2-0001-Fix-logical-replication-conflict-detection-on-fro-without-callback.patchtext/plain; charset=UTF-8; name=nocfbot-v2-0001-Fix-logical-replication-conflict-detection-on-fro-without-callback.patchDownload+100-14
v2-0001-Fix-logical-replication-conflict-detection-on-fro.patchtext/plain; charset=UTF-8; name=v2-0001-Fix-logical-replication-conflict-detection-on-fro.patchDownload+148-14
Hi,
On Mon, 9 Nov, 2026, 5:05 pm Zhijie Hou, <houzhijie22@gmail.com> wrote:
Hi,
On Fri, Nov 6, 2026 at 12:13 PM Zhijie Hou wrote:
Hi Andres,
On Wed, Nov 4, 2026 at 09:33 PM Andres Freund wrote:
That surely is not the right fix, it breaks any sense of abstraction
across table AMs. There's afaict no guarantee that the target table
is heapam. And even if there currently is, we shouldn't bake that in
further.Thanks for pointing it out. I agree that conflict.c shouldn't be
reaching into BufferHeapTupleTableSlot or HeapTupleHeaderGetXmin()
directly.To avoid that, on master, it seems cleanest to handle this at the
TupleTableSlotOps level (alongside getsysattr and
is_current_xact_tuple in execTuples.c), for example, via an optional
callback that returns the effective xmin (HeapTupleHeaderGetXmin() for
heap/buffer-heap slots, and InvalidTransactionId when NULL for
virtual/minimal slots or AMs that don't store tuple xmin).TransactionId (*get_effective_xmin) (TupleTableSlot *slot);
Following this, I am attaching two patches implementing the approaches
discussed above for reference:v2-0001-Fix-logical-xx.patch adds an optional get_effective_xmin
callback to TupleTableSlotOps (alongside getsysattr and
is_current_xact_tuple). Heap and buffer-heap slots implement it via
HeapTupleHeaderGetXmin(), while slots that do not provide this
information leave the callback unset so slot_get_effective_xmin()
returns InvalidTransactionId.nocfbot-v2-0001-Fix-logical-xx.patch avoids adding a new
TupleTableSlotOps callback (which is also needed for back-patching to
18) by checking TTS_IS_BUFFERTUPLE(localslot) ||
TTS_IS_HEAPTUPLE(localslot) in GetTupleTransactionInfo() to use
HeapTupleHeaderGetXmin(), and falling back to slot_getsysattr() for
other slot types.What do you think ?
--
Best Regards,
Zhijie Hou
I'm open to review the new patches, I'll reply again after reviewing it,
not only that I'm also open to fix the issues that comes after reviewing
it.
I regret for the inconvenience I caused earlier, I assure to everyone here
that this won't happen again.
With Regards,
Osama Abdul Qader.
Hi again fellow hackers, may y'all stay happy and fresh.
On Mon, 9 Nov 2026 at 17:12, Osama Abdul Qader <
osamaabdulqaderpostgres@gmail.com> wrote:
Hi,
On Mon, 9 Nov, 2026, 5:05 pm Zhijie Hou, <houzhijie22@gmail.com> wrote:
Hi,
On Fri, Nov 6, 2026 at 12:13 PM Zhijie Hou wrote:
Hi Andres,
On Wed, Nov 4, 2026 at 09:33 PM Andres Freund wrote:
That surely is not the right fix, it breaks any sense of abstraction
across table AMs. There's afaict no guarantee that the target table
is heapam. And even if there currently is, we shouldn't bake that in
further.Thanks for pointing it out. I agree that conflict.c shouldn't be
reaching into BufferHeapTupleTableSlot or HeapTupleHeaderGetXmin()
directly.To avoid that, on master, it seems cleanest to handle this at the
TupleTableSlotOps level (alongside getsysattr and
is_current_xact_tuple in execTuples.c), for example, via an optional
callback that returns the effective xmin (HeapTupleHeaderGetXmin() for
heap/buffer-heap slots, and InvalidTransactionId when NULL for
virtual/minimal slots or AMs that don't store tuple xmin).TransactionId (*get_effective_xmin) (TupleTableSlot *slot);
Following this, I am attaching two patches implementing the approaches
discussed above for reference:v2-0001-Fix-logical-xx.patch adds an optional get_effective_xmin
callback to TupleTableSlotOps (alongside getsysattr and
is_current_xact_tuple). Heap and buffer-heap slots implement it via
HeapTupleHeaderGetXmin(), while slots that do not provide this
information leave the callback unset so slot_get_effective_xmin()
returns InvalidTransactionId.nocfbot-v2-0001-Fix-logical-xx.patch avoids adding a new
TupleTableSlotOps callback (which is also needed for back-patching to
18) by checking TTS_IS_BUFFERTUPLE(localslot) ||
TTS_IS_HEAPTUPLE(localslot) in GetTupleTransactionInfo() to use
HeapTupleHeaderGetXmin(), and falling back to slot_getsysattr() for
other slot types.What do you think ?
--
Best Regards,
Zhijie HouI'm open to review the new patches, I'll reply again after reviewing it,
not only that I'm also open to fix the issues that comes after reviewing
it.I regret for the inconvenience I caused earlier, I assure to everyone here
that this won't happen again.With Regards,
Osama Abdul Qader.
I've finished testing both patch alternatives for the frozen tuple xmin
issue.
I tested the following versions independently:
1. The callback-based v2 patch (get_effective_xmin).
2. The no-callback alternative.
3. The unpatched baseline.
The unpatched baseline reproduced the original issue. In particular, UPDATE
and DELETE operations against frozen tuples triggered false
update_origin_differs and delete_origin_differs conflicts.
I also verified the unique-conflict reporting behaviour. With the patched
version, the conflict detail still identifies the unique index, conflicting
key and local row, but no longer attributes a misleading transaction ID and
timestamp to the frozen tuple.
*Code review observation*
For the callback-based approach, I think adding get_effective_xmin to
TupleTableSlotOps is a cleaner abstraction. The caller requests the
effective xmin through the slot interface without needing to know the
underlying tuple representation.
For the no-callback alternative, I inspected the TTS_IS_HEAPTUPLE and
TTS_IS_BUFFERTUPLE checks, along with ExecFetchSlotHeapTuple() and the
corresponding get_heap_tuple implementations. The heap-slot handling
appears correct, including the shouldFree assertion.
My main architectural question concerns the no-callback alternative:
conflict.c now explicitly handles the two built-in heap slot types and
falls back to slot_getsysattr() for other slot types. Is this considered an
acceptable compromise for stable-branch backpatching, while the
callback-based approach is preferred for master?
I haven't identified a concrete correctness defect in either implementation
from the paths I've reviewed so far.
I'd appreciate your thoughts on the preferred approach and whether there
are any additional cases you think should be covered.
With Regards,
Osama Abdul Qader.
On Thu, Nov 5, 2026 at 12:05 AM Zhijie Hou <houzhijie22@gmail.com> wrote:
Hi,
Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.
I have a question about the basic premise here.
As far as I can see, being frozen doesn't make the raw xmin wrong by
itself. TransactionIdGetCommitTsData() already returns false when the
xid falls outside [oldestCommitTsXid, newestCommitTsXid], so if
commit_ts has been truncated past the xid, we just don't report a
conflict, which is fine. Bogus data only comes back once the same
32-bit value has been reassigned and is back inside the commit_ts
window, which needs on the order of 2^32 XIDs after the tuple was
inserted. Until then, the raw xmin of a frozen tuple gives the correct
origin and commit timestamp.
With the patch, every frozen tuple is treated as having no xmin, so
update_origin_differs / delete_origin_differs are never reported for
it. When a tuple gets frozen depends on vacuum: vacuum_freeze_min_age,
page-level opportunistic freezing since PG16, eager scanning of
all-visible pages in PG18, or a manual VACUUM FREEZE. So for the same
row and the same remote change, whether we report a genuine origin
conflict would depend on whether autovacuum happened to freeze that
page first, which makes it largely non-deterministic. That seems hard
to explain to users, why the behaviour changes from case to case.
The new test shows this. Rows (3,1) and (4,2) are inserted locally on
node A and then frozen, and the subsequent UPDATE/DELETE come from B.
The local origin (0) really does differ from B’s origin, so without
VACUUM FREEZE these would be reported as update_origin_differs and
delete_origin_differs, the mirror image of the delete_origin_differs
case checked later in 035_conflicts.pl. The new test asserts that
these real conflicts are no longer reported.
So my question is whether being frozen is the right criterion, or
whether we should try to address only the XID-reuse case. I don't
immediately see a cheap way to tell a reused xid from the original
with just 32 bits, so maybe the conservative behaviour is the price we
have to pay.
regards,
Ajin Cherian
Fujitsu Australia
Hi there,
On Wed, 11 Nov 2026 at 08:09, Ajin Cherian <itsajin@gmail.com> wrote:
On Thu, Nov 5, 2026 at 12:05 AM Zhijie Hou <houzhijie22@gmail.com> wrote:
Hi,
Once a tuple is frozen, its raw t_xmin is retained only for debugging
purposes; its commit timestamp entry may no longer be valid, or the
32-bit XID may have wrapped around and been reassigned in a later epoch.
Using the raw xmin of a frozen tuple could therefore trigger spurious
update_origin_differs or delete_origin_differs conflicts, or report
bogus transaction ID, origin, and commit timestamp details for unique
constraint conflicts.I have a question about the basic premise here.
As far as I can see, being frozen doesn't make the raw xmin wrong by
itself. TransactionIdGetCommitTsData() already returns false when the
xid falls outside [oldestCommitTsXid, newestCommitTsXid], so if
commit_ts has been truncated past the xid, we just don't report a
conflict, which is fine. Bogus data only comes back once the same
32-bit value has been reassigned and is back inside the commit_ts
window, which needs on the order of 2^32 XIDs after the tuple was
inserted. Until then, the raw xmin of a frozen tuple gives the correct
origin and commit timestamp.With the patch, every frozen tuple is treated as having no xmin, so
update_origin_differs / delete_origin_differs are never reported for
it. When a tuple gets frozen depends on vacuum: vacuum_freeze_min_age,
page-level opportunistic freezing since PG16, eager scanning of
all-visible pages in PG18, or a manual VACUUM FREEZE. So for the same
row and the same remote change, whether we report a genuine origin
conflict would depend on whether autovacuum happened to freeze that
page first, which makes it largely non-deterministic. That seems hard
to explain to users, why the behaviour changes from case to case.The new test shows this. Rows (3,1) and (4,2) are inserted locally on
node A and then frozen, and the subsequent UPDATE/DELETE come from B.
The local origin (0) really does differ from B’s origin, so without
VACUUM FREEZE these would be reported as update_origin_differs and
delete_origin_differs, the mirror image of the delete_origin_differs
case checked later in 035_conflicts.pl. The new test asserts that
these real conflicts are no longer reported.So my question is whether being frozen is the right criterion, or
whether we should try to address only the XID-reuse case. I don't
immediately see a cheap way to tell a reused xid from the original
with just 32 bits, so maybe the conservative behaviour is the price we
have to pay.regards,
Ajin Cherian
Fujitsu Australia
Thanks for raising this. I think your concern about the behaviour becoming
dependent on vacuum timing is valid.
I agree that freezing does not necessarily make the original raw xmin
unusable. It changes how PostgreSQL interprets xmin for tuple visibility,
while the original XID remains stored in the tuple header. If that XID has
not been reused and its commit timestamp information is still available,
using it to retrieve the original transaction commit timestamp and
replication origin can still be legitimate.
I also agree that TransactionIdGetCommitTsData() cannot inherently
distinguish the original transaction from a later transaction that reused
the same 32-bit XID. Its commit-timestamp range check does not, by itself,
establish the identity of the transaction that originally inserted the
frozen tuple.
This leaves an important trade-off:
- Continuing to use raw xmin preserves origin-conflict detection while
the original transaction metadata remains valid, but risks attributing
metadata from a newer transaction after XID reuse.
- Treating every frozen tuple as having no usable xmin avoids that
stale-XID attribution, but can suppress genuine origin conflicts even when
the original metadata is still valid.
I think your observation about the regression test is particularly
important: the test currently verifies that conflicts disappear after VACUUM
FREEZE, but that alone does not establish that suppressing those conflicts
is the correct behaviour.
The current patch chooses the latter, but I think the resulting change in
conflict-detection behaviour deserves explicit justification.
Regards,
Osama Abdul Qader