[PATCH] Fix references in comments, and sync up heap_page_is_all_visible() with heap_page_prune_and_freeze()
While working on [1]/messages/by-id/78574B24-BE0A-42C5-8075-3FA9FA63B8FC@amazon.com I found an outdated comment in heap_page_is_all_visible() and two other small fixes.
0001: Updates that comment so future authors know that this "stripped down function" should retain the logic in heap_page_prune_and_freeze(), not lazy_scan_prune() as was the case before 6dbb490.
0002: Mimics the same loop logic as in heap_page_is_all_visible() so as to a) stay in sync and b) benefit from the mentioned CPU prefetching optimization.
0003: Moves the ItemSetPointer() just a bit further down in the function again to a) stay in sync and b) to sometimes avoid that tiny overhead.
best,
-greg
PS: per-community standards I've switched to my personal email address rather than gregburd@amazon.com
[1]: /messages/by-id/78574B24-BE0A-42C5-8075-3FA9FA63B8FC@amazon.com
Attachments:
v1-0002-Reverse-loop-to-match-counterpart-and-optimize-fo.patchapplication/octet-stream; filename=v1-0002-Reverse-loop-to-match-counterpart-and-optimize-fo.patch; name=v1-0002-Reverse-loop-to-match-counterpart-and-optimize-fo.patchDownload
From dff9c509856223db78178a0f6fdd1a8b3f0affd7 Mon Sep 17 00:00:00 2001
From: Greg Burd <greg@burd.me>
Date: Tue, 6 May 2025 11:14:19 -0400
Subject: [PATCH v1 2/3] Reverse loop to match counterpart and optimize for CPU
prefetching.
Small change to ensure that the loop in heap_page_is_all_visible()
takes advantage of the CPU prefetching optimization found in its
counterpart heap_page_prune_and_freeze() and is generally more in
sync.
Author: Greg Burd <greg@burd.me>
---
src/backend/access/heap/vacuumlazy.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index e450911424f..14cf880340b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3625,10 +3625,15 @@ heap_page_is_all_visible(LVRelState *vacrel, Buffer buf,
*visibility_cutoff_xid = InvalidTransactionId;
*all_frozen = true;
+ /*
+ * Processing the items in reverse order (and thus the tuples in
+ * increasing order) increases prefetching efficiency significantly /
+ * decreases the number of cache misses.
+ */
maxoff = PageGetMaxOffsetNumber(page);
- for (offnum = FirstOffsetNumber;
- offnum <= maxoff && all_visible;
- offnum = OffsetNumberNext(offnum))
+ for (offnum = maxoff;
+ offnum >= FirstOffsetNumber && all_visible;
+ offnum = OffsetNumberPrev(offnum))
{
ItemId itemid;
HeapTupleData tuple;
--
2.39.5 (Apple Git-154)
v1-0001-Fix-references-in-comments-in-heap_page_is_all_vi.patchapplication/octet-stream; filename=v1-0001-Fix-references-in-comments-in-heap_page_is_all_vi.patch; name=v1-0001-Fix-references-in-comments-in-heap_page_is_all_vi.patchDownload
From 15a2b3168f1b38995f8c031528c76d211ba83e32 Mon Sep 17 00:00:00 2001
From: Greg Burd <greg@burd.me>
Date: Tue, 6 May 2025 10:59:13 -0400
Subject: [PATCH v1 1/3] Fix references in comments in
heap_page_is_all_visible().
Update comments to reference the function heap_page_prune_and_freeze()
rather than lazy_scan_prune() resulting from the changes in 6dbb490.
Author: Greg Burd <greg@burd.me>
---
src/backend/access/heap/vacuumlazy.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f28326bad09..e450911424f 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3606,10 +3606,10 @@ dead_items_cleanup(LVRelState *vacrel)
* xmin amongst the visible tuples. Set *all_frozen to true if every tuple
* on this page is frozen.
*
- * This is a stripped down version of lazy_scan_prune(). If you change
- * anything here, make sure that everything stays in sync. Note that an
- * assertion calls us to verify that everybody still agrees. Be sure to avoid
- * introducing new side-effects here.
+ * This is a stripped down version of heap_page_prune_and_freeze() called by
+ * lazy_scan_prune(). If you change anything here, make sure that everything
+ * stays in sync. Note that an assertion calls us to verify that everybody
+ * still agrees. Be sure to avoid introducing new side-effects here.
*/
static bool
heap_page_is_all_visible(LVRelState *vacrel, Buffer buf,
@@ -3670,7 +3670,7 @@ heap_page_is_all_visible(LVRelState *vacrel, Buffer buf,
{
TransactionId xmin;
- /* Check comments in lazy_scan_prune. */
+ /* Check comments in heap_page_prune_and_freeze(). */
if (!HeapTupleHeaderXminCommitted(tuple.t_data))
{
all_visible = false;
--
2.39.5 (Apple Git-154)
v1-0003-Move-ItemPointerSet-call-to-better-match-heap_pag.patchapplication/octet-stream; filename=v1-0003-Move-ItemPointerSet-call-to-better-match-heap_pag.patch; name=v1-0003-Move-ItemPointerSet-call-to-better-match-heap_pag.patchDownload
From 6beaa6c988dabc24c4af34cec9609ab95f3fd109 Mon Sep 17 00:00:00 2001
From: Greg Burd <greg@burd.me>
Date: Tue, 6 May 2025 11:19:29 -0400
Subject: [PATCH v1 3/3] Move ItemPointerSet() call to better match
heap_page_prune_and_freeze().
Avoid unnecessary call in some cases and better align this slimmed down
version of the prune and freeze logic.
Author: Greg Burd <greg@burd.me>
---
src/backend/access/heap/vacuumlazy.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 14cf880340b..2c3bf86ca60 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3649,8 +3649,6 @@ heap_page_is_all_visible(LVRelState *vacrel, Buffer buf,
if (!ItemIdIsUsed(itemid) || ItemIdIsRedirected(itemid))
continue;
- ItemPointerSet(&(tuple.t_self), blockno, offnum);
-
/*
* Dead line pointers can have index pointers pointing to them. So
* they can't be treated as visible
@@ -3667,6 +3665,7 @@ heap_page_is_all_visible(LVRelState *vacrel, Buffer buf,
tuple.t_data = (HeapTupleHeader) PageGetItem(page, itemid);
tuple.t_len = ItemIdGetLength(itemid);
tuple.t_tableOid = RelationGetRelid(vacrel->rel);
+ ItemPointerSet(&(tuple.t_self), blockno, offnum);
switch (HeapTupleSatisfiesVacuum(&tuple, vacrel->cutoffs.OldestXmin,
buf))
--
2.39.5 (Apple Git-154)
On Tue, May 6, 2025 at 11:08 PM Gregory Burd <greg@burd.me> wrote:
While working on [1] I found an outdated comment in
heap_page_is_all_visible() and two other small fixes.0001: Updates that comment so future authors know that this "stripped down
function" should retain the logic in heap_page_prune_and_freeze(), not
lazy_scan_prune() as was the case before 6dbb490.
0002: Mimics the same loop logic as in heap_page_is_all_visible() so as to
a) stay in sync and b) benefit from the mentioned CPU prefetching
optimization.
0003: Moves the ItemSetPointer() just a bit further down in the function
again to a) stay in sync and b) to sometimes avoid that tiny overhead.best,
-greg
PS: per-community standards I've switched to my personal email address
rather than gregburd@amazon.com[1]
/messages/by-id/78574B24-BE0A-42C5-8075-3FA9FA63B8FC@amazon.com
Hi, looks good for me.
Best regards,
Stepan Neretin.
On Tue, May 06, 2025 at 03:39:07PM +0000, Gregory Burd wrote:
0001: Updates that comment so future authors know that this "stripped
down function" should retain the logic in heap_page_prune_and_freeze(),
not lazy_scan_prune() as was the case before 6dbb490.
Hm. It certainly had some resemblance before commit 6dbb490, but looking
at the two code paths now, I'm not sure whether this comment is useful
anymore. I do think it's more accurate to point to
heap_page_prune_and_freeze(), though. And there's still an assertion in
lazy_scan_prune() to make sure things are in agreement. Perhaps we should
rewrite the comment to something like
* This is a specialized version of the logic from
* heap_page_prune_and_freeze(). If you change anything here, make sure
* that everything says in sync. Note that an assertion in
* lazy_scan_prune() calls us to verify that everybody still agrees. Be
* sure to avoid introducing new side effects here.
0002: Mimics the same loop logic as in heap_page_is_all_visible() so as
to a) stay in sync and b) benefit from the mentioned CPU prefetching
optimization.
0003: Moves the ItemSetPointer() just a bit further down in the function
again to a) stay in sync and b) to sometimes avoid that tiny overhead.
These look generally reasonable to me, but they might be v19 material at
this point. Have you been able to measure the impact of 0002? I didn't
see much discussion about this in the thread that changed it for what is
now known as heap_page_prune_and_freeze() [0]/messages/by-id/17255-14c0ac58d0f9b583@postgresql.org.
[0]: /messages/by-id/17255-14c0ac58d0f9b583@postgresql.org
--
nathan