Add macros for ReorderBufferTXN toptxn

Started by Peter Smithalmost 3 years ago16 messages
#1Peter Smith
smithpb2250@gmail.com
1 attachment(s)

Hi,

During a recent code review, I was confused multiple times by the
toptxn member of ReorderBufferTXN, which is defined only for
sub-transactions.

e.g. txn->toptxn member == NULL means the txn is a top level txn.
e.g. txn->toptxn member != NULL means the txn is not a top level txn

It makes sense if you squint and read it slowly enough, but IMO it's
too easy to accidentally misinterpret the meaning when reading code
that uses this member.

~

Such code can be made easier to read just by introducing some simple macros:

#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)

~

PSA a small patch that does this.

(Tests OK using make check-world)

Thoughts?

------
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

v1-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchapplication/octet-stream; name=v1-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchDownload
From ae3312b8669246dc425179e083b61602d197dfc6 Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Fri, 10 Mar 2023 09:03:15 +1100
Subject: [PATCH v1] Add macros for ReorderBufferTXN toptxn.

Make toptxn related code more readable by introducing some simple macros.
---
 contrib/test_decoding/test_decoding.c           |  4 +--
 src/backend/replication/logical/reorderbuffer.c | 38 +++++++++++--------------
 src/backend/replication/pgoutput/pgoutput.c     |  6 ++--
 src/include/replication/reorderbuffer.h         |  3 ++
 4 files changed, 24 insertions(+), 27 deletions(-)

diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index b7e6048..d8710ee 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -815,11 +815,11 @@ pg_decode_stream_abort(LogicalDecodingContext *ctx,
 	 * maintain the output_plugin_private only under the toptxn so if this is
 	 * not the toptxn then fetch the toptxn.
 	 */
-	ReorderBufferTXN *toptxn = txn->toptxn ? txn->toptxn : txn;
+	ReorderBufferTXN *toptxn = get_toptxn(txn);
 	TestDecodingTxnData *txndata = toptxn->output_plugin_private;
 	bool		xact_wrote_changes = txndata->xact_wrote_changes;
 
-	if (txn->toptxn == NULL)
+	if (isa_toptxn(txn))
 	{
 		Assert(txn->output_plugin_private != NULL);
 		pfree(txndata);
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2d17c55..3e2edc9 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -717,10 +717,7 @@ ReorderBufferProcessPartialChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
 		return;
 
 	/* Get the top transaction. */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = get_toptxn(txn);
 
 	/*
 	 * Indicate a partial change for toast inserts.  The change will be
@@ -812,10 +809,7 @@ ReorderBufferQueueChange(ReorderBuffer *rb, TransactionId xid, XLogRecPtr lsn,
 		ReorderBufferTXN *toptxn;
 
 		/* get the top transaction */
-		if (txn->toptxn != NULL)
-			toptxn = txn->toptxn;
-		else
-			toptxn = txn;
+		toptxn = get_toptxn(txn);
 
 		toptxn->txn_flags |= RBTXN_HAS_STREAMABLE_CHANGE;
 	}
@@ -1667,7 +1661,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb, ReorderBufferTXN *txn, bool txn_prep
 	 * about the toplevel xact (we send the XID in all messages), but we never
 	 * stream XIDs of empty subxacts.
 	 */
-	if ((!txn_prepared) && ((!txn->toptxn) || (txn->nentries_mem != 0)))
+	if ((!txn_prepared) && (isa_toptxn(txn) || (txn->nentries_mem != 0)))
 		txn->txn_flags |= RBTXN_IS_STREAMED;
 
 	if (txn_prepared)
@@ -3207,10 +3201,7 @@ ReorderBufferChangeMemoryUpdate(ReorderBuffer *rb,
 	 * Update the total size in top level as well. This is later used to
 	 * compute the decoding stats.
 	 */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = get_toptxn(txn);
 
 	if (addition)
 	{
@@ -3295,8 +3286,8 @@ ReorderBufferAddInvalidations(ReorderBuffer *rb, TransactionId xid,
 	 * so that we can execute them all together.  See comments atop this
 	 * function.
 	 */
-	if (txn->toptxn)
-		txn = txn->toptxn;
+	if (isa_subtxn(txn))
+		txn = get_toptxn(txn);
 
 	Assert(nmsgs > 0);
 
@@ -3354,7 +3345,6 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 								  XLogRecPtr lsn)
 {
 	ReorderBufferTXN *txn;
-	ReorderBufferTXN *toptxn;
 
 	txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
 
@@ -3370,11 +3360,15 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 	 * conveniently check just top-level transaction and decide whether to
 	 * build the hash table or not.
 	 */
-	toptxn = txn->toptxn;
-	if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+	if (isa_subtxn(txn))
 	{
-		toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-		dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		ReorderBufferTXN *toptxn = get_toptxn(txn);
+
+		if (!rbtxn_has_catalog_changes(toptxn))
+		{
+			toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
+			dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		}
 	}
 }
 
@@ -3619,7 +3613,7 @@ ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
 			(txn = ReorderBufferLargestStreamableTopTXN(rb)) != NULL)
 		{
 			/* we know there has to be one, because the size is not zero */
-			Assert(txn && !txn->toptxn);
+			Assert(txn && isa_toptxn(txn));
 			Assert(txn->total_size > 0);
 			Assert(rb->size >= txn->total_size);
 
@@ -4007,7 +4001,7 @@ ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
 	bool		txn_is_streamed;
 
 	/* We can never reach here for a subtransaction. */
-	Assert(txn->toptxn == NULL);
+	Assert(isa_toptxn(txn));
 
 	/*
 	 * We can't make any assumptions about base snapshot here, similar to what
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00a2d73..abfeea6 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -694,8 +694,8 @@ maybe_send_schema(LogicalDecodingContext *ctx,
 	if (in_streaming)
 		xid = change->txn->xid;
 
-	if (change->txn->toptxn)
-		topxid = change->txn->toptxn->xid;
+	if (isa_subtxn(change->txn))
+		topxid = get_toptxn(change->txn)->xid;
 	else
 		topxid = xid;
 
@@ -1879,7 +1879,7 @@ pgoutput_stream_abort(struct LogicalDecodingContext *ctx,
 	Assert(!in_streaming);
 
 	/* determine the toplevel transaction */
-	toptxn = (txn->toptxn) ? txn->toptxn : txn;
+	toptxn = get_toptxn(txn);
 
 	Assert(rbtxn_is_streamed(toptxn));
 
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 215d149..f7234ef 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -296,6 +296,9 @@ typedef struct ReorderBufferTXN
 	XLogRecPtr	end_lsn;
 
 	/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
 	struct ReorderBufferTXN *toptxn;
 
 	/*
-- 
1.8.3.1

#2Amit Kapila
amit.kapila16@gmail.com
In reply to: Peter Smith (#1)
Re: Add macros for ReorderBufferTXN toptxn

On Fri, Mar 10, 2023 at 4:36 AM Peter Smith <smithpb2250@gmail.com> wrote:

During a recent code review, I was confused multiple times by the
toptxn member of ReorderBufferTXN, which is defined only for
sub-transactions.

e.g. txn->toptxn member == NULL means the txn is a top level txn.
e.g. txn->toptxn member != NULL means the txn is not a top level txn

It makes sense if you squint and read it slowly enough, but IMO it's
too easy to accidentally misinterpret the meaning when reading code
that uses this member.

~

Such code can be made easier to read just by introducing some simple macros:

#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)

~

PSA a small patch that does this.

I also find it will make code easier to read. So, +1 to the idea. I'll
do the detailed review and test next week unless there are objections
to the idea.

--
With Regards,
Amit Kapila.

#3vignesh C
vignesh21@gmail.com
In reply to: Peter Smith (#1)
Re: Add macros for ReorderBufferTXN toptxn

On Fri, 10 Mar 2023 at 04:36, Peter Smith <smithpb2250@gmail.com> wrote:

Hi,

During a recent code review, I was confused multiple times by the
toptxn member of ReorderBufferTXN, which is defined only for
sub-transactions.

e.g. txn->toptxn member == NULL means the txn is a top level txn.
e.g. txn->toptxn member != NULL means the txn is not a top level txn

It makes sense if you squint and read it slowly enough, but IMO it's
too easy to accidentally misinterpret the meaning when reading code
that uses this member.

~

Such code can be made easier to read just by introducing some simple macros:

#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)

~

PSA a small patch that does this.

(Tests OK using make check-world)

Thoughts?

Few comments:
1) Can we move the macros along with the other macros present in this
file, just above this structure, similar to the macros added for
txn_flags:
        /* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
2) The macro name can be changed to rbtxn_is_toptxn, rbtxn_is_subtxn
and rbtxn_get_toptxn to keep it consistent with others:
        /* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
3) We could add separate comments for each of the macros:
        /* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Regards,
Vignesh

#4Peter Smith
smithpb2250@gmail.com
In reply to: vignesh C (#3)
1 attachment(s)
Re: Add macros for ReorderBufferTXN toptxn

Thanks for the review!

On Mon, Mar 13, 2023 at 6:19 PM vignesh C <vignesh21@gmail.com> wrote:
...

Few comments:
1) Can we move the macros along with the other macros present in this
file, just above this structure, similar to the macros added for
txn_flags:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
2) The macro name can be changed to rbtxn_is_toptxn, rbtxn_is_subtxn
and rbtxn_get_toptxn to keep it consistent with others:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
3) We could add separate comments for each of the macros:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)

All the above are fixed as suggested.

4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Yeah, that is true, but I chose not to keep the original assignment in
this case mainly because then it is consistent with the other changed
code --- e.g. Every other direct member assignment/access of the
'toptxn' member now hides behind the macros so I did not want this
single place to be the odd one out. TBH, I don't think 1 extra check
is of any significance, but it is not a problem to change like you
suggested if other people also want it done that way.

------
Kind Regards,
Peter Smith.
Fujitsu Australia.

Attachments:

v2-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchapplication/octet-stream; name=v2-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchDownload
From 38de68f8ab7be9857e2c922db351f8d0d8c4a656 Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Tue, 14 Mar 2023 16:33:24 +1100
Subject: [PATCH v2] Add macros for ReorderBufferTXN toptxn.

Make toptxn related code more readable by introducing some simple macros.
---
 contrib/test_decoding/test_decoding.c           |  4 +--
 src/backend/replication/logical/reorderbuffer.c | 38 +++++++++++--------------
 src/backend/replication/pgoutput/pgoutput.c     |  6 ++--
 src/include/replication/reorderbuffer.h         | 18 ++++++++++++
 4 files changed, 39 insertions(+), 27 deletions(-)

diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index b7e6048..628c6a2 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -815,11 +815,11 @@ pg_decode_stream_abort(LogicalDecodingContext *ctx,
 	 * maintain the output_plugin_private only under the toptxn so if this is
 	 * not the toptxn then fetch the toptxn.
 	 */
-	ReorderBufferTXN *toptxn = txn->toptxn ? txn->toptxn : txn;
+	ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
 	TestDecodingTxnData *txndata = toptxn->output_plugin_private;
 	bool		xact_wrote_changes = txndata->xact_wrote_changes;
 
-	if (txn->toptxn == NULL)
+	if (rbtxn_is_toptxn(txn))
 	{
 		Assert(txn->output_plugin_private != NULL);
 		pfree(txndata);
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2d17c55..fe15762 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -717,10 +717,7 @@ ReorderBufferProcessPartialChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
 		return;
 
 	/* Get the top transaction. */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	/*
 	 * Indicate a partial change for toast inserts.  The change will be
@@ -812,10 +809,7 @@ ReorderBufferQueueChange(ReorderBuffer *rb, TransactionId xid, XLogRecPtr lsn,
 		ReorderBufferTXN *toptxn;
 
 		/* get the top transaction */
-		if (txn->toptxn != NULL)
-			toptxn = txn->toptxn;
-		else
-			toptxn = txn;
+		toptxn = rbtxn_get_toptxn(txn);
 
 		toptxn->txn_flags |= RBTXN_HAS_STREAMABLE_CHANGE;
 	}
@@ -1667,7 +1661,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb, ReorderBufferTXN *txn, bool txn_prep
 	 * about the toplevel xact (we send the XID in all messages), but we never
 	 * stream XIDs of empty subxacts.
 	 */
-	if ((!txn_prepared) && ((!txn->toptxn) || (txn->nentries_mem != 0)))
+	if ((!txn_prepared) && (rbtxn_is_toptxn(txn) || (txn->nentries_mem != 0)))
 		txn->txn_flags |= RBTXN_IS_STREAMED;
 
 	if (txn_prepared)
@@ -3207,10 +3201,7 @@ ReorderBufferChangeMemoryUpdate(ReorderBuffer *rb,
 	 * Update the total size in top level as well. This is later used to
 	 * compute the decoding stats.
 	 */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	if (addition)
 	{
@@ -3295,8 +3286,8 @@ ReorderBufferAddInvalidations(ReorderBuffer *rb, TransactionId xid,
 	 * so that we can execute them all together.  See comments atop this
 	 * function.
 	 */
-	if (txn->toptxn)
-		txn = txn->toptxn;
+	if (rbtxn_is_subtxn(txn))
+		txn = rbtxn_get_toptxn(txn);
 
 	Assert(nmsgs > 0);
 
@@ -3354,7 +3345,6 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 								  XLogRecPtr lsn)
 {
 	ReorderBufferTXN *txn;
-	ReorderBufferTXN *toptxn;
 
 	txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
 
@@ -3370,11 +3360,15 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 	 * conveniently check just top-level transaction and decide whether to
 	 * build the hash table or not.
 	 */
-	toptxn = txn->toptxn;
-	if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+	if (rbtxn_is_subtxn(txn))
 	{
-		toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-		dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
+
+		if (!rbtxn_has_catalog_changes(toptxn))
+		{
+			toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
+			dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		}
 	}
 }
 
@@ -3619,7 +3613,7 @@ ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
 			(txn = ReorderBufferLargestStreamableTopTXN(rb)) != NULL)
 		{
 			/* we know there has to be one, because the size is not zero */
-			Assert(txn && !txn->toptxn);
+			Assert(txn && rbtxn_is_toptxn(txn));
 			Assert(txn->total_size > 0);
 			Assert(rb->size >= txn->total_size);
 
@@ -4007,7 +4001,7 @@ ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
 	bool		txn_is_streamed;
 
 	/* We can never reach here for a subtransaction. */
-	Assert(txn->toptxn == NULL);
+	Assert(rbtxn_is_toptxn(txn));
 
 	/*
 	 * We can't make any assumptions about base snapshot here, similar to what
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00a2d73..3a2d2e3 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -694,8 +694,8 @@ maybe_send_schema(LogicalDecodingContext *ctx,
 	if (in_streaming)
 		xid = change->txn->xid;
 
-	if (change->txn->toptxn)
-		topxid = change->txn->toptxn->xid;
+	if (rbtxn_is_subtxn(change->txn))
+		topxid = rbtxn_get_toptxn(change->txn)->xid;
 	else
 		topxid = xid;
 
@@ -1879,7 +1879,7 @@ pgoutput_stream_abort(struct LogicalDecodingContext *ctx,
 	Assert(!in_streaming);
 
 	/* determine the toplevel transaction */
-	toptxn = (txn->toptxn) ? txn->toptxn : txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	Assert(rbtxn_is_streamed(toptxn));
 
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 215d149..ea1e271 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -249,6 +249,24 @@ typedef struct ReorderBufferChange
 	((txn)->txn_flags & RBTXN_SKIPPED_PREPARE) != 0 \
 )
 
+/* Is this a top-level transaction? */
+#define rbtxn_is_toptxn(txn)\
+(\
+	(txn)->toptxn == NULL\
+)
+
+/* Is this a sub-transaction? */
+#define rbtxn_is_subtxn(txn)\
+(\
+	(txn)->toptxn != NULL\
+)
+
+/* Get the top-level transaction of this (sub-)transaction. */
+#define rbtxn_get_toptxn(txn)\
+(\
+	rbtxn_is_subtxn(txn) ? (txn)->toptxn : (txn)\
+)
+
 typedef struct ReorderBufferTXN
 {
 	/* See above */
-- 
1.8.3.1

#5vignesh C
vignesh21@gmail.com
In reply to: Peter Smith (#4)
Re: Add macros for ReorderBufferTXN toptxn

On Tue, 14 Mar 2023 at 12:37, Peter Smith <smithpb2250@gmail.com> wrote:

Thanks for the review!

On Mon, Mar 13, 2023 at 6:19 PM vignesh C <vignesh21@gmail.com> wrote:
...

Few comments:
1) Can we move the macros along with the other macros present in this
file, just above this structure, similar to the macros added for
txn_flags:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
2) The macro name can be changed to rbtxn_is_toptxn, rbtxn_is_subtxn
and rbtxn_get_toptxn to keep it consistent with others:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
3) We could add separate comments for each of the macros:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)

All the above are fixed as suggested.

4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Yeah, that is true, but I chose not to keep the original assignment in
this case mainly because then it is consistent with the other changed
code --- e.g. Every other direct member assignment/access of the
'toptxn' member now hides behind the macros so I did not want this
single place to be the odd one out. TBH, I don't think 1 extra check
is of any significance, but it is not a problem to change like you
suggested if other people also want it done that way.

The same issue exists here too:
1)
-       if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+       if (rbtxn_is_subtxn(txn))
        {
-               toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-               dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+               ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
2)
 -       if (change->txn->toptxn)
-               topxid = change->txn->toptxn->xid;
+       if (rbtxn_is_subtxn(change->txn))
+               topxid = rbtxn_get_toptxn(change->txn)->xid;

If you plan to fix, bothe the above also should be handled.

3) The comment on top of rbtxn_get_toptxn could be kept similar in
both the below places. I know it is not because of your change, but
since it is a very small change probably we could include it along
with this patch:
@@ -717,10 +717,7 @@ ReorderBufferProcessPartialChange(ReorderBuffer
*rb, ReorderBufferTXN *txn,
return;

        /* Get the top transaction. */
-       if (txn->toptxn != NULL)
-               toptxn = txn->toptxn;
-       else
-               toptxn = txn;
+       toptxn = rbtxn_get_toptxn(txn);

/*
* Indicate a partial change for toast inserts. The change will be
@@ -812,10 +809,7 @@ ReorderBufferQueueChange(ReorderBuffer *rb,
TransactionId xid, XLogRecPtr lsn,
ReorderBufferTXN *toptxn;

                /* get the top transaction */
-               if (txn->toptxn != NULL)
-                       toptxn = txn->toptxn;
-               else
-                       toptxn = txn;
+               toptxn = rbtxn_get_toptxn(txn);

Regards,
Vignesh

#6Amit Kapila
amit.kapila16@gmail.com
In reply to: Peter Smith (#4)
Re: Add macros for ReorderBufferTXN toptxn

On Tue, Mar 14, 2023 at 12:37 PM Peter Smith <smithpb2250@gmail.com> wrote:

Thanks for the review!

On Mon, Mar 13, 2023 at 6:19 PM vignesh C <vignesh21@gmail.com> wrote:
...

4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Yeah, that is true, but I chose not to keep the original assignment in
this case mainly because then it is consistent with the other changed
code --- e.g. Every other direct member assignment/access of the
'toptxn' member now hides behind the macros so I did not want this
single place to be the odd one out. TBH, I don't think 1 extra check
is of any significance, but it is not a problem to change like you
suggested if other people also want it done that way.

Can't we directly use rbtxn_get_toptxn() for this case? I think that
way code will look neat. I see that it is not exactly matching the
existing check so you might be worried but I feel the new code will
achieve the same purpose and will be easy to follow.

--
With Regards,
Amit Kapila.

#7Amit Kapila
amit.kapila16@gmail.com
In reply to: vignesh C (#5)
Re: Add macros for ReorderBufferTXN toptxn

On Tue, Mar 14, 2023 at 5:03 PM vignesh C <vignesh21@gmail.com> wrote:

On Tue, 14 Mar 2023 at 12:37, Peter Smith <smithpb2250@gmail.com> wrote:

The same issue exists here too:
1)
-       if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+       if (rbtxn_is_subtxn(txn))
{
-               toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-               dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+               ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
2)
-       if (change->txn->toptxn)
-               topxid = change->txn->toptxn->xid;
+       if (rbtxn_is_subtxn(change->txn))
+               topxid = rbtxn_get_toptxn(change->txn)->xid;

If you plan to fix, bothe the above also should be handled.

I don't know if it would be any better to change the above two as
compared to what the proposed patch has.

--
With Regards,
Amit Kapila.

#8Peter Smith
smithpb2250@gmail.com
In reply to: Amit Kapila (#6)
1 attachment(s)
Re: Add macros for ReorderBufferTXN toptxn

On Tue, Mar 14, 2023 at 10:43 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Tue, Mar 14, 2023 at 12:37 PM Peter Smith <smithpb2250@gmail.com> wrote:

Thanks for the review!

On Mon, Mar 13, 2023 at 6:19 PM vignesh C <vignesh21@gmail.com> wrote:
...

4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Yeah, that is true, but I chose not to keep the original assignment in
this case mainly because then it is consistent with the other changed
code --- e.g. Every other direct member assignment/access of the
'toptxn' member now hides behind the macros so I did not want this
single place to be the odd one out. TBH, I don't think 1 extra check
is of any significance, but it is not a problem to change like you
suggested if other people also want it done that way.

Can't we directly use rbtxn_get_toptxn() for this case? I think that
way code will look neat. I see that it is not exactly matching the
existing check so you might be worried but I feel the new code will
achieve the same purpose and will be easy to follow.

OK. Done as suggested.

PSA v3.

------
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

v3-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchapplication/octet-stream; name=v3-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchDownload
From 9b660e136afa6f820af3c3d27841ece4e021121e Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Wed, 15 Mar 2023 09:37:20 +1100
Subject: [PATCH v3] Add macros for ReorderBufferTXN toptxn.

Make toptxn related code more readable by introducing some simple macros.
---
 contrib/test_decoding/test_decoding.c           |  4 +--
 src/backend/replication/logical/reorderbuffer.c | 40 ++++++++++---------------
 src/backend/replication/pgoutput/pgoutput.c     |  6 ++--
 src/include/replication/reorderbuffer.h         | 18 +++++++++++
 4 files changed, 38 insertions(+), 30 deletions(-)

diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index b7e6048..628c6a2 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -815,11 +815,11 @@ pg_decode_stream_abort(LogicalDecodingContext *ctx,
 	 * maintain the output_plugin_private only under the toptxn so if this is
 	 * not the toptxn then fetch the toptxn.
 	 */
-	ReorderBufferTXN *toptxn = txn->toptxn ? txn->toptxn : txn;
+	ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
 	TestDecodingTxnData *txndata = toptxn->output_plugin_private;
 	bool		xact_wrote_changes = txndata->xact_wrote_changes;
 
-	if (txn->toptxn == NULL)
+	if (rbtxn_is_toptxn(txn))
 	{
 		Assert(txn->output_plugin_private != NULL);
 		pfree(txndata);
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2d17c55..0f063b0 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -717,10 +717,7 @@ ReorderBufferProcessPartialChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
 		return;
 
 	/* Get the top transaction. */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	/*
 	 * Indicate a partial change for toast inserts.  The change will be
@@ -809,13 +806,7 @@ ReorderBufferQueueChange(ReorderBuffer *rb, TransactionId xid, XLogRecPtr lsn,
 		change->action == REORDER_BUFFER_CHANGE_TRUNCATE ||
 		change->action == REORDER_BUFFER_CHANGE_MESSAGE)
 	{
-		ReorderBufferTXN *toptxn;
-
-		/* get the top transaction */
-		if (txn->toptxn != NULL)
-			toptxn = txn->toptxn;
-		else
-			toptxn = txn;
+		ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
 
 		toptxn->txn_flags |= RBTXN_HAS_STREAMABLE_CHANGE;
 	}
@@ -1667,7 +1658,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb, ReorderBufferTXN *txn, bool txn_prep
 	 * about the toplevel xact (we send the XID in all messages), but we never
 	 * stream XIDs of empty subxacts.
 	 */
-	if ((!txn_prepared) && ((!txn->toptxn) || (txn->nentries_mem != 0)))
+	if ((!txn_prepared) && (rbtxn_is_toptxn(txn) || (txn->nentries_mem != 0)))
 		txn->txn_flags |= RBTXN_IS_STREAMED;
 
 	if (txn_prepared)
@@ -3207,10 +3198,7 @@ ReorderBufferChangeMemoryUpdate(ReorderBuffer *rb,
 	 * Update the total size in top level as well. This is later used to
 	 * compute the decoding stats.
 	 */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	if (addition)
 	{
@@ -3295,8 +3283,7 @@ ReorderBufferAddInvalidations(ReorderBuffer *rb, TransactionId xid,
 	 * so that we can execute them all together.  See comments atop this
 	 * function.
 	 */
-	if (txn->toptxn)
-		txn = txn->toptxn;
+	txn = rbtxn_get_toptxn(txn);
 
 	Assert(nmsgs > 0);
 
@@ -3354,7 +3341,6 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 								  XLogRecPtr lsn)
 {
 	ReorderBufferTXN *txn;
-	ReorderBufferTXN *toptxn;
 
 	txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
 
@@ -3370,11 +3356,15 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 	 * conveniently check just top-level transaction and decide whether to
 	 * build the hash table or not.
 	 */
-	toptxn = txn->toptxn;
-	if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+	if (rbtxn_is_subtxn(txn))
 	{
-		toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-		dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
+
+		if (!rbtxn_has_catalog_changes(toptxn))
+		{
+			toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
+			dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		}
 	}
 }
 
@@ -3619,7 +3609,7 @@ ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
 			(txn = ReorderBufferLargestStreamableTopTXN(rb)) != NULL)
 		{
 			/* we know there has to be one, because the size is not zero */
-			Assert(txn && !txn->toptxn);
+			Assert(txn && rbtxn_is_toptxn(txn));
 			Assert(txn->total_size > 0);
 			Assert(rb->size >= txn->total_size);
 
@@ -4007,7 +3997,7 @@ ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
 	bool		txn_is_streamed;
 
 	/* We can never reach here for a subtransaction. */
-	Assert(txn->toptxn == NULL);
+	Assert(rbtxn_is_toptxn(txn));
 
 	/*
 	 * We can't make any assumptions about base snapshot here, similar to what
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00a2d73..3a2d2e3 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -694,8 +694,8 @@ maybe_send_schema(LogicalDecodingContext *ctx,
 	if (in_streaming)
 		xid = change->txn->xid;
 
-	if (change->txn->toptxn)
-		topxid = change->txn->toptxn->xid;
+	if (rbtxn_is_subtxn(change->txn))
+		topxid = rbtxn_get_toptxn(change->txn)->xid;
 	else
 		topxid = xid;
 
@@ -1879,7 +1879,7 @@ pgoutput_stream_abort(struct LogicalDecodingContext *ctx,
 	Assert(!in_streaming);
 
 	/* determine the toplevel transaction */
-	toptxn = (txn->toptxn) ? txn->toptxn : txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	Assert(rbtxn_is_streamed(toptxn));
 
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 215d149..08dc9ee 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -249,6 +249,24 @@ typedef struct ReorderBufferChange
 	((txn)->txn_flags & RBTXN_SKIPPED_PREPARE) != 0 \
 )
 
+/* Is this a top-level transaction? */
+#define rbtxn_is_toptxn(txn)\
+(\
+	(txn)->toptxn == NULL\
+)
+
+/* Is this a subtransaction? */
+#define rbtxn_is_subtxn(txn)\
+(\
+	(txn)->toptxn != NULL\
+)
+
+/* Get the top-level transaction of this (sub)transaction. */
+#define rbtxn_get_toptxn(txn)\
+(\
+	rbtxn_is_subtxn(txn) ? (txn)->toptxn : (txn)\
+)
+
 typedef struct ReorderBufferTXN
 {
 	/* See above */
-- 
1.8.3.1

#9Peter Smith
smithpb2250@gmail.com
In reply to: vignesh C (#5)
Re: Add macros for ReorderBufferTXN toptxn

On Tue, Mar 14, 2023 at 10:33 PM vignesh C <vignesh21@gmail.com> wrote:

On Tue, 14 Mar 2023 at 12:37, Peter Smith <smithpb2250@gmail.com> wrote:

Thanks for the review!

On Mon, Mar 13, 2023 at 6:19 PM vignesh C <vignesh21@gmail.com> wrote:
...

Few comments:
1) Can we move the macros along with the other macros present in this
file, just above this structure, similar to the macros added for
txn_flags:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
2) The macro name can be changed to rbtxn_is_toptxn, rbtxn_is_subtxn
and rbtxn_get_toptxn to keep it consistent with others:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)
3) We could add separate comments for each of the macros:
/* Toplevel transaction for this subxact (NULL for top-level). */
+#define isa_toptxn(rbtxn) (rbtxn->toptxn == NULL)
+#define isa_subtxn(rbtxn) (rbtxn->toptxn != NULL)
+#define get_toptxn(rbtxn) (isa_subtxn(rbtxn) ? rbtxn->toptxn : rbtxn)

All the above are fixed as suggested.

4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Yeah, that is true, but I chose not to keep the original assignment in
this case mainly because then it is consistent with the other changed
code --- e.g. Every other direct member assignment/access of the
'toptxn' member now hides behind the macros so I did not want this
single place to be the odd one out. TBH, I don't think 1 extra check
is of any significance, but it is not a problem to change like you
suggested if other people also want it done that way.

The same issue exists here too:
1)
-       if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+       if (rbtxn_is_subtxn(txn))
{
-               toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-               dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+               ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
2)
-       if (change->txn->toptxn)
-               topxid = change->txn->toptxn->xid;
+       if (rbtxn_is_subtxn(change->txn))
+               topxid = rbtxn_get_toptxn(change->txn)->xid;

If you plan to fix, bothe the above also should be handled.

OK, noted. Anyway, for now, I preferred the 'toptxn' member to be
consistently hidden in the code so I don't plan to remove those
macros.

Also, please see Amit's reply [1]Amit reply to your suggestion - /messages/by-id/CAA4eK1+oqfUSC3vpu6bJzgfnSmu_yaeoLS=Rb3416GuS5iRP1Q@mail.gmail.com to your suggestion.

3) The comment on top of rbtxn_get_toptxn could be kept similar in
both the below places. I know it is not because of your change, but
since it is a very small change probably we could include it along
with this patch:
@@ -717,10 +717,7 @@ ReorderBufferProcessPartialChange(ReorderBuffer
*rb, ReorderBufferTXN *txn,
return;

/* Get the top transaction. */
-       if (txn->toptxn != NULL)
-               toptxn = txn->toptxn;
-       else
-               toptxn = txn;
+       toptxn = rbtxn_get_toptxn(txn);

/*
* Indicate a partial change for toast inserts. The change will be
@@ -812,10 +809,7 @@ ReorderBufferQueueChange(ReorderBuffer *rb,
TransactionId xid, XLogRecPtr lsn,
ReorderBufferTXN *toptxn;

/* get the top transaction */
-               if (txn->toptxn != NULL)
-                       toptxn = txn->toptxn;
-               else
-                       toptxn = txn;
+               toptxn = rbtxn_get_toptxn(txn);

IMO the comment ("/* get the top transaction */") was not really
saying anything useful that is not already obvious from the macro name
("rbtxn_get_toptxn"). So I've removed it entirely in your 2nd case.
This change is consistent with other parts of the patch where the
toptxn is just assigned in the declaration.

PSA v3. [2]v3 - /messages/by-id/CAHut+PtrD4xU4OPUB64ZK+DPDhfKn3zph=nDpEWUFFzUvMKo2w@mail.gmail.com

------
[1]: Amit reply to your suggestion - /messages/by-id/CAA4eK1+oqfUSC3vpu6bJzgfnSmu_yaeoLS=Rb3416GuS5iRP1Q@mail.gmail.com
/messages/by-id/CAA4eK1+oqfUSC3vpu6bJzgfnSmu_yaeoLS=Rb3416GuS5iRP1Q@mail.gmail.com
[2]: v3 - /messages/by-id/CAHut+PtrD4xU4OPUB64ZK+DPDhfKn3zph=nDpEWUFFzUvMKo2w@mail.gmail.com

Kind Regards,
Peter Smith.
Fujitsu Australia

#10Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Peter Smith (#8)
Re: Add macros for ReorderBufferTXN toptxn

Hi,

On Wed, Mar 15, 2023 at 8:55 AM Peter Smith <smithpb2250@gmail.com> wrote:

On Tue, Mar 14, 2023 at 10:43 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Tue, Mar 14, 2023 at 12:37 PM Peter Smith <smithpb2250@gmail.com> wrote:

Thanks for the review!

On Mon, Mar 13, 2023 at 6:19 PM vignesh C <vignesh21@gmail.com> wrote:
...

4) We check if txn->toptxn is not null twice here both in if condition
and in the assignment, we could retain the assignment operation as
earlier to remove the 2nd check:
-       if (txn->toptxn)
-               txn = txn->toptxn;
+       if (isa_subtxn(txn))
+               txn = get_toptxn(txn);
We could avoid one check again by:
+       if (isa_subtxn(txn))
+               txn = txn->toptxn;

Yeah, that is true, but I chose not to keep the original assignment in
this case mainly because then it is consistent with the other changed
code --- e.g. Every other direct member assignment/access of the
'toptxn' member now hides behind the macros so I did not want this
single place to be the odd one out. TBH, I don't think 1 extra check
is of any significance, but it is not a problem to change like you
suggested if other people also want it done that way.

Can't we directly use rbtxn_get_toptxn() for this case? I think that
way code will look neat. I see that it is not exactly matching the
existing check so you might be worried but I feel the new code will
achieve the same purpose and will be easy to follow.

OK. Done as suggested.

+1 to the idea. Here are some minor comments:

@@ -1667,7 +1658,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb,
ReorderBufferTXN *txn, bool txn_prep
  * about the toplevel xact (we send the XID in all messages), but we never
  * stream XIDs of empty subxacts.
  */
- if ((!txn_prepared) && ((!txn->toptxn) || (txn->nentries_mem != 0)))
+ if ((!txn_prepared) && (rbtxn_is_toptxn(txn) || (txn->nentries_mem != 0)))
  txn->txn_flags |= RBTXN_IS_STREAMED;

Probably the following comment of the above lines also needs to be updated?

* The toplevel transaction, identified by (toptxn==NULL), is marked as
* streamed always,

---
+/* Is this a top-level transaction? */
+#define rbtxn_is_toptxn(txn)\
+(\
+ (txn)->toptxn == NULL\
+)
+
+/* Is this a subtransaction? */
+#define rbtxn_is_subtxn(txn)\
+(\
+ (txn)->toptxn != NULL\
+)
+
+/* Get the top-level transaction of this (sub)transaction. */
+#define rbtxn_get_toptxn(txn)\
+(\
+ rbtxn_is_subtxn(txn) ? (txn)->toptxn : (txn)\
+)

We need a whitespace before backslashes.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#11Peter Smith
smithpb2250@gmail.com
In reply to: Masahiko Sawada (#10)
1 attachment(s)
Re: Add macros for ReorderBufferTXN toptxn

On Wed, Mar 15, 2023 at 4:55 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

Hi,

...

+1 to the idea. Here are some minor comments:

@@ -1667,7 +1658,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb,
ReorderBufferTXN *txn, bool txn_prep
* about the toplevel xact (we send the XID in all messages), but we never
* stream XIDs of empty subxacts.
*/
- if ((!txn_prepared) && ((!txn->toptxn) || (txn->nentries_mem != 0)))
+ if ((!txn_prepared) && (rbtxn_is_toptxn(txn) || (txn->nentries_mem != 0)))
txn->txn_flags |= RBTXN_IS_STREAMED;

Probably the following comment of the above lines also needs to be updated?

* The toplevel transaction, identified by (toptxn==NULL), is marked as
* streamed always,

---
+/* Is this a top-level transaction? */
+#define rbtxn_is_toptxn(txn)\
+(\
+ (txn)->toptxn == NULL\
+)
+
+/* Is this a subtransaction? */
+#define rbtxn_is_subtxn(txn)\
+(\
+ (txn)->toptxn != NULL\
+)
+
+/* Get the top-level transaction of this (sub)transaction. */
+#define rbtxn_get_toptxn(txn)\
+(\
+ rbtxn_is_subtxn(txn) ? (txn)->toptxn : (txn)\
+)

We need a whitespace before backslashes.

Thanks for your interest in my patch.

PSA v4 which addresses both of your review comments.

------
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

v4-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchapplication/octet-stream; name=v4-0001-Add-macros-for-ReorderBufferTXN-toptxn.patchDownload
From 8318ccb4c2cea70280ef1142f76773cf40267427 Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Thu, 16 Mar 2023 12:00:50 +1100
Subject: [PATCH v4] Add macros for ReorderBufferTXN toptxn.

Make toptxn related code more readable by introducing some simple macros.
---
 contrib/test_decoding/test_decoding.c           |  4 +--
 src/backend/replication/logical/reorderbuffer.c | 46 ++++++++++---------------
 src/backend/replication/pgoutput/pgoutput.c     |  6 ++--
 src/include/replication/reorderbuffer.h         | 18 ++++++++++
 4 files changed, 41 insertions(+), 33 deletions(-)

diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index b7e6048..628c6a2 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -815,11 +815,11 @@ pg_decode_stream_abort(LogicalDecodingContext *ctx,
 	 * maintain the output_plugin_private only under the toptxn so if this is
 	 * not the toptxn then fetch the toptxn.
 	 */
-	ReorderBufferTXN *toptxn = txn->toptxn ? txn->toptxn : txn;
+	ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
 	TestDecodingTxnData *txndata = toptxn->output_plugin_private;
 	bool		xact_wrote_changes = txndata->xact_wrote_changes;
 
-	if (txn->toptxn == NULL)
+	if (rbtxn_is_toptxn(txn))
 	{
 		Assert(txn->output_plugin_private != NULL);
 		pfree(txndata);
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2d17c55..6714258 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -717,10 +717,7 @@ ReorderBufferProcessPartialChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
 		return;
 
 	/* Get the top transaction. */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	/*
 	 * Indicate a partial change for toast inserts.  The change will be
@@ -809,13 +806,7 @@ ReorderBufferQueueChange(ReorderBuffer *rb, TransactionId xid, XLogRecPtr lsn,
 		change->action == REORDER_BUFFER_CHANGE_TRUNCATE ||
 		change->action == REORDER_BUFFER_CHANGE_MESSAGE)
 	{
-		ReorderBufferTXN *toptxn;
-
-		/* get the top transaction */
-		if (txn->toptxn != NULL)
-			toptxn = txn->toptxn;
-		else
-			toptxn = txn;
+		ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
 
 		toptxn->txn_flags |= RBTXN_HAS_STREAMABLE_CHANGE;
 	}
@@ -1655,9 +1646,9 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb, ReorderBufferTXN *txn, bool txn_prep
 	/*
 	 * Mark the transaction as streamed.
 	 *
-	 * The toplevel transaction, identified by (toptxn==NULL), is marked as
-	 * streamed always, even if it does not contain any changes (that is, when
-	 * all the changes are in subtransactions).
+	 * The top-level transaction, is marked as streamed always, even if it does
+	 * not contain any changes (that is, when all the changes are in
+	 * subtransactions).
 	 *
 	 * For subtransactions, we only mark them as streamed when there are
 	 * changes in them.
@@ -1667,7 +1658,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb, ReorderBufferTXN *txn, bool txn_prep
 	 * about the toplevel xact (we send the XID in all messages), but we never
 	 * stream XIDs of empty subxacts.
 	 */
-	if ((!txn_prepared) && ((!txn->toptxn) || (txn->nentries_mem != 0)))
+	if ((!txn_prepared) && (rbtxn_is_toptxn(txn) || (txn->nentries_mem != 0)))
 		txn->txn_flags |= RBTXN_IS_STREAMED;
 
 	if (txn_prepared)
@@ -3207,10 +3198,7 @@ ReorderBufferChangeMemoryUpdate(ReorderBuffer *rb,
 	 * Update the total size in top level as well. This is later used to
 	 * compute the decoding stats.
 	 */
-	if (txn->toptxn != NULL)
-		toptxn = txn->toptxn;
-	else
-		toptxn = txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	if (addition)
 	{
@@ -3295,8 +3283,7 @@ ReorderBufferAddInvalidations(ReorderBuffer *rb, TransactionId xid,
 	 * so that we can execute them all together.  See comments atop this
 	 * function.
 	 */
-	if (txn->toptxn)
-		txn = txn->toptxn;
+	txn = rbtxn_get_toptxn(txn);
 
 	Assert(nmsgs > 0);
 
@@ -3354,7 +3341,6 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 								  XLogRecPtr lsn)
 {
 	ReorderBufferTXN *txn;
-	ReorderBufferTXN *toptxn;
 
 	txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
 
@@ -3370,11 +3356,15 @@ ReorderBufferXidSetCatalogChanges(ReorderBuffer *rb, TransactionId xid,
 	 * conveniently check just top-level transaction and decide whether to
 	 * build the hash table or not.
 	 */
-	toptxn = txn->toptxn;
-	if (toptxn != NULL && !rbtxn_has_catalog_changes(toptxn))
+	if (rbtxn_is_subtxn(txn))
 	{
-		toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
-		dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
+
+		if (!rbtxn_has_catalog_changes(toptxn))
+		{
+			toptxn->txn_flags |= RBTXN_HAS_CATALOG_CHANGES;
+			dclist_push_tail(&rb->catchange_txns, &toptxn->catchange_node);
+		}
 	}
 }
 
@@ -3619,7 +3609,7 @@ ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
 			(txn = ReorderBufferLargestStreamableTopTXN(rb)) != NULL)
 		{
 			/* we know there has to be one, because the size is not zero */
-			Assert(txn && !txn->toptxn);
+			Assert(txn && rbtxn_is_toptxn(txn));
 			Assert(txn->total_size > 0);
 			Assert(rb->size >= txn->total_size);
 
@@ -4007,7 +3997,7 @@ ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
 	bool		txn_is_streamed;
 
 	/* We can never reach here for a subtransaction. */
-	Assert(txn->toptxn == NULL);
+	Assert(rbtxn_is_toptxn(txn));
 
 	/*
 	 * We can't make any assumptions about base snapshot here, similar to what
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 00a2d73..3a2d2e3 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -694,8 +694,8 @@ maybe_send_schema(LogicalDecodingContext *ctx,
 	if (in_streaming)
 		xid = change->txn->xid;
 
-	if (change->txn->toptxn)
-		topxid = change->txn->toptxn->xid;
+	if (rbtxn_is_subtxn(change->txn))
+		topxid = rbtxn_get_toptxn(change->txn)->xid;
 	else
 		topxid = xid;
 
@@ -1879,7 +1879,7 @@ pgoutput_stream_abort(struct LogicalDecodingContext *ctx,
 	Assert(!in_streaming);
 
 	/* determine the toplevel transaction */
-	toptxn = (txn->toptxn) ? txn->toptxn : txn;
+	toptxn = rbtxn_get_toptxn(txn);
 
 	Assert(rbtxn_is_streamed(toptxn));
 
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 215d149..e37f512 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -249,6 +249,24 @@ typedef struct ReorderBufferChange
 	((txn)->txn_flags & RBTXN_SKIPPED_PREPARE) != 0 \
 )
 
+/* Is this a top-level transaction? */
+#define rbtxn_is_toptxn(txn) \
+( \
+	(txn)->toptxn == NULL \
+)
+
+/* Is this a subtransaction? */
+#define rbtxn_is_subtxn(txn) \
+( \
+	(txn)->toptxn != NULL \
+)
+
+/* Get the top-level transaction of this (sub)transaction. */
+#define rbtxn_get_toptxn(txn) \
+( \
+	rbtxn_is_subtxn(txn) ? (txn)->toptxn : (txn) \
+)
+
 typedef struct ReorderBufferTXN
 {
 	/* See above */
-- 
1.8.3.1

#12Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Peter Smith (#11)
Re: Add macros for ReorderBufferTXN toptxn

On Thu, Mar 16, 2023 at 7:20 AM Peter Smith <smithpb2250@gmail.com> wrote:

PSA v4 which addresses both of your review comments.

Looks like a reasonable change to me.

A nitpick: how about using rbtxn_get_toptxn instead of an explicit
variable toptxn for single use?

1.
Change
ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
TestDecodingTxnData *txndata = toptxn->output_plugin_private;

To
TestDecodingTxnData *txndata = rbtxn_get_toptxn(txn)->output_plugin_private;

2.
Change
ReorderBufferTXN *toptxn = rbtxn_get_toptxn(txn);
toptxn->txn_flags |= RBTXN_HAS_STREAMABLE_CHANGE;

To
rbtxn_get_toptxn(txn)->txn_flags |= RBTXN_HAS_STREAMABLE_CHANGE;

3.
Change
/*
* Update the total size in top level as well. This is later used to
* compute the decoding stats.
*/
toptxn = rbtxn_get_toptxn(txn);

if (addition)
{
txn->size += sz;
rb->size += sz;

/* Update the total size in the top transaction. */
toptxn->total_size += sz;
}
else
{
Assert((rb->size >= sz) && (txn->size >= sz));
txn->size -= sz;
rb->size -= sz;

/* Update the total size in the top transaction. */
toptxn->total_size -= sz;
}

To

/*
* Update the total size in top level as well. This is later used to
* compute the decoding stats.
*/
if (addition)
{
txn->size += sz;
rb->size += sz;
rbtxn_get_toptxn(txn)->total_size += sz;
}
else
{
Assert((rb->size >= sz) && (txn->size >= sz));
txn->size -= sz;
rb->size -= sz;
rbtxn_get_toptxn(txn)->total_size -= sz;
}

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#13Amit Kapila
amit.kapila16@gmail.com
In reply to: Bharath Rupireddy (#12)
Re: Add macros for ReorderBufferTXN toptxn

On Thu, Mar 16, 2023 at 10:40 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Thu, Mar 16, 2023 at 7:20 AM Peter Smith <smithpb2250@gmail.com> wrote:

PSA v4 which addresses both of your review comments.

Looks like a reasonable change to me.

A nitpick: how about using rbtxn_get_toptxn instead of an explicit
variable toptxn for single use?

I find all three suggestions are similar. Personally, I think the
current code looks better. The v4 patch LGTM and I am planning to
commit it unless there are more comments or people find your
suggestions as an improvement.

--
With Regards,
Amit Kapila.

#14Amit Kapila
amit.kapila16@gmail.com
In reply to: Peter Smith (#11)
Re: Add macros for ReorderBufferTXN toptxn

On Thu, Mar 16, 2023 at 7:20 AM Peter Smith <smithpb2250@gmail.com> wrote:

PSA v4 which addresses both of your review comments.

Pushed.

--
With Regards,
Amit Kapila.

#15Peter Smith
smithpb2250@gmail.com
In reply to: Amit Kapila (#14)
Re: Add macros for ReorderBufferTXN toptxn

The build-farm was OK for the last 18hrs after this push, except there
was one error on mamba [1]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&amp;dt=2023-03-17%2005%3A36%3A10 in test-decoding-check.

This patch did change the test_decoding.c file, so it seems an
unlikely coincidence, but OTOH the change was very small and I don't
see yet how it could have caused a problem here but nowhere else.

Although, mamba has since passed again since that failure.

Any thoughts?

------
[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&amp;dt=2023-03-17%2005%3A36%3A10

Kind Regards,
Peter Smith.
Fujitsu Australia

#16Peter Smith
smithpb2250@gmail.com
In reply to: Peter Smith (#15)
Re: Add macros for ReorderBufferTXN toptxn

On Sat, Mar 18, 2023 at 8:47 AM Peter Smith <smithpb2250@gmail.com> wrote:

The build-farm was OK for the last 18hrs after this push, except there
was one error on mamba [1] in test-decoding-check.

This patch did change the test_decoding.c file, so it seems an
unlikely coincidence, but OTOH the change was very small and I don't
see yet how it could have caused a problem here but nowhere else.

Although, mamba has since passed again since that failure.

Any thoughts?

------
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mamba&amp;dt=2023-03-17%2005%3A36%3A10

Subsequent testing with this "toptxn" patch reverted [1]/messages/by-id/CAHut+PvVrjwJm_9ZqnXJk4x9k8dN0dYrV+T5_Rd30BSneDhv1A@mail.gmail.com was able to
reproduce the same error, so it seems this "toptxn" patch is unrelated
to the reported build-farm failure.

[1]: /messages/by-id/CAHut+PvVrjwJm_9ZqnXJk4x9k8dN0dYrV+T5_Rd30BSneDhv1A@mail.gmail.com

Kind Regards,
Peter Smith.
Fujitsu Australia