Add common function ReplicationOriginName.

Started by Peter Smithover 3 years ago17 messages
#1Peter Smith
smithpb2250@gmail.com
1 attachment(s)

Hi.

There are already multiple places that are building the subscription
origin name, and there are more coming with some new patches [1]/messages/by-id/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com doing
the same:

e.g.
snprintf(originname, sizeof(originname), "pg_%u", subid);

~~

IMO it is better to encapsulate this name formatting in common code
instead of the format string being scattered around the place.

PSA a patch to add a common function ReplicationOriginName. This is
the equivalent of a similar function for tablesync
(ReplicationOriginNameForTablesync) which already existed.

------
[1]: /messages/by-id/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com

Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

v1-0001-Add-common-function-ReplicationOriginName.patchapplication/octet-stream; name=v1-0001-Add-common-function-ReplicationOriginName.patchDownload
From a95f2907b3805b51c8480f56c5b21ca286970c55 Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 19 Sep 2022 12:21:56 +1000
Subject: [PATCH v1] Add common function ReplicationOriginName.

---
 src/backend/commands/subscriptioncmds.c   |  6 +++---
 src/backend/replication/logical/worker.c  | 13 ++++++++++++-
 src/include/replication/worker_internal.h |  1 +
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index d042abe..8166b3c 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -659,7 +659,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
 
 	recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
 
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginName(subid, originname, sizeof(originname));
 	replorigin_create(originname);
 
 	/*
@@ -1317,7 +1317,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
 					char		originname[NAMEDATALEN];
 					XLogRecPtr	remote_lsn;
 
-					snprintf(originname, sizeof(originname), "pg_%u", subid);
+					ReplicationOriginName(subid, originname, sizeof(originname));
 					originid = replorigin_by_name(originname, false);
 					remote_lsn = replorigin_get_progress(originid, false);
 
@@ -1535,7 +1535,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
 	RemoveSubscriptionRel(subid, InvalidOid);
 
 	/* Remove the origin tracking if exists. */
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginName(subid, originname, sizeof(originname));
 	replorigin_drop_by_name(originname, true, false);
 
 	/*
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index eaca406..7e77de5 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -364,6 +364,17 @@ static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr ls
 static inline void reset_apply_error_context_info(void);
 
 /*
+ * Form the origin name for subscription.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginName(Oid suboid, char *originname, int szname)
+{
+	snprintf(originname, szname, "pg_%u", suboid);
+}
+
+/*
  * Should this worker apply changes for given relation.
  *
  * This is mainly needed for initial relation data sync as that runs in
@@ -3706,7 +3717,7 @@ ApplyWorkerMain(Datum main_arg)
 
 		/* Setup replication origin tracking. */
 		StartTransactionCommand();
-		snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+		ReplicationOriginName(MySubscription->oid, originname, sizeof(originname));
 		originid = replorigin_by_name(originname, true);
 		if (!OidIsValid(originid))
 			originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 901845a..3714782 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,6 +92,7 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
 
 extern int	logicalrep_sync_worker_count(Oid subid);
 
+extern void ReplicationOriginName(Oid suboid, char *originname, int szname);
 extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
 											  char *originname, int szorgname);
 extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
-- 
1.8.3.1

#2Aleksander Alekseev
aleksander@timescale.com
In reply to: Peter Smith (#1)
Re: Add common function ReplicationOriginName.

Hi Peter,

PSA a patch to add a common function ReplicationOriginName

The patch looks good to me.

One nitpick I have is that the 2nd argument of snprintf is size_t
while we are passing int's. Your patch is consistent with the current
implementation of ReplicationOriginNameForTablesync() and similar
functions in tablesync.c. However I would like to mention this in case
the committer will be interested in replacing ints with Size's while
on it.

--
Best regards,
Aleksander Alekseev

#3Amit Kapila
amit.kapila16@gmail.com
In reply to: Aleksander Alekseev (#2)
Re: Add common function ReplicationOriginName.

On Mon, Sep 19, 2022 at 2:27 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:

Hi Peter,

PSA a patch to add a common function ReplicationOriginName

The patch looks good to me.

One nitpick I have is that the 2nd argument of snprintf is size_t
while we are passing int's. Your patch is consistent with the current
implementation of ReplicationOriginNameForTablesync() and similar
functions in tablesync.c.

I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.

--
With Regards,
Amit Kapila.

#4Aleksander Alekseev
aleksander@timescale.com
In reply to: Amit Kapila (#3)
2 attachment(s)
Re: Add common function ReplicationOriginName.

Hi Amit,

I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.

OK, here is the updated patchset.

* 0001 replaces int's with Size's in the existing code
* 0002 applies Peter's patch on top of 0001

--
Best regards,
Aleksander Alekseev

Attachments:

v2-0001-Pass-Size-as-a-2nd-argument-for-snprintf-in-table.patchapplication/octet-stream; name=v2-0001-Pass-Size-as-a-2nd-argument-for-snprintf-in-table.patchDownload
From a9e96b297d0e00ab3d117de81d0796c46e46cb7a Mon Sep 17 00:00:00 2001
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 20 Sep 2022 11:17:45 +0300
Subject: [PATCH v2 1/2] Pass Size as a 2nd argument for snprintf() in
 tablesync.c

Previously the following snprintf() wrappers:

* ReplicationSlotNameForTablesync()
* ReplicationOriginNameForTablesync()

... used int as a second argument of snprintf() while the actual type of it
is size_t. Although it doesn't fail at present better replace it with Size
for consistency with the rest of the system.

Aleksander Alekseev, reviewed by TODO FIXME
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
 src/backend/replication/logical/tablesync.c | 4 ++--
 src/include/replication/slot.h              | 2 +-
 src/include/replication/worker_internal.h   | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 831d42016c..8eff69c7de 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -1187,7 +1187,7 @@ copy_table(Relation rel)
  */
 void
 ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
-								char *syncslotname, int szslot)
+								char *syncslotname, Size szslot)
 {
 	snprintf(syncslotname, szslot, "pg_%u_sync_%u_" UINT64_FORMAT, suboid,
 			 relid, GetSystemIdentifier());
@@ -1200,7 +1200,7 @@ ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
  */
 void
 ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
-								  char *originname, int szorgname)
+								  char *originname, Size szorgname)
 {
 	snprintf(originname, szorgname, "pg_%u_%u", suboid, relid);
 }
diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h
index 81e31f002a..8d5e764aef 100644
--- a/src/include/replication/slot.h
+++ b/src/include/replication/slot.h
@@ -218,7 +218,7 @@ extern void ReplicationSlotsDropDBSlots(Oid dboid);
 extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno);
 extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock);
 extern int	ReplicationSlotIndex(ReplicationSlot *slot);
-extern void ReplicationSlotNameForTablesync(Oid suboid, Oid relid, char *syncslotname, int szslot);
+extern void ReplicationSlotNameForTablesync(Oid suboid, Oid relid, char *syncslotname, Size szslot);
 extern void ReplicationSlotDropAtPubNode(WalReceiverConn *wrconn, char *slotname, bool missing_ok);
 
 extern void StartupReplicationSlots(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 901845abc2..f82bc518c3 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -93,7 +93,7 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
 extern int	logicalrep_sync_worker_count(Oid subid);
 
 extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
-											  char *originname, int szorgname);
+											  char *originname, Size szorgname);
 extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
 
 extern bool AllTablesyncsReady(void);
-- 
2.37.2

v2-0002-Add-common-function-ReplicationOriginName.patchapplication/octet-stream; name=v2-0002-Add-common-function-ReplicationOriginName.patchDownload
From 6b3bf2d2f7353d7519248e0338d2d06c23710fde Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 19 Sep 2022 12:21:56 +1000
Subject: [PATCH v2 2/2] Add common function ReplicationOriginName()

Replace common snprintf() expression with a wrapper function similar to
ReplicationOriginNameForTablesync().

Peter Smith, reviewed by Aleksander Alekseev
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
 src/backend/commands/subscriptioncmds.c   |  6 +++---
 src/backend/replication/logical/worker.c  | 13 ++++++++++++-
 src/include/replication/worker_internal.h |  1 +
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index d042abe341..8166b3c1b2 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -659,7 +659,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
 
 	recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
 
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginName(subid, originname, sizeof(originname));
 	replorigin_create(originname);
 
 	/*
@@ -1317,7 +1317,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
 					char		originname[NAMEDATALEN];
 					XLogRecPtr	remote_lsn;
 
-					snprintf(originname, sizeof(originname), "pg_%u", subid);
+					ReplicationOriginName(subid, originname, sizeof(originname));
 					originid = replorigin_by_name(originname, false);
 					remote_lsn = replorigin_get_progress(originid, false);
 
@@ -1535,7 +1535,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
 	RemoveSubscriptionRel(subid, InvalidOid);
 
 	/* Remove the origin tracking if exists. */
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginName(subid, originname, sizeof(originname));
 	replorigin_drop_by_name(originname, true, false);
 
 	/*
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 56f753d987..b2db7861c2 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -364,6 +364,17 @@ static void apply_error_callback(void *arg);
 static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr lsn);
 static inline void reset_apply_error_context_info(void);
 
+/*
+ * Form the origin name for subscription.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginName(Oid suboid, char *originname, Size szname)
+{
+	snprintf(originname, szname, "pg_%u", suboid);
+}
+
 /*
  * Should this worker apply changes for given relation.
  *
@@ -3707,7 +3718,7 @@ ApplyWorkerMain(Datum main_arg)
 
 		/* Setup replication origin tracking. */
 		StartTransactionCommand();
-		snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+		ReplicationOriginName(MySubscription->oid, originname, sizeof(originname));
 		originid = replorigin_by_name(originname, true);
 		if (!OidIsValid(originid))
 			originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index f82bc518c3..8c22c08bf9 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,6 +92,7 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
 
 extern int	logicalrep_sync_worker_count(Oid subid);
 
+extern void ReplicationOriginName(Oid suboid, char *originname, Size szname);
 extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
 											  char *originname, Size szorgname);
 extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
-- 
2.37.2

#5Peter Smith
smithpb2250@gmail.com
In reply to: Aleksander Alekseev (#4)
Re: Add common function ReplicationOriginName.

On Tue, Sep 20, 2022 at 6:36 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:

Hi Amit,

I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.

OK, here is the updated patchset.

* 0001 replaces int's with Size's in the existing code
* 0002 applies Peter's patch on top of 0001

LGTM. Thanks!

-----
Kind Regards,
Peter Smith.
Fujitsu Australia

#6Amit Kapila
amit.kapila16@gmail.com
In reply to: Aleksander Alekseev (#4)
Re: Add common function ReplicationOriginName.

On Tue, Sep 20, 2022 at 2:06 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:

Hi Amit,

I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.

OK, here is the updated patchset.

* 0001 replaces int's with Size's in the existing code

Pushed this one.

* 0002 applies Peter's patch on top of 0001

Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.

--
With Regards,
Amit Kapila.

#7Peter Smith
smithpb2250@gmail.com
In reply to: Amit Kapila (#6)
Re: Add common function ReplicationOriginName.

On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

...

Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.

This suggestion attaches special meaning to the reild param.

Won't it seem a bit strange for the non-tablesync callers (who
probably have a perfectly valid 'relid') to have to pass an InvalidOid
relid just so they can format the correct origin name?

------
Kind Regards,
Peter Smith.
Fujitsu Australia

#8Amit Kapila
amit.kapila16@gmail.com
In reply to: Peter Smith (#7)
Re: Add common function ReplicationOriginName.

On Wed, Sep 21, 2022 at 3:09 PM Peter Smith <smithpb2250@gmail.com> wrote:

On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

...

Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.

This suggestion attaches special meaning to the reild param.

Won't it seem a bit strange for the non-tablesync callers (who
probably have a perfectly valid 'relid') to have to pass an InvalidOid
relid just so they can format the correct origin name?

For non-tablesync workers, relid should always be InvalidOid. See, how
we launch apply workers in ApplyLauncherMain(). Do you see any case
for non-tablesync workers where relid is not InvalidOid?

--
With Regards,
Amit Kapila.

#9Peter Smith
smithpb2250@gmail.com
In reply to: Amit Kapila (#8)
Re: Add common function ReplicationOriginName.

On Wed, Sep 21, 2022 at 8:08 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Wed, Sep 21, 2022 at 3:09 PM Peter Smith <smithpb2250@gmail.com> wrote:

On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

...

Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.

This suggestion attaches special meaning to the reild param.

Won't it seem a bit strange for the non-tablesync callers (who
probably have a perfectly valid 'relid') to have to pass an InvalidOid
relid just so they can format the correct origin name?

For non-tablesync workers, relid should always be InvalidOid. See, how
we launch apply workers in ApplyLauncherMain(). Do you see any case
for non-tablesync workers where relid is not InvalidOid?

Hmm, my mistake. I was thinking more of all the calls coming from the
subscriptioncmds.c, but now that I look at it maybe none of those has
any relid either.

OK, I can unify the 2 functions as you suggested. I will post another
patch in a few days.

------
Kind Regards,
Peter Smith,
Fujitsu Australia.

#10Peter Smith
smithpb2250@gmail.com
In reply to: Peter Smith (#9)
1 attachment(s)
Re: Add common function ReplicationOriginName.

On Wed, Sep 21, 2022 at 8:22 PM Peter Smith <smithpb2250@gmail.com> wrote:

...

On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
...

Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.

...

OK, I can unify the 2 functions as you suggested. I will post another
patch in a few days.

PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.

------
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachments:

v3-0001-Add-common-function-ReplicationOriginNameForLogic.patchapplication/octet-stream; name=v3-0001-Add-common-function-ReplicationOriginNameForLogic.patchDownload
From 65ad6d794bb242e43232dd34a5f843d2f4c3c3dd Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 26 Sep 2022 12:59:43 +1000
Subject: [PATCH v3] Add common function ReplicationOriginNameForLogicalRep.

Make a common replication origin name formatting function to replace multiple
snprintf() expressions. This also includes logic previously done by
ReplicationOriginNameForTablesync().

Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
 src/backend/commands/subscriptioncmds.c     | 11 ++++++-----
 src/backend/replication/logical/tablesync.c | 18 +++---------------
 src/backend/replication/logical/worker.c    | 29 +++++++++++++++++++++++++++--
 src/include/replication/worker_internal.h   |  4 ++--
 4 files changed, 38 insertions(+), 24 deletions(-)

diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index f3bfcca..0793234 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -657,7 +657,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
 
 	recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
 
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
 	replorigin_create(originname);
 
 	/*
@@ -946,7 +946,7 @@ AlterSubscription_refresh(Subscription *sub, bool copy_data,
 					 * origin and by this time the origin might be already
 					 * removed. For these reasons, passing missing_ok = true.
 					 */
-					ReplicationOriginNameForTablesync(sub->oid, relid, originname,
+					ReplicationOriginNameForLogicalRep(sub->oid, relid, originname,
 													  sizeof(originname));
 					replorigin_drop_by_name(originname, true, false);
 				}
@@ -1315,7 +1315,8 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
 					char		originname[NAMEDATALEN];
 					XLogRecPtr	remote_lsn;
 
-					snprintf(originname, sizeof(originname), "pg_%u", subid);
+					ReplicationOriginNameForLogicalRep(subid, InvalidOid,
+													   originname, sizeof(originname));
 					originid = replorigin_by_name(originname, false);
 					remote_lsn = replorigin_get_progress(originid, false);
 
@@ -1521,7 +1522,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
 		 * worker so passing missing_ok = true. This can happen for the states
 		 * before SUBREL_STATE_FINISHEDCOPY.
 		 */
-		ReplicationOriginNameForTablesync(subid, relid, originname,
+		ReplicationOriginNameForLogicalRep(subid, relid, originname,
 										  sizeof(originname));
 		replorigin_drop_by_name(originname, true, false);
 	}
@@ -1533,7 +1534,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
 	RemoveSubscriptionRel(subid, InvalidOid);
 
 	/* Remove the origin tracking if exists. */
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
 	replorigin_drop_by_name(originname, true, false);
 
 	/*
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 9e52fc4..ff0f359 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -353,7 +353,7 @@ process_syncing_tables_for_sync(XLogRecPtr current_lsn)
 		 */
 		StartTransactionCommand();
 
-		ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
+		ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
 										  MyLogicalRepWorker->relid,
 										  originname,
 										  sizeof(originname));
@@ -505,7 +505,7 @@ process_syncing_tables_for_apply(XLogRecPtr current_lsn)
 				 * error while dropping we won't restart it to drop the
 				 * origin. So passing missing_ok = true.
 				 */
-				ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
+				ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
 												  rstate->relid,
 												  originname,
 												  sizeof(originname));
@@ -1194,18 +1194,6 @@ ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
 }
 
 /*
- * Form the origin name for tablesync.
- *
- * Return the name in the supplied buffer.
- */
-void
-ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
-								  char *originname, Size szorgname)
-{
-	snprintf(originname, szorgname, "pg_%u_%u", suboid, relid);
-}
-
-/*
  * Start syncing the table in the sync worker.
  *
  * If nothing needs to be done to sync the table, we exit the worker without
@@ -1274,7 +1262,7 @@ LogicalRepSyncTableStart(XLogRecPtr *origin_startpos)
 		   MyLogicalRepWorker->relstate == SUBREL_STATE_FINISHEDCOPY);
 
 	/* Assign the origin tracking record name. */
-	ReplicationOriginNameForTablesync(MySubscription->oid,
+	ReplicationOriginNameForLogicalRep(MySubscription->oid,
 									  MyLogicalRepWorker->relid,
 									  originname,
 									  sizeof(originname));
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 207a580..c2cd61a 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -365,6 +365,30 @@ static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr ls
 static inline void reset_apply_error_context_info(void);
 
 /*
+ * Form the origin name for the subscription.
+ *
+ * This is a common function for tablesync and other workers. Tablesync workers
+ * must pass a valid relid. Other callers must pass relid = InvalidOid.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+								   char *originname, Size szoriginname)
+{
+	if (OidIsValid(relid))
+	{
+		/* Replication origin name for tablesync workers. */
+		snprintf(originname, szoriginname, "pg_%u_%u", suboid, relid);
+	}
+	else
+	{
+		/* Replication origin name for non-tablesync workers. */
+		snprintf(originname, szoriginname, "pg_%u", suboid);
+	}
+}
+
+/*
  * Should this worker apply changes for given relation.
  *
  * This is mainly needed for initial relation data sync as that runs in
@@ -3679,7 +3703,7 @@ ApplyWorkerMain(Datum main_arg)
 		 * Allocate the origin name in long-lived context for error context
 		 * message.
 		 */
-		ReplicationOriginNameForTablesync(MySubscription->oid,
+		ReplicationOriginNameForLogicalRep(MySubscription->oid,
 										  MyLogicalRepWorker->relid,
 										  originname,
 										  sizeof(originname));
@@ -3707,7 +3731,8 @@ ApplyWorkerMain(Datum main_arg)
 
 		/* Setup replication origin tracking. */
 		StartTransactionCommand();
-		snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+		ReplicationOriginNameForLogicalRep(MySubscription->oid, InvalidOid,
+										   originname, sizeof(originname));
 		originid = replorigin_by_name(originname, true);
 		if (!OidIsValid(originid))
 			originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index f82bc51..40a0b06 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,8 +92,8 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
 
 extern int	logicalrep_sync_worker_count(Oid subid);
 
-extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
-											  char *originname, Size szorgname);
+extern void ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+											  char *originname, Size szoriginname);
 extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
 
 extern bool AllTablesyncsReady(void);
-- 
1.8.3.1

#11Peter Smith
smithpb2250@gmail.com
In reply to: Aleksander Alekseev (#4)
Re: Add common function ReplicationOriginName.

On Tue, Sep 20, 2022 at 6:36 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:

Hi Amit,

I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.

OK, here is the updated patchset.

* 0001 replaces int's with Size's in the existing code
* 0002 applies Peter's patch on top of 0001

Hi Aleksander.

FYI - although it is outside the scope of this thread, I did notice at
least one other example where you might want to substitute Size for
int in the same way as your v2-0001 patch did.

e.g. Just searching code for 'snprintf' where there is some parameter
for the size I quickly found:

File: src/bin/pg_dump/pg_dump_sort.c:

static void
describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)

caller:
describeDumpableObject(loop[i], buf, sizeof(buf));

~~

I expect you can find more like just this if you look harder than I did.

------
Kind Regards,
Peter Smith.
Fujitsu Australia.

#12Aleksander Alekseev
aleksander@timescale.com
In reply to: Peter Smith (#11)
2 attachment(s)
Re: Add common function ReplicationOriginName.

Hi Peter,

PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.

LGTM except for minor issues with the formatting; fixed.

I expect you can find more like just this if you look harder than I did.

Thanks. You are right, there are more places that pass int as the
second argument of *nprintf(). I used a command:

$ grep -r nprintf ./ | perl -lne 'print if($_ !~
/nprintf\([^\,]+,\s*(sizeof|([0-9A-Z_ \-]+\,))/ )' > found.txt

... and then re-checked the results manually. This excludes patterns
like *nprintf(..., sizeof(...)) and *nprintf(..., MACRO) and leaves
only something like *nprintf(..., variable). The cases where we
subtract an integer from a Size, etc were ignored.

I don't have a strong opinion on whether we should be really worried
by this. But in case we do, here is the patch. The order of 0001 and
0002 doesn't matter.

As I understand, ecpg uses size_t rather than Size, so for this
library I used size_t. Not 100% sure if the changes I made to
src/backend/utils/adt/jsonb.c add much value. I leave this to the
committer to decide.

--
Best regards,
Aleksander Alekseev

Attachments:

v4-0002-Add-common-function-ReplicationOriginNameForLogic.patchapplication/octet-stream; name=v4-0002-Add-common-function-ReplicationOriginNameForLogic.patchDownload
From 55eb5d8980c59cd5743f941517bc3af2744d761e Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 26 Sep 2022 12:59:43 +1000
Subject: [PATCH v4 2/2] Add common function
 ReplicationOriginNameForLogicalRep.

Make a common replication origin name formatting function to replace multiple
snprintf() expressions. This also includes logic previously done by
ReplicationOriginNameForTablesync().

Peter Smith, reviewed by Aleksander Alekseev
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
 src/backend/commands/subscriptioncmds.c     | 15 +++++----
 src/backend/replication/logical/tablesync.c | 36 +++++++--------------
 src/backend/replication/logical/worker.c    | 35 +++++++++++++++++---
 src/include/replication/worker_internal.h   |  4 +--
 4 files changed, 52 insertions(+), 38 deletions(-)

diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index f3bfcca434..97594cd9b1 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -657,7 +657,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
 
 	recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
 
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
 	replorigin_create(originname);
 
 	/*
@@ -946,8 +946,8 @@ AlterSubscription_refresh(Subscription *sub, bool copy_data,
 					 * origin and by this time the origin might be already
 					 * removed. For these reasons, passing missing_ok = true.
 					 */
-					ReplicationOriginNameForTablesync(sub->oid, relid, originname,
-													  sizeof(originname));
+					ReplicationOriginNameForLogicalRep(sub->oid, relid, originname,
+													   sizeof(originname));
 					replorigin_drop_by_name(originname, true, false);
 				}
 
@@ -1315,7 +1315,8 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
 					char		originname[NAMEDATALEN];
 					XLogRecPtr	remote_lsn;
 
-					snprintf(originname, sizeof(originname), "pg_%u", subid);
+					ReplicationOriginNameForLogicalRep(subid, InvalidOid,
+													   originname, sizeof(originname));
 					originid = replorigin_by_name(originname, false);
 					remote_lsn = replorigin_get_progress(originid, false);
 
@@ -1521,8 +1522,8 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
 		 * worker so passing missing_ok = true. This can happen for the states
 		 * before SUBREL_STATE_FINISHEDCOPY.
 		 */
-		ReplicationOriginNameForTablesync(subid, relid, originname,
-										  sizeof(originname));
+		ReplicationOriginNameForLogicalRep(subid, relid, originname,
+										   sizeof(originname));
 		replorigin_drop_by_name(originname, true, false);
 	}
 
@@ -1533,7 +1534,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
 	RemoveSubscriptionRel(subid, InvalidOid);
 
 	/* Remove the origin tracking if exists. */
-	snprintf(originname, sizeof(originname), "pg_%u", subid);
+	ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
 	replorigin_drop_by_name(originname, true, false);
 
 	/*
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 9e52fc401c..b6e0af0a76 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -353,10 +353,10 @@ process_syncing_tables_for_sync(XLogRecPtr current_lsn)
 		 */
 		StartTransactionCommand();
 
-		ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
-										  MyLogicalRepWorker->relid,
-										  originname,
-										  sizeof(originname));
+		ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
+										   MyLogicalRepWorker->relid,
+										   originname,
+										   sizeof(originname));
 
 		/*
 		 * Resetting the origin session removes the ownership of the slot.
@@ -505,10 +505,10 @@ process_syncing_tables_for_apply(XLogRecPtr current_lsn)
 				 * error while dropping we won't restart it to drop the
 				 * origin. So passing missing_ok = true.
 				 */
-				ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
-												  rstate->relid,
-												  originname,
-												  sizeof(originname));
+				ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
+												   rstate->relid,
+												   originname,
+												   sizeof(originname));
 				replorigin_drop_by_name(originname, true, false);
 
 				/*
@@ -1193,18 +1193,6 @@ ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
 			 relid, GetSystemIdentifier());
 }
 
-/*
- * Form the origin name for tablesync.
- *
- * Return the name in the supplied buffer.
- */
-void
-ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
-								  char *originname, Size szorgname)
-{
-	snprintf(originname, szorgname, "pg_%u_%u", suboid, relid);
-}
-
 /*
  * Start syncing the table in the sync worker.
  *
@@ -1274,10 +1262,10 @@ LogicalRepSyncTableStart(XLogRecPtr *origin_startpos)
 		   MyLogicalRepWorker->relstate == SUBREL_STATE_FINISHEDCOPY);
 
 	/* Assign the origin tracking record name. */
-	ReplicationOriginNameForTablesync(MySubscription->oid,
-									  MyLogicalRepWorker->relid,
-									  originname,
-									  sizeof(originname));
+	ReplicationOriginNameForLogicalRep(MySubscription->oid,
+									   MyLogicalRepWorker->relid,
+									   originname,
+									   sizeof(originname));
 
 	if (MyLogicalRepWorker->relstate == SUBREL_STATE_DATASYNC)
 	{
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 188c51660e..bfa32391bf 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -364,6 +364,30 @@ static void apply_error_callback(void *arg);
 static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr lsn);
 static inline void reset_apply_error_context_info(void);
 
+/*
+ * Form the origin name for the subscription.
+ *
+ * This is a common function for tablesync and other workers. Tablesync workers
+ * must pass a valid relid. Other callers must pass relid = InvalidOid.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+								   char *originname, Size szoriginname)
+{
+	if (OidIsValid(relid))
+	{
+		/* Replication origin name for tablesync workers. */
+		snprintf(originname, szoriginname, "pg_%u_%u", suboid, relid);
+	}
+	else
+	{
+		/* Replication origin name for non-tablesync workers. */
+		snprintf(originname, szoriginname, "pg_%u", suboid);
+	}
+}
+
 /*
  * Should this worker apply changes for given relation.
  *
@@ -3679,10 +3703,10 @@ ApplyWorkerMain(Datum main_arg)
 		 * Allocate the origin name in long-lived context for error context
 		 * message.
 		 */
-		ReplicationOriginNameForTablesync(MySubscription->oid,
-										  MyLogicalRepWorker->relid,
-										  originname,
-										  sizeof(originname));
+		ReplicationOriginNameForLogicalRep(MySubscription->oid,
+										   MyLogicalRepWorker->relid,
+										   originname,
+										   sizeof(originname));
 		apply_error_callback_arg.origin_name = MemoryContextStrdup(ApplyContext,
 																   originname);
 	}
@@ -3707,7 +3731,8 @@ ApplyWorkerMain(Datum main_arg)
 
 		/* Setup replication origin tracking. */
 		StartTransactionCommand();
-		snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+		ReplicationOriginNameForLogicalRep(MySubscription->oid, InvalidOid,
+										   originname, sizeof(originname));
 		originid = replorigin_by_name(originname, true);
 		if (!OidIsValid(originid))
 			originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index f82bc518c3..2b7114ff6d 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,8 +92,8 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
 
 extern int	logicalrep_sync_worker_count(Oid subid);
 
-extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
-											  char *originname, Size szorgname);
+extern void ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+											   char *originname, Size szoriginname);
 extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
 
 extern bool AllTablesyncsReady(void);
-- 
2.37.2

v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchapplication/octet-stream; name=v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchDownload
From 0194f66dba4f599ba66971d97e8c213d95979441 Mon Sep 17 00:00:00 2001
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 27 Sep 2022 13:33:21 +0300
Subject: [PATCH v4 1/2] Pass Size/size_t as a 2nd argument of snprintf()

This is a follow-up to a932824d. The previous patch made sure the second
argument of snprintf() is passed as Size rather than int in tablesync.c.
This patch does the same change to the rest of the project.

Aleksander Alekseev, reviewed by Peter Smith
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
 src/backend/replication/logical/worker.c | 4 ++--
 src/backend/utils/adt/jsonb.c            | 6 +++---
 src/bin/pg_dump/parallel.c               | 4 ++--
 src/bin/pg_dump/pg_dump_sort.c           | 4 ++--
 src/bin/psql/command.c                   | 2 +-
 src/common/ip.c                          | 4 ++--
 src/interfaces/ecpg/ecpglib/execute.c    | 2 +-
 src/interfaces/ecpg/ecpglib/misc.c       | 2 +-
 src/interfaces/ecpg/ecpglib/prepare.c    | 2 +-
 src/interfaces/libpq/fe-connect.c        | 2 +-
 src/interfaces/libpq/fe-gssapi-common.c  | 2 +-
 src/interfaces/libpq/libpq-int.h         | 2 +-
 src/test/regress/pg_regress.c            | 2 +-
 13 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 207a5805ba..188c51660e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -349,7 +349,7 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
 									   CmdType operation);
 
 /* Compute GID for two_phase transactions */
-static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
+static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid);
 
 /* Common streaming function to apply all the spooled messages */
 static void apply_spooled_messages(TransactionId xid, XLogRecPtr lsn);
@@ -3480,7 +3480,7 @@ cleanup_subxact_info()
  * Return the GID in the supplied buffer.
  */
 static void
-TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid)
+TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid)
 {
 	Assert(subid != InvalidRepOriginId);
 
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9e14922ec2..1ccdca349e 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -1592,7 +1592,7 @@ jsonb_agg_transfn(PG_FUNCTION_ARGS)
 					/* copy string values in the aggregate context */
 					char	   *buf = palloc(v.val.string.len + 1);
 
-					snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+					snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
 					v.val.string.val = buf;
 				}
 				else if (v.type == jbvNumeric)
@@ -1763,7 +1763,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
 					/* copy string values in the aggregate context */
 					char	   *buf = palloc(v.val.string.len + 1);
 
-					snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+					snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
 					v.val.string.val = buf;
 				}
 				else
@@ -1822,7 +1822,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
 					/* copy string values in the aggregate context */
 					char	   *buf = palloc(v.val.string.len + 1);
 
-					snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+					snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
 					v.val.string.val = buf;
 				}
 				else if (v.type == jbvNumeric)
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index c8a70d9bc1..f5e4c63e8f 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -1106,7 +1106,7 @@ ParallelBackupEnd(ArchiveHandle *AH, ParallelState *pstate)
  */
 static void
 buildWorkerCommand(ArchiveHandle *AH, TocEntry *te, T_Action act,
-				   char *buf, int buflen)
+				   char *buf, Size buflen)
 {
 	if (act == ACT_DUMP)
 		snprintf(buf, buflen, "DUMP %d", te->dumpId);
@@ -1154,7 +1154,7 @@ parseWorkerCommand(ArchiveHandle *AH, TocEntry **te, T_Action *act,
  */
 static void
 buildWorkerResponse(ArchiveHandle *AH, TocEntry *te, T_Action act, int status,
-					char *buf, int buflen)
+					char *buf, Size buflen)
 {
 	snprintf(buf, buflen, "OK %d %d %d",
 			 te->dumpId,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5de3241eb4..51337ff798 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -173,7 +173,7 @@ static int	findLoop(DumpableObject *obj,
 static void repairDependencyLoop(DumpableObject **loop,
 								 int nLoop);
 static void describeDumpableObject(DumpableObject *obj,
-								   char *buf, int bufsize);
+								   char *buf, Size bufsize);
 
 
 /*
@@ -1268,7 +1268,7 @@ repairDependencyLoop(DumpableObject **loop,
  * This should probably go somewhere else...
  */
 static void
-describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
+describeDumpableObject(DumpableObject *obj, char *buf, Size bufsize)
 {
 	switch (obj->objType)
 	{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a141146e70..fc882138fb 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -5008,7 +5008,7 @@ do_watch(PQExpBuffer query_buf, double sleep)
 	char	   *title;
 	const char *pagerprog = NULL;
 	FILE	   *pagerpipe = NULL;
-	int			title_len;
+	Size		title_len;
 	int			res = 0;
 #ifndef WIN32
 	sigset_t	sigalrm_sigchld_sigint;
diff --git a/src/common/ip.c b/src/common/ip.c
index 6343f49a70..5f09be9195 100644
--- a/src/common/ip.c
+++ b/src/common/ip.c
@@ -42,7 +42,7 @@ static int	getaddrinfo_unix(const char *path,
 
 static int	getnameinfo_unix(const struct sockaddr_un *sa, int salen,
 							 char *node, int nodelen,
-							 char *service, int servicelen,
+							 char *service, Size servicelen,
 							 int flags);
 
 
@@ -227,7 +227,7 @@ getaddrinfo_unix(const char *path, const struct addrinfo *hintsp,
 static int
 getnameinfo_unix(const struct sockaddr_un *sa, int salen,
 				 char *node, int nodelen,
-				 char *service, int servicelen,
+				 char *service, Size servicelen,
 				 int flags)
 {
 	int			ret;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index bd94bd4e6c..f1aa833911 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1533,7 +1533,7 @@ ecpg_build_params(struct statement *stmt)
 			if (stmt->command[position] == '?')
 			{
 				/* yes, replace with new style */
-				int			buffersize = sizeof(int) * CHAR_BIT * 10 / 3;	/* a rough guess of the
+				size_t		buffersize = sizeof(int) * CHAR_BIT * 10 / 3;	/* a rough guess of the
 																			 * size we need */
 
 				if (!(tobeinserted = (char *) ecpg_alloc(buffersize, stmt->lineno)))
diff --git a/src/interfaces/ecpg/ecpglib/misc.c b/src/interfaces/ecpg/ecpglib/misc.c
index 1eef1ec044..d861d06993 100644
--- a/src/interfaces/ecpg/ecpglib/misc.c
+++ b/src/interfaces/ecpg/ecpglib/misc.c
@@ -267,7 +267,7 @@ ecpg_log(const char *format,...)
 	va_list		ap;
 	struct sqlca_t *sqlca = ECPGget_sqlca();
 	const char *intl_format;
-	int			bufsize;
+	size_t		bufsize;
 	char	   *fmt;
 
 	if (!simple_debug)
diff --git a/src/interfaces/ecpg/ecpglib/prepare.c b/src/interfaces/ecpg/ecpglib/prepare.c
index ea1146f520..5ff8feac7d 100644
--- a/src/interfaces/ecpg/ecpglib/prepare.c
+++ b/src/interfaces/ecpg/ecpglib/prepare.c
@@ -120,7 +120,7 @@ replace_variables(char **text, int lineno)
 		else
 		{
 			/* a rough guess of the size we need: */
-			int			buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
+			size_t		buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
 			int			len;
 			char	   *buffer,
 					   *newcopy;
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 746e9b4f1e..cdccd59c54 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -7313,7 +7313,7 @@ sslVerifyProtocolRange(const char *min, const char *max)
  * there (which it isn't).
  */
 bool
-pqGetHomeDirectory(char *buf, int bufsize)
+pqGetHomeDirectory(char *buf, size_t bufsize)
 {
 #ifndef WIN32
 	const char *home;
diff --git a/src/interfaces/libpq/fe-gssapi-common.c b/src/interfaces/libpq/fe-gssapi-common.c
index fa08526ee2..9f03d02c32 100644
--- a/src/interfaces/libpq/fe-gssapi-common.c
+++ b/src/interfaces/libpq/fe-gssapi-common.c
@@ -83,7 +83,7 @@ pg_GSS_load_servicename(PGconn *conn)
 {
 	OM_uint32	maj_stat,
 				min_stat;
-	int			maxlen;
+	Size		maxlen;
 	gss_buffer_desc temp_gbuf;
 	char	   *host;
 
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c75ed63a2c..2d570017a3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -648,7 +648,7 @@ extern char *const pgresStatus[];
 extern void pqDropConnection(PGconn *conn, bool flushInput);
 extern int	pqPacketSend(PGconn *conn, char pack_type,
 						 const void *buf, size_t buf_len);
-extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqGetHomeDirectory(char *buf, size_t bufsize);
 
 #ifdef ENABLE_THREAD_SAFETY
 extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index dda076847a..229165a684 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -1206,7 +1206,7 @@ static char *
 get_alternative_expectfile(const char *expectfile, int i)
 {
 	char	   *last_dot;
-	int			ssize = strlen(expectfile) + 2 + 1;
+	Size		ssize = strlen(expectfile) + 2 + 1;
 	char	   *tmp;
 	char	   *s;
 
-- 
2.37.2

#13Amit Kapila
amit.kapila16@gmail.com
In reply to: Aleksander Alekseev (#12)
Re: Add common function ReplicationOriginName.

On Tue, Sep 27, 2022 at 5:04 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:

Hi Peter,

PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.

LGTM except for minor issues with the formatting; fixed.

LGTM as well. I'll push this tomorrow unless there are any more comments.

--
With Regards,
Amit Kapila.

#14Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#13)
Re: Add common function ReplicationOriginName.

On Mon, Oct 10, 2022 at 7:09 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

On Tue, Sep 27, 2022 at 5:04 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:

Hi Peter,

PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.

LGTM except for minor issues with the formatting; fixed.

LGTM as well. I'll push this tomorrow unless there are any more comments.

Pushed.

--
With Regards,
Amit Kapila.

#15Aleksander Alekseev
aleksander@timescale.com
In reply to: Amit Kapila (#14)
1 attachment(s)
Re: Add common function ReplicationOriginName.

Hi Amit,

Pushed.

Thanks!

I don't have a strong opinion on whether we should be really worried
by this. But in case we do, here is the patch.

This leaves us one patch to deal with.

--
Best regards,
Aleksander Alekseev

Attachments:

v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchapplication/octet-stream; name=v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchDownload
From 0194f66dba4f599ba66971d97e8c213d95979441 Mon Sep 17 00:00:00 2001
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 27 Sep 2022 13:33:21 +0300
Subject: [PATCH v4 1/2] Pass Size/size_t as a 2nd argument of snprintf()

This is a follow-up to a932824d. The previous patch made sure the second
argument of snprintf() is passed as Size rather than int in tablesync.c.
This patch does the same change to the rest of the project.

Aleksander Alekseev, reviewed by Peter Smith
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
 src/backend/replication/logical/worker.c | 4 ++--
 src/backend/utils/adt/jsonb.c            | 6 +++---
 src/bin/pg_dump/parallel.c               | 4 ++--
 src/bin/pg_dump/pg_dump_sort.c           | 4 ++--
 src/bin/psql/command.c                   | 2 +-
 src/common/ip.c                          | 4 ++--
 src/interfaces/ecpg/ecpglib/execute.c    | 2 +-
 src/interfaces/ecpg/ecpglib/misc.c       | 2 +-
 src/interfaces/ecpg/ecpglib/prepare.c    | 2 +-
 src/interfaces/libpq/fe-connect.c        | 2 +-
 src/interfaces/libpq/fe-gssapi-common.c  | 2 +-
 src/interfaces/libpq/libpq-int.h         | 2 +-
 src/test/regress/pg_regress.c            | 2 +-
 13 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 207a5805ba..188c51660e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -349,7 +349,7 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
 									   CmdType operation);
 
 /* Compute GID for two_phase transactions */
-static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
+static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid);
 
 /* Common streaming function to apply all the spooled messages */
 static void apply_spooled_messages(TransactionId xid, XLogRecPtr lsn);
@@ -3480,7 +3480,7 @@ cleanup_subxact_info()
  * Return the GID in the supplied buffer.
  */
 static void
-TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid)
+TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid)
 {
 	Assert(subid != InvalidRepOriginId);
 
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9e14922ec2..1ccdca349e 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -1592,7 +1592,7 @@ jsonb_agg_transfn(PG_FUNCTION_ARGS)
 					/* copy string values in the aggregate context */
 					char	   *buf = palloc(v.val.string.len + 1);
 
-					snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+					snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
 					v.val.string.val = buf;
 				}
 				else if (v.type == jbvNumeric)
@@ -1763,7 +1763,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
 					/* copy string values in the aggregate context */
 					char	   *buf = palloc(v.val.string.len + 1);
 
-					snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+					snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
 					v.val.string.val = buf;
 				}
 				else
@@ -1822,7 +1822,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
 					/* copy string values in the aggregate context */
 					char	   *buf = palloc(v.val.string.len + 1);
 
-					snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+					snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
 					v.val.string.val = buf;
 				}
 				else if (v.type == jbvNumeric)
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index c8a70d9bc1..f5e4c63e8f 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -1106,7 +1106,7 @@ ParallelBackupEnd(ArchiveHandle *AH, ParallelState *pstate)
  */
 static void
 buildWorkerCommand(ArchiveHandle *AH, TocEntry *te, T_Action act,
-				   char *buf, int buflen)
+				   char *buf, Size buflen)
 {
 	if (act == ACT_DUMP)
 		snprintf(buf, buflen, "DUMP %d", te->dumpId);
@@ -1154,7 +1154,7 @@ parseWorkerCommand(ArchiveHandle *AH, TocEntry **te, T_Action *act,
  */
 static void
 buildWorkerResponse(ArchiveHandle *AH, TocEntry *te, T_Action act, int status,
-					char *buf, int buflen)
+					char *buf, Size buflen)
 {
 	snprintf(buf, buflen, "OK %d %d %d",
 			 te->dumpId,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5de3241eb4..51337ff798 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -173,7 +173,7 @@ static int	findLoop(DumpableObject *obj,
 static void repairDependencyLoop(DumpableObject **loop,
 								 int nLoop);
 static void describeDumpableObject(DumpableObject *obj,
-								   char *buf, int bufsize);
+								   char *buf, Size bufsize);
 
 
 /*
@@ -1268,7 +1268,7 @@ repairDependencyLoop(DumpableObject **loop,
  * This should probably go somewhere else...
  */
 static void
-describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
+describeDumpableObject(DumpableObject *obj, char *buf, Size bufsize)
 {
 	switch (obj->objType)
 	{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a141146e70..fc882138fb 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -5008,7 +5008,7 @@ do_watch(PQExpBuffer query_buf, double sleep)
 	char	   *title;
 	const char *pagerprog = NULL;
 	FILE	   *pagerpipe = NULL;
-	int			title_len;
+	Size		title_len;
 	int			res = 0;
 #ifndef WIN32
 	sigset_t	sigalrm_sigchld_sigint;
diff --git a/src/common/ip.c b/src/common/ip.c
index 6343f49a70..5f09be9195 100644
--- a/src/common/ip.c
+++ b/src/common/ip.c
@@ -42,7 +42,7 @@ static int	getaddrinfo_unix(const char *path,
 
 static int	getnameinfo_unix(const struct sockaddr_un *sa, int salen,
 							 char *node, int nodelen,
-							 char *service, int servicelen,
+							 char *service, Size servicelen,
 							 int flags);
 
 
@@ -227,7 +227,7 @@ getaddrinfo_unix(const char *path, const struct addrinfo *hintsp,
 static int
 getnameinfo_unix(const struct sockaddr_un *sa, int salen,
 				 char *node, int nodelen,
-				 char *service, int servicelen,
+				 char *service, Size servicelen,
 				 int flags)
 {
 	int			ret;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index bd94bd4e6c..f1aa833911 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1533,7 +1533,7 @@ ecpg_build_params(struct statement *stmt)
 			if (stmt->command[position] == '?')
 			{
 				/* yes, replace with new style */
-				int			buffersize = sizeof(int) * CHAR_BIT * 10 / 3;	/* a rough guess of the
+				size_t		buffersize = sizeof(int) * CHAR_BIT * 10 / 3;	/* a rough guess of the
 																			 * size we need */
 
 				if (!(tobeinserted = (char *) ecpg_alloc(buffersize, stmt->lineno)))
diff --git a/src/interfaces/ecpg/ecpglib/misc.c b/src/interfaces/ecpg/ecpglib/misc.c
index 1eef1ec044..d861d06993 100644
--- a/src/interfaces/ecpg/ecpglib/misc.c
+++ b/src/interfaces/ecpg/ecpglib/misc.c
@@ -267,7 +267,7 @@ ecpg_log(const char *format,...)
 	va_list		ap;
 	struct sqlca_t *sqlca = ECPGget_sqlca();
 	const char *intl_format;
-	int			bufsize;
+	size_t		bufsize;
 	char	   *fmt;
 
 	if (!simple_debug)
diff --git a/src/interfaces/ecpg/ecpglib/prepare.c b/src/interfaces/ecpg/ecpglib/prepare.c
index ea1146f520..5ff8feac7d 100644
--- a/src/interfaces/ecpg/ecpglib/prepare.c
+++ b/src/interfaces/ecpg/ecpglib/prepare.c
@@ -120,7 +120,7 @@ replace_variables(char **text, int lineno)
 		else
 		{
 			/* a rough guess of the size we need: */
-			int			buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
+			size_t		buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
 			int			len;
 			char	   *buffer,
 					   *newcopy;
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 746e9b4f1e..cdccd59c54 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -7313,7 +7313,7 @@ sslVerifyProtocolRange(const char *min, const char *max)
  * there (which it isn't).
  */
 bool
-pqGetHomeDirectory(char *buf, int bufsize)
+pqGetHomeDirectory(char *buf, size_t bufsize)
 {
 #ifndef WIN32
 	const char *home;
diff --git a/src/interfaces/libpq/fe-gssapi-common.c b/src/interfaces/libpq/fe-gssapi-common.c
index fa08526ee2..9f03d02c32 100644
--- a/src/interfaces/libpq/fe-gssapi-common.c
+++ b/src/interfaces/libpq/fe-gssapi-common.c
@@ -83,7 +83,7 @@ pg_GSS_load_servicename(PGconn *conn)
 {
 	OM_uint32	maj_stat,
 				min_stat;
-	int			maxlen;
+	Size		maxlen;
 	gss_buffer_desc temp_gbuf;
 	char	   *host;
 
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c75ed63a2c..2d570017a3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -648,7 +648,7 @@ extern char *const pgresStatus[];
 extern void pqDropConnection(PGconn *conn, bool flushInput);
 extern int	pqPacketSend(PGconn *conn, char pack_type,
 						 const void *buf, size_t buf_len);
-extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqGetHomeDirectory(char *buf, size_t bufsize);
 
 #ifdef ENABLE_THREAD_SAFETY
 extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index dda076847a..229165a684 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -1206,7 +1206,7 @@ static char *
 get_alternative_expectfile(const char *expectfile, int i)
 {
 	char	   *last_dot;
-	int			ssize = strlen(expectfile) + 2 + 1;
+	Size		ssize = strlen(expectfile) + 2 + 1;
 	char	   *tmp;
 	char	   *s;
 
-- 
2.37.2

#16Tom Lane
tgl@sss.pgh.pa.us
In reply to: Aleksander Alekseev (#15)
Re: Add common function ReplicationOriginName.

Aleksander Alekseev <aleksander@timescale.com> writes:

This leaves us one patch to deal with.
[ v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patch ]

I looked at this and am inclined to reject it. None of these
places realistically need to deal with strings longer than
MAXPATHLEN or so, let alone multiple gigabytes. So it's just
code churn, creating backpatch hazards (admittedly not big ones)
for no real gain.

regards, tom lane

#17Aleksander Alekseev
aleksander@timescale.com
In reply to: Tom Lane (#16)
Re: Add common function ReplicationOriginName.

Hi Tom,

I looked at this and am inclined to reject it. [...]

OK, thanks. Then we are done with this thread. I closed the
corresponding CF entry.

--
Best regards,
Aleksander Alekseev