Add common function ReplicationOriginName.
Hi.
There are already multiple places that are building the subscription
origin name, and there are more coming with some new patches [1]/messages/by-id/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com doing
the same:
e.g.
snprintf(originname, sizeof(originname), "pg_%u", subid);
~~
IMO it is better to encapsulate this name formatting in common code
instead of the format string being scattered around the place.
PSA a patch to add a common function ReplicationOriginName. This is
the equivalent of a similar function for tablesync
(ReplicationOriginNameForTablesync) which already existed.
------
[1]: /messages/by-id/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
Kind Regards,
Peter Smith.
Fujitsu Australia
Attachments:
v1-0001-Add-common-function-ReplicationOriginName.patchapplication/octet-stream; name=v1-0001-Add-common-function-ReplicationOriginName.patchDownload
From a95f2907b3805b51c8480f56c5b21ca286970c55 Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 19 Sep 2022 12:21:56 +1000
Subject: [PATCH v1] Add common function ReplicationOriginName.
---
src/backend/commands/subscriptioncmds.c | 6 +++---
src/backend/replication/logical/worker.c | 13 ++++++++++++-
src/include/replication/worker_internal.h | 1 +
3 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index d042abe..8166b3c 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -659,7 +659,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginName(subid, originname, sizeof(originname));
replorigin_create(originname);
/*
@@ -1317,7 +1317,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
char originname[NAMEDATALEN];
XLogRecPtr remote_lsn;
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginName(subid, originname, sizeof(originname));
originid = replorigin_by_name(originname, false);
remote_lsn = replorigin_get_progress(originid, false);
@@ -1535,7 +1535,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
RemoveSubscriptionRel(subid, InvalidOid);
/* Remove the origin tracking if exists. */
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginName(subid, originname, sizeof(originname));
replorigin_drop_by_name(originname, true, false);
/*
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index eaca406..7e77de5 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -364,6 +364,17 @@ static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr ls
static inline void reset_apply_error_context_info(void);
/*
+ * Form the origin name for subscription.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginName(Oid suboid, char *originname, int szname)
+{
+ snprintf(originname, szname, "pg_%u", suboid);
+}
+
+/*
* Should this worker apply changes for given relation.
*
* This is mainly needed for initial relation data sync as that runs in
@@ -3706,7 +3717,7 @@ ApplyWorkerMain(Datum main_arg)
/* Setup replication origin tracking. */
StartTransactionCommand();
- snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+ ReplicationOriginName(MySubscription->oid, originname, sizeof(originname));
originid = replorigin_by_name(originname, true);
if (!OidIsValid(originid))
originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 901845a..3714782 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,6 +92,7 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
extern int logicalrep_sync_worker_count(Oid subid);
+extern void ReplicationOriginName(Oid suboid, char *originname, int szname);
extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
char *originname, int szorgname);
extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
--
1.8.3.1
Hi Peter,
PSA a patch to add a common function ReplicationOriginName
The patch looks good to me.
One nitpick I have is that the 2nd argument of snprintf is size_t
while we are passing int's. Your patch is consistent with the current
implementation of ReplicationOriginNameForTablesync() and similar
functions in tablesync.c. However I would like to mention this in case
the committer will be interested in replacing ints with Size's while
on it.
--
Best regards,
Aleksander Alekseev
On Mon, Sep 19, 2022 at 2:27 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
Hi Peter,
PSA a patch to add a common function ReplicationOriginName
The patch looks good to me.
One nitpick I have is that the 2nd argument of snprintf is size_t
while we are passing int's. Your patch is consistent with the current
implementation of ReplicationOriginNameForTablesync() and similar
functions in tablesync.c.
I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.
--
With Regards,
Amit Kapila.
Hi Amit,
I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.
OK, here is the updated patchset.
* 0001 replaces int's with Size's in the existing code
* 0002 applies Peter's patch on top of 0001
--
Best regards,
Aleksander Alekseev
Attachments:
v2-0001-Pass-Size-as-a-2nd-argument-for-snprintf-in-table.patchapplication/octet-stream; name=v2-0001-Pass-Size-as-a-2nd-argument-for-snprintf-in-table.patchDownload
From a9e96b297d0e00ab3d117de81d0796c46e46cb7a Mon Sep 17 00:00:00 2001
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 20 Sep 2022 11:17:45 +0300
Subject: [PATCH v2 1/2] Pass Size as a 2nd argument for snprintf() in
tablesync.c
Previously the following snprintf() wrappers:
* ReplicationSlotNameForTablesync()
* ReplicationOriginNameForTablesync()
... used int as a second argument of snprintf() while the actual type of it
is size_t. Although it doesn't fail at present better replace it with Size
for consistency with the rest of the system.
Aleksander Alekseev, reviewed by TODO FIXME
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
src/backend/replication/logical/tablesync.c | 4 ++--
src/include/replication/slot.h | 2 +-
src/include/replication/worker_internal.h | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 831d42016c..8eff69c7de 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -1187,7 +1187,7 @@ copy_table(Relation rel)
*/
void
ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
- char *syncslotname, int szslot)
+ char *syncslotname, Size szslot)
{
snprintf(syncslotname, szslot, "pg_%u_sync_%u_" UINT64_FORMAT, suboid,
relid, GetSystemIdentifier());
@@ -1200,7 +1200,7 @@ ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
*/
void
ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
- char *originname, int szorgname)
+ char *originname, Size szorgname)
{
snprintf(originname, szorgname, "pg_%u_%u", suboid, relid);
}
diff --git a/src/include/replication/slot.h b/src/include/replication/slot.h
index 81e31f002a..8d5e764aef 100644
--- a/src/include/replication/slot.h
+++ b/src/include/replication/slot.h
@@ -218,7 +218,7 @@ extern void ReplicationSlotsDropDBSlots(Oid dboid);
extern bool InvalidateObsoleteReplicationSlots(XLogSegNo oldestSegno);
extern ReplicationSlot *SearchNamedReplicationSlot(const char *name, bool need_lock);
extern int ReplicationSlotIndex(ReplicationSlot *slot);
-extern void ReplicationSlotNameForTablesync(Oid suboid, Oid relid, char *syncslotname, int szslot);
+extern void ReplicationSlotNameForTablesync(Oid suboid, Oid relid, char *syncslotname, Size szslot);
extern void ReplicationSlotDropAtPubNode(WalReceiverConn *wrconn, char *slotname, bool missing_ok);
extern void StartupReplicationSlots(void);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index 901845abc2..f82bc518c3 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -93,7 +93,7 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
extern int logicalrep_sync_worker_count(Oid subid);
extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
- char *originname, int szorgname);
+ char *originname, Size szorgname);
extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
extern bool AllTablesyncsReady(void);
--
2.37.2
v2-0002-Add-common-function-ReplicationOriginName.patchapplication/octet-stream; name=v2-0002-Add-common-function-ReplicationOriginName.patchDownload
From 6b3bf2d2f7353d7519248e0338d2d06c23710fde Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 19 Sep 2022 12:21:56 +1000
Subject: [PATCH v2 2/2] Add common function ReplicationOriginName()
Replace common snprintf() expression with a wrapper function similar to
ReplicationOriginNameForTablesync().
Peter Smith, reviewed by Aleksander Alekseev
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
src/backend/commands/subscriptioncmds.c | 6 +++---
src/backend/replication/logical/worker.c | 13 ++++++++++++-
src/include/replication/worker_internal.h | 1 +
3 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index d042abe341..8166b3c1b2 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -659,7 +659,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginName(subid, originname, sizeof(originname));
replorigin_create(originname);
/*
@@ -1317,7 +1317,7 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
char originname[NAMEDATALEN];
XLogRecPtr remote_lsn;
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginName(subid, originname, sizeof(originname));
originid = replorigin_by_name(originname, false);
remote_lsn = replorigin_get_progress(originid, false);
@@ -1535,7 +1535,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
RemoveSubscriptionRel(subid, InvalidOid);
/* Remove the origin tracking if exists. */
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginName(subid, originname, sizeof(originname));
replorigin_drop_by_name(originname, true, false);
/*
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 56f753d987..b2db7861c2 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -364,6 +364,17 @@ static void apply_error_callback(void *arg);
static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr lsn);
static inline void reset_apply_error_context_info(void);
+/*
+ * Form the origin name for subscription.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginName(Oid suboid, char *originname, Size szname)
+{
+ snprintf(originname, szname, "pg_%u", suboid);
+}
+
/*
* Should this worker apply changes for given relation.
*
@@ -3707,7 +3718,7 @@ ApplyWorkerMain(Datum main_arg)
/* Setup replication origin tracking. */
StartTransactionCommand();
- snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+ ReplicationOriginName(MySubscription->oid, originname, sizeof(originname));
originid = replorigin_by_name(originname, true);
if (!OidIsValid(originid))
originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index f82bc518c3..8c22c08bf9 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,6 +92,7 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
extern int logicalrep_sync_worker_count(Oid subid);
+extern void ReplicationOriginName(Oid suboid, char *originname, Size szname);
extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
char *originname, Size szorgname);
extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
--
2.37.2
On Tue, Sep 20, 2022 at 6:36 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
Hi Amit,
I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.OK, here is the updated patchset.
* 0001 replaces int's with Size's in the existing code
* 0002 applies Peter's patch on top of 0001
LGTM. Thanks!
-----
Kind Regards,
Peter Smith.
Fujitsu Australia
On Tue, Sep 20, 2022 at 2:06 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
Hi Amit,
I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.OK, here is the updated patchset.
* 0001 replaces int's with Size's in the existing code
Pushed this one.
* 0002 applies Peter's patch on top of 0001
Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.
--
With Regards,
Amit Kapila.
On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
...
Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.
This suggestion attaches special meaning to the reild param.
Won't it seem a bit strange for the non-tablesync callers (who
probably have a perfectly valid 'relid') to have to pass an InvalidOid
relid just so they can format the correct origin name?
------
Kind Regards,
Peter Smith.
Fujitsu Australia
On Wed, Sep 21, 2022 at 3:09 PM Peter Smith <smithpb2250@gmail.com> wrote:
On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
...
Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.This suggestion attaches special meaning to the reild param.
Won't it seem a bit strange for the non-tablesync callers (who
probably have a perfectly valid 'relid') to have to pass an InvalidOid
relid just so they can format the correct origin name?
For non-tablesync workers, relid should always be InvalidOid. See, how
we launch apply workers in ApplyLauncherMain(). Do you see any case
for non-tablesync workers where relid is not InvalidOid?
--
With Regards,
Amit Kapila.
On Wed, Sep 21, 2022 at 8:08 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Sep 21, 2022 at 3:09 PM Peter Smith <smithpb2250@gmail.com> wrote:
On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
...
Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.This suggestion attaches special meaning to the reild param.
Won't it seem a bit strange for the non-tablesync callers (who
probably have a perfectly valid 'relid') to have to pass an InvalidOid
relid just so they can format the correct origin name?For non-tablesync workers, relid should always be InvalidOid. See, how
we launch apply workers in ApplyLauncherMain(). Do you see any case
for non-tablesync workers where relid is not InvalidOid?
Hmm, my mistake. I was thinking more of all the calls coming from the
subscriptioncmds.c, but now that I look at it maybe none of those has
any relid either.
OK, I can unify the 2 functions as you suggested. I will post another
patch in a few days.
------
Kind Regards,
Peter Smith,
Fujitsu Australia.
On Wed, Sep 21, 2022 at 8:22 PM Peter Smith <smithpb2250@gmail.com> wrote:
...
On Wed, Sep 21, 2022 at 3:23 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
...Can't we use the existing function ReplicationOriginNameForTablesync()
by passing relid as InvalidOid for this purpose? We need a check
inside to decide which name to construct, otherwise, it should be
fine. If we agree with this, then we can change the name of the
function to something like ReplicationOriginNameForLogicalRep or
ReplicationOriginNameForLogicalRepWorkers.
...
OK, I can unify the 2 functions as you suggested. I will post another
patch in a few days.
PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.
------
Kind Regards,
Peter Smith.
Fujitsu Australia
Attachments:
v3-0001-Add-common-function-ReplicationOriginNameForLogic.patchapplication/octet-stream; name=v3-0001-Add-common-function-ReplicationOriginNameForLogic.patchDownload
From 65ad6d794bb242e43232dd34a5f843d2f4c3c3dd Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 26 Sep 2022 12:59:43 +1000
Subject: [PATCH v3] Add common function ReplicationOriginNameForLogicalRep.
Make a common replication origin name formatting function to replace multiple
snprintf() expressions. This also includes logic previously done by
ReplicationOriginNameForTablesync().
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
src/backend/commands/subscriptioncmds.c | 11 ++++++-----
src/backend/replication/logical/tablesync.c | 18 +++---------------
src/backend/replication/logical/worker.c | 29 +++++++++++++++++++++++++++--
src/include/replication/worker_internal.h | 4 ++--
4 files changed, 38 insertions(+), 24 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index f3bfcca..0793234 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -657,7 +657,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_create(originname);
/*
@@ -946,7 +946,7 @@ AlterSubscription_refresh(Subscription *sub, bool copy_data,
* origin and by this time the origin might be already
* removed. For these reasons, passing missing_ok = true.
*/
- ReplicationOriginNameForTablesync(sub->oid, relid, originname,
+ ReplicationOriginNameForLogicalRep(sub->oid, relid, originname,
sizeof(originname));
replorigin_drop_by_name(originname, true, false);
}
@@ -1315,7 +1315,8 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
char originname[NAMEDATALEN];
XLogRecPtr remote_lsn;
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginNameForLogicalRep(subid, InvalidOid,
+ originname, sizeof(originname));
originid = replorigin_by_name(originname, false);
remote_lsn = replorigin_get_progress(originid, false);
@@ -1521,7 +1522,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
* worker so passing missing_ok = true. This can happen for the states
* before SUBREL_STATE_FINISHEDCOPY.
*/
- ReplicationOriginNameForTablesync(subid, relid, originname,
+ ReplicationOriginNameForLogicalRep(subid, relid, originname,
sizeof(originname));
replorigin_drop_by_name(originname, true, false);
}
@@ -1533,7 +1534,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
RemoveSubscriptionRel(subid, InvalidOid);
/* Remove the origin tracking if exists. */
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_drop_by_name(originname, true, false);
/*
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 9e52fc4..ff0f359 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -353,7 +353,7 @@ process_syncing_tables_for_sync(XLogRecPtr current_lsn)
*/
StartTransactionCommand();
- ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
+ ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
MyLogicalRepWorker->relid,
originname,
sizeof(originname));
@@ -505,7 +505,7 @@ process_syncing_tables_for_apply(XLogRecPtr current_lsn)
* error while dropping we won't restart it to drop the
* origin. So passing missing_ok = true.
*/
- ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
+ ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
rstate->relid,
originname,
sizeof(originname));
@@ -1194,18 +1194,6 @@ ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
}
/*
- * Form the origin name for tablesync.
- *
- * Return the name in the supplied buffer.
- */
-void
-ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
- char *originname, Size szorgname)
-{
- snprintf(originname, szorgname, "pg_%u_%u", suboid, relid);
-}
-
-/*
* Start syncing the table in the sync worker.
*
* If nothing needs to be done to sync the table, we exit the worker without
@@ -1274,7 +1262,7 @@ LogicalRepSyncTableStart(XLogRecPtr *origin_startpos)
MyLogicalRepWorker->relstate == SUBREL_STATE_FINISHEDCOPY);
/* Assign the origin tracking record name. */
- ReplicationOriginNameForTablesync(MySubscription->oid,
+ ReplicationOriginNameForLogicalRep(MySubscription->oid,
MyLogicalRepWorker->relid,
originname,
sizeof(originname));
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 207a580..c2cd61a 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -365,6 +365,30 @@ static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr ls
static inline void reset_apply_error_context_info(void);
/*
+ * Form the origin name for the subscription.
+ *
+ * This is a common function for tablesync and other workers. Tablesync workers
+ * must pass a valid relid. Other callers must pass relid = InvalidOid.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+ char *originname, Size szoriginname)
+{
+ if (OidIsValid(relid))
+ {
+ /* Replication origin name for tablesync workers. */
+ snprintf(originname, szoriginname, "pg_%u_%u", suboid, relid);
+ }
+ else
+ {
+ /* Replication origin name for non-tablesync workers. */
+ snprintf(originname, szoriginname, "pg_%u", suboid);
+ }
+}
+
+/*
* Should this worker apply changes for given relation.
*
* This is mainly needed for initial relation data sync as that runs in
@@ -3679,7 +3703,7 @@ ApplyWorkerMain(Datum main_arg)
* Allocate the origin name in long-lived context for error context
* message.
*/
- ReplicationOriginNameForTablesync(MySubscription->oid,
+ ReplicationOriginNameForLogicalRep(MySubscription->oid,
MyLogicalRepWorker->relid,
originname,
sizeof(originname));
@@ -3707,7 +3731,8 @@ ApplyWorkerMain(Datum main_arg)
/* Setup replication origin tracking. */
StartTransactionCommand();
- snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+ ReplicationOriginNameForLogicalRep(MySubscription->oid, InvalidOid,
+ originname, sizeof(originname));
originid = replorigin_by_name(originname, true);
if (!OidIsValid(originid))
originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index f82bc51..40a0b06 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,8 +92,8 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
extern int logicalrep_sync_worker_count(Oid subid);
-extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
- char *originname, Size szorgname);
+extern void ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+ char *originname, Size szoriginname);
extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
extern bool AllTablesyncsReady(void);
--
1.8.3.1
On Tue, Sep 20, 2022 at 6:36 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
Hi Amit,
I think it is better to use Size. Even though, it may not fail now as
the size of names for origin will always be much lesser but it is
better if we are consistent. If we agree with this, then as a first
patch, we can make it to use Size in existing places and then
introduce this new function.OK, here is the updated patchset.
* 0001 replaces int's with Size's in the existing code
* 0002 applies Peter's patch on top of 0001
Hi Aleksander.
FYI - although it is outside the scope of this thread, I did notice at
least one other example where you might want to substitute Size for
int in the same way as your v2-0001 patch did.
e.g. Just searching code for 'snprintf' where there is some parameter
for the size I quickly found:
File: src/bin/pg_dump/pg_dump_sort.c:
static void
describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
caller:
describeDumpableObject(loop[i], buf, sizeof(buf));
~~
I expect you can find more like just this if you look harder than I did.
------
Kind Regards,
Peter Smith.
Fujitsu Australia.
Hi Peter,
PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.
LGTM except for minor issues with the formatting; fixed.
I expect you can find more like just this if you look harder than I did.
Thanks. You are right, there are more places that pass int as the
second argument of *nprintf(). I used a command:
$ grep -r nprintf ./ | perl -lne 'print if($_ !~
/nprintf\([^\,]+,\s*(sizeof|([0-9A-Z_ \-]+\,))/ )' > found.txt
... and then re-checked the results manually. This excludes patterns
like *nprintf(..., sizeof(...)) and *nprintf(..., MACRO) and leaves
only something like *nprintf(..., variable). The cases where we
subtract an integer from a Size, etc were ignored.
I don't have a strong opinion on whether we should be really worried
by this. But in case we do, here is the patch. The order of 0001 and
0002 doesn't matter.
As I understand, ecpg uses size_t rather than Size, so for this
library I used size_t. Not 100% sure if the changes I made to
src/backend/utils/adt/jsonb.c add much value. I leave this to the
committer to decide.
--
Best regards,
Aleksander Alekseev
Attachments:
v4-0002-Add-common-function-ReplicationOriginNameForLogic.patchapplication/octet-stream; name=v4-0002-Add-common-function-ReplicationOriginNameForLogic.patchDownload
From 55eb5d8980c59cd5743f941517bc3af2744d761e Mon Sep 17 00:00:00 2001
From: Peter Smith <peter.b.smith@fujitsu.com>
Date: Mon, 26 Sep 2022 12:59:43 +1000
Subject: [PATCH v4 2/2] Add common function
ReplicationOriginNameForLogicalRep.
Make a common replication origin name formatting function to replace multiple
snprintf() expressions. This also includes logic previously done by
ReplicationOriginNameForTablesync().
Peter Smith, reviewed by Aleksander Alekseev
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
src/backend/commands/subscriptioncmds.c | 15 +++++----
src/backend/replication/logical/tablesync.c | 36 +++++++--------------
src/backend/replication/logical/worker.c | 35 +++++++++++++++++---
src/include/replication/worker_internal.h | 4 +--
4 files changed, 52 insertions(+), 38 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index f3bfcca434..97594cd9b1 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -657,7 +657,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
recordDependencyOnOwner(SubscriptionRelationId, subid, owner);
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_create(originname);
/*
@@ -946,8 +946,8 @@ AlterSubscription_refresh(Subscription *sub, bool copy_data,
* origin and by this time the origin might be already
* removed. For these reasons, passing missing_ok = true.
*/
- ReplicationOriginNameForTablesync(sub->oid, relid, originname,
- sizeof(originname));
+ ReplicationOriginNameForLogicalRep(sub->oid, relid, originname,
+ sizeof(originname));
replorigin_drop_by_name(originname, true, false);
}
@@ -1315,7 +1315,8 @@ AlterSubscription(ParseState *pstate, AlterSubscriptionStmt *stmt,
char originname[NAMEDATALEN];
XLogRecPtr remote_lsn;
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginNameForLogicalRep(subid, InvalidOid,
+ originname, sizeof(originname));
originid = replorigin_by_name(originname, false);
remote_lsn = replorigin_get_progress(originid, false);
@@ -1521,8 +1522,8 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
* worker so passing missing_ok = true. This can happen for the states
* before SUBREL_STATE_FINISHEDCOPY.
*/
- ReplicationOriginNameForTablesync(subid, relid, originname,
- sizeof(originname));
+ ReplicationOriginNameForLogicalRep(subid, relid, originname,
+ sizeof(originname));
replorigin_drop_by_name(originname, true, false);
}
@@ -1533,7 +1534,7 @@ DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel)
RemoveSubscriptionRel(subid, InvalidOid);
/* Remove the origin tracking if exists. */
- snprintf(originname, sizeof(originname), "pg_%u", subid);
+ ReplicationOriginNameForLogicalRep(subid, InvalidOid, originname, sizeof(originname));
replorigin_drop_by_name(originname, true, false);
/*
diff --git a/src/backend/replication/logical/tablesync.c b/src/backend/replication/logical/tablesync.c
index 9e52fc401c..b6e0af0a76 100644
--- a/src/backend/replication/logical/tablesync.c
+++ b/src/backend/replication/logical/tablesync.c
@@ -353,10 +353,10 @@ process_syncing_tables_for_sync(XLogRecPtr current_lsn)
*/
StartTransactionCommand();
- ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
- MyLogicalRepWorker->relid,
- originname,
- sizeof(originname));
+ ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
+ MyLogicalRepWorker->relid,
+ originname,
+ sizeof(originname));
/*
* Resetting the origin session removes the ownership of the slot.
@@ -505,10 +505,10 @@ process_syncing_tables_for_apply(XLogRecPtr current_lsn)
* error while dropping we won't restart it to drop the
* origin. So passing missing_ok = true.
*/
- ReplicationOriginNameForTablesync(MyLogicalRepWorker->subid,
- rstate->relid,
- originname,
- sizeof(originname));
+ ReplicationOriginNameForLogicalRep(MyLogicalRepWorker->subid,
+ rstate->relid,
+ originname,
+ sizeof(originname));
replorigin_drop_by_name(originname, true, false);
/*
@@ -1193,18 +1193,6 @@ ReplicationSlotNameForTablesync(Oid suboid, Oid relid,
relid, GetSystemIdentifier());
}
-/*
- * Form the origin name for tablesync.
- *
- * Return the name in the supplied buffer.
- */
-void
-ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
- char *originname, Size szorgname)
-{
- snprintf(originname, szorgname, "pg_%u_%u", suboid, relid);
-}
-
/*
* Start syncing the table in the sync worker.
*
@@ -1274,10 +1262,10 @@ LogicalRepSyncTableStart(XLogRecPtr *origin_startpos)
MyLogicalRepWorker->relstate == SUBREL_STATE_FINISHEDCOPY);
/* Assign the origin tracking record name. */
- ReplicationOriginNameForTablesync(MySubscription->oid,
- MyLogicalRepWorker->relid,
- originname,
- sizeof(originname));
+ ReplicationOriginNameForLogicalRep(MySubscription->oid,
+ MyLogicalRepWorker->relid,
+ originname,
+ sizeof(originname));
if (MyLogicalRepWorker->relstate == SUBREL_STATE_DATASYNC)
{
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 188c51660e..bfa32391bf 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -364,6 +364,30 @@ static void apply_error_callback(void *arg);
static inline void set_apply_error_context_xact(TransactionId xid, XLogRecPtr lsn);
static inline void reset_apply_error_context_info(void);
+/*
+ * Form the origin name for the subscription.
+ *
+ * This is a common function for tablesync and other workers. Tablesync workers
+ * must pass a valid relid. Other callers must pass relid = InvalidOid.
+ *
+ * Return the name in the supplied buffer.
+ */
+void
+ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+ char *originname, Size szoriginname)
+{
+ if (OidIsValid(relid))
+ {
+ /* Replication origin name for tablesync workers. */
+ snprintf(originname, szoriginname, "pg_%u_%u", suboid, relid);
+ }
+ else
+ {
+ /* Replication origin name for non-tablesync workers. */
+ snprintf(originname, szoriginname, "pg_%u", suboid);
+ }
+}
+
/*
* Should this worker apply changes for given relation.
*
@@ -3679,10 +3703,10 @@ ApplyWorkerMain(Datum main_arg)
* Allocate the origin name in long-lived context for error context
* message.
*/
- ReplicationOriginNameForTablesync(MySubscription->oid,
- MyLogicalRepWorker->relid,
- originname,
- sizeof(originname));
+ ReplicationOriginNameForLogicalRep(MySubscription->oid,
+ MyLogicalRepWorker->relid,
+ originname,
+ sizeof(originname));
apply_error_callback_arg.origin_name = MemoryContextStrdup(ApplyContext,
originname);
}
@@ -3707,7 +3731,8 @@ ApplyWorkerMain(Datum main_arg)
/* Setup replication origin tracking. */
StartTransactionCommand();
- snprintf(originname, sizeof(originname), "pg_%u", MySubscription->oid);
+ ReplicationOriginNameForLogicalRep(MySubscription->oid, InvalidOid,
+ originname, sizeof(originname));
originid = replorigin_by_name(originname, true);
if (!OidIsValid(originid))
originid = replorigin_create(originname);
diff --git a/src/include/replication/worker_internal.h b/src/include/replication/worker_internal.h
index f82bc518c3..2b7114ff6d 100644
--- a/src/include/replication/worker_internal.h
+++ b/src/include/replication/worker_internal.h
@@ -92,8 +92,8 @@ extern void logicalrep_worker_wakeup_ptr(LogicalRepWorker *worker);
extern int logicalrep_sync_worker_count(Oid subid);
-extern void ReplicationOriginNameForTablesync(Oid suboid, Oid relid,
- char *originname, Size szorgname);
+extern void ReplicationOriginNameForLogicalRep(Oid suboid, Oid relid,
+ char *originname, Size szoriginname);
extern char *LogicalRepSyncTableStart(XLogRecPtr *origin_startpos);
extern bool AllTablesyncsReady(void);
--
2.37.2
v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchapplication/octet-stream; name=v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchDownload
From 0194f66dba4f599ba66971d97e8c213d95979441 Mon Sep 17 00:00:00 2001
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 27 Sep 2022 13:33:21 +0300
Subject: [PATCH v4 1/2] Pass Size/size_t as a 2nd argument of snprintf()
This is a follow-up to a932824d. The previous patch made sure the second
argument of snprintf() is passed as Size rather than int in tablesync.c.
This patch does the same change to the rest of the project.
Aleksander Alekseev, reviewed by Peter Smith
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
src/backend/replication/logical/worker.c | 4 ++--
src/backend/utils/adt/jsonb.c | 6 +++---
src/bin/pg_dump/parallel.c | 4 ++--
src/bin/pg_dump/pg_dump_sort.c | 4 ++--
src/bin/psql/command.c | 2 +-
src/common/ip.c | 4 ++--
src/interfaces/ecpg/ecpglib/execute.c | 2 +-
src/interfaces/ecpg/ecpglib/misc.c | 2 +-
src/interfaces/ecpg/ecpglib/prepare.c | 2 +-
src/interfaces/libpq/fe-connect.c | 2 +-
src/interfaces/libpq/fe-gssapi-common.c | 2 +-
src/interfaces/libpq/libpq-int.h | 2 +-
src/test/regress/pg_regress.c | 2 +-
13 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 207a5805ba..188c51660e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -349,7 +349,7 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
CmdType operation);
/* Compute GID for two_phase transactions */
-static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
+static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid);
/* Common streaming function to apply all the spooled messages */
static void apply_spooled_messages(TransactionId xid, XLogRecPtr lsn);
@@ -3480,7 +3480,7 @@ cleanup_subxact_info()
* Return the GID in the supplied buffer.
*/
static void
-TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid)
+TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid)
{
Assert(subid != InvalidRepOriginId);
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9e14922ec2..1ccdca349e 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -1592,7 +1592,7 @@ jsonb_agg_transfn(PG_FUNCTION_ARGS)
/* copy string values in the aggregate context */
char *buf = palloc(v.val.string.len + 1);
- snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+ snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
v.val.string.val = buf;
}
else if (v.type == jbvNumeric)
@@ -1763,7 +1763,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
/* copy string values in the aggregate context */
char *buf = palloc(v.val.string.len + 1);
- snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+ snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
v.val.string.val = buf;
}
else
@@ -1822,7 +1822,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
/* copy string values in the aggregate context */
char *buf = palloc(v.val.string.len + 1);
- snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+ snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
v.val.string.val = buf;
}
else if (v.type == jbvNumeric)
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index c8a70d9bc1..f5e4c63e8f 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -1106,7 +1106,7 @@ ParallelBackupEnd(ArchiveHandle *AH, ParallelState *pstate)
*/
static void
buildWorkerCommand(ArchiveHandle *AH, TocEntry *te, T_Action act,
- char *buf, int buflen)
+ char *buf, Size buflen)
{
if (act == ACT_DUMP)
snprintf(buf, buflen, "DUMP %d", te->dumpId);
@@ -1154,7 +1154,7 @@ parseWorkerCommand(ArchiveHandle *AH, TocEntry **te, T_Action *act,
*/
static void
buildWorkerResponse(ArchiveHandle *AH, TocEntry *te, T_Action act, int status,
- char *buf, int buflen)
+ char *buf, Size buflen)
{
snprintf(buf, buflen, "OK %d %d %d",
te->dumpId,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5de3241eb4..51337ff798 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -173,7 +173,7 @@ static int findLoop(DumpableObject *obj,
static void repairDependencyLoop(DumpableObject **loop,
int nLoop);
static void describeDumpableObject(DumpableObject *obj,
- char *buf, int bufsize);
+ char *buf, Size bufsize);
/*
@@ -1268,7 +1268,7 @@ repairDependencyLoop(DumpableObject **loop,
* This should probably go somewhere else...
*/
static void
-describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
+describeDumpableObject(DumpableObject *obj, char *buf, Size bufsize)
{
switch (obj->objType)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a141146e70..fc882138fb 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -5008,7 +5008,7 @@ do_watch(PQExpBuffer query_buf, double sleep)
char *title;
const char *pagerprog = NULL;
FILE *pagerpipe = NULL;
- int title_len;
+ Size title_len;
int res = 0;
#ifndef WIN32
sigset_t sigalrm_sigchld_sigint;
diff --git a/src/common/ip.c b/src/common/ip.c
index 6343f49a70..5f09be9195 100644
--- a/src/common/ip.c
+++ b/src/common/ip.c
@@ -42,7 +42,7 @@ static int getaddrinfo_unix(const char *path,
static int getnameinfo_unix(const struct sockaddr_un *sa, int salen,
char *node, int nodelen,
- char *service, int servicelen,
+ char *service, Size servicelen,
int flags);
@@ -227,7 +227,7 @@ getaddrinfo_unix(const char *path, const struct addrinfo *hintsp,
static int
getnameinfo_unix(const struct sockaddr_un *sa, int salen,
char *node, int nodelen,
- char *service, int servicelen,
+ char *service, Size servicelen,
int flags)
{
int ret;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index bd94bd4e6c..f1aa833911 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1533,7 +1533,7 @@ ecpg_build_params(struct statement *stmt)
if (stmt->command[position] == '?')
{
/* yes, replace with new style */
- int buffersize = sizeof(int) * CHAR_BIT * 10 / 3; /* a rough guess of the
+ size_t buffersize = sizeof(int) * CHAR_BIT * 10 / 3; /* a rough guess of the
* size we need */
if (!(tobeinserted = (char *) ecpg_alloc(buffersize, stmt->lineno)))
diff --git a/src/interfaces/ecpg/ecpglib/misc.c b/src/interfaces/ecpg/ecpglib/misc.c
index 1eef1ec044..d861d06993 100644
--- a/src/interfaces/ecpg/ecpglib/misc.c
+++ b/src/interfaces/ecpg/ecpglib/misc.c
@@ -267,7 +267,7 @@ ecpg_log(const char *format,...)
va_list ap;
struct sqlca_t *sqlca = ECPGget_sqlca();
const char *intl_format;
- int bufsize;
+ size_t bufsize;
char *fmt;
if (!simple_debug)
diff --git a/src/interfaces/ecpg/ecpglib/prepare.c b/src/interfaces/ecpg/ecpglib/prepare.c
index ea1146f520..5ff8feac7d 100644
--- a/src/interfaces/ecpg/ecpglib/prepare.c
+++ b/src/interfaces/ecpg/ecpglib/prepare.c
@@ -120,7 +120,7 @@ replace_variables(char **text, int lineno)
else
{
/* a rough guess of the size we need: */
- int buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
+ size_t buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
int len;
char *buffer,
*newcopy;
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 746e9b4f1e..cdccd59c54 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -7313,7 +7313,7 @@ sslVerifyProtocolRange(const char *min, const char *max)
* there (which it isn't).
*/
bool
-pqGetHomeDirectory(char *buf, int bufsize)
+pqGetHomeDirectory(char *buf, size_t bufsize)
{
#ifndef WIN32
const char *home;
diff --git a/src/interfaces/libpq/fe-gssapi-common.c b/src/interfaces/libpq/fe-gssapi-common.c
index fa08526ee2..9f03d02c32 100644
--- a/src/interfaces/libpq/fe-gssapi-common.c
+++ b/src/interfaces/libpq/fe-gssapi-common.c
@@ -83,7 +83,7 @@ pg_GSS_load_servicename(PGconn *conn)
{
OM_uint32 maj_stat,
min_stat;
- int maxlen;
+ Size maxlen;
gss_buffer_desc temp_gbuf;
char *host;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c75ed63a2c..2d570017a3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -648,7 +648,7 @@ extern char *const pgresStatus[];
extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
-extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqGetHomeDirectory(char *buf, size_t bufsize);
#ifdef ENABLE_THREAD_SAFETY
extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index dda076847a..229165a684 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -1206,7 +1206,7 @@ static char *
get_alternative_expectfile(const char *expectfile, int i)
{
char *last_dot;
- int ssize = strlen(expectfile) + 2 + 1;
+ Size ssize = strlen(expectfile) + 2 + 1;
char *tmp;
char *s;
--
2.37.2
On Tue, Sep 27, 2022 at 5:04 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:
Hi Peter,
PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.LGTM except for minor issues with the formatting; fixed.
LGTM as well. I'll push this tomorrow unless there are any more comments.
--
With Regards,
Amit Kapila.
On Mon, Oct 10, 2022 at 7:09 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Tue, Sep 27, 2022 at 5:04 PM Aleksander Alekseev
<aleksander@timescale.com> wrote:Hi Peter,
PSA patch v3 to combine the different replication origin name
formatting in a single function ReplicationOriginNameForLogicalRep as
suggested.LGTM except for minor issues with the formatting; fixed.
LGTM as well. I'll push this tomorrow unless there are any more comments.
Pushed.
--
With Regards,
Amit Kapila.
Hi Amit,
Pushed.
Thanks!
I don't have a strong opinion on whether we should be really worried
by this. But in case we do, here is the patch.
This leaves us one patch to deal with.
--
Best regards,
Aleksander Alekseev
Attachments:
v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchapplication/octet-stream; name=v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patchDownload
From 0194f66dba4f599ba66971d97e8c213d95979441 Mon Sep 17 00:00:00 2001
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 27 Sep 2022 13:33:21 +0300
Subject: [PATCH v4 1/2] Pass Size/size_t as a 2nd argument of snprintf()
This is a follow-up to a932824d. The previous patch made sure the second
argument of snprintf() is passed as Size rather than int in tablesync.c.
This patch does the same change to the rest of the project.
Aleksander Alekseev, reviewed by Peter Smith
Discussion: https://postgr.es/m/CAHut%2BPsa8hhfSE6ozUK-ih7GkQziAVAf4f3bqiXEj2nQiu-43g%40mail.gmail.com
---
src/backend/replication/logical/worker.c | 4 ++--
src/backend/utils/adt/jsonb.c | 6 +++---
src/bin/pg_dump/parallel.c | 4 ++--
src/bin/pg_dump/pg_dump_sort.c | 4 ++--
src/bin/psql/command.c | 2 +-
src/common/ip.c | 4 ++--
src/interfaces/ecpg/ecpglib/execute.c | 2 +-
src/interfaces/ecpg/ecpglib/misc.c | 2 +-
src/interfaces/ecpg/ecpglib/prepare.c | 2 +-
src/interfaces/libpq/fe-connect.c | 2 +-
src/interfaces/libpq/fe-gssapi-common.c | 2 +-
src/interfaces/libpq/libpq-int.h | 2 +-
src/test/regress/pg_regress.c | 2 +-
13 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 207a5805ba..188c51660e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -349,7 +349,7 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
CmdType operation);
/* Compute GID for two_phase transactions */
-static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
+static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid);
/* Common streaming function to apply all the spooled messages */
static void apply_spooled_messages(TransactionId xid, XLogRecPtr lsn);
@@ -3480,7 +3480,7 @@ cleanup_subxact_info()
* Return the GID in the supplied buffer.
*/
static void
-TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid)
+TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, Size szgid)
{
Assert(subid != InvalidRepOriginId);
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index 9e14922ec2..1ccdca349e 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -1592,7 +1592,7 @@ jsonb_agg_transfn(PG_FUNCTION_ARGS)
/* copy string values in the aggregate context */
char *buf = palloc(v.val.string.len + 1);
- snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+ snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
v.val.string.val = buf;
}
else if (v.type == jbvNumeric)
@@ -1763,7 +1763,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
/* copy string values in the aggregate context */
char *buf = palloc(v.val.string.len + 1);
- snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+ snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
v.val.string.val = buf;
}
else
@@ -1822,7 +1822,7 @@ jsonb_object_agg_transfn(PG_FUNCTION_ARGS)
/* copy string values in the aggregate context */
char *buf = palloc(v.val.string.len + 1);
- snprintf(buf, v.val.string.len + 1, "%s", v.val.string.val);
+ snprintf(buf, (Size) (v.val.string.len + 1), "%s", v.val.string.val);
v.val.string.val = buf;
}
else if (v.type == jbvNumeric)
diff --git a/src/bin/pg_dump/parallel.c b/src/bin/pg_dump/parallel.c
index c8a70d9bc1..f5e4c63e8f 100644
--- a/src/bin/pg_dump/parallel.c
+++ b/src/bin/pg_dump/parallel.c
@@ -1106,7 +1106,7 @@ ParallelBackupEnd(ArchiveHandle *AH, ParallelState *pstate)
*/
static void
buildWorkerCommand(ArchiveHandle *AH, TocEntry *te, T_Action act,
- char *buf, int buflen)
+ char *buf, Size buflen)
{
if (act == ACT_DUMP)
snprintf(buf, buflen, "DUMP %d", te->dumpId);
@@ -1154,7 +1154,7 @@ parseWorkerCommand(ArchiveHandle *AH, TocEntry **te, T_Action *act,
*/
static void
buildWorkerResponse(ArchiveHandle *AH, TocEntry *te, T_Action act, int status,
- char *buf, int buflen)
+ char *buf, Size buflen)
{
snprintf(buf, buflen, "OK %d %d %d",
te->dumpId,
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5de3241eb4..51337ff798 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -173,7 +173,7 @@ static int findLoop(DumpableObject *obj,
static void repairDependencyLoop(DumpableObject **loop,
int nLoop);
static void describeDumpableObject(DumpableObject *obj,
- char *buf, int bufsize);
+ char *buf, Size bufsize);
/*
@@ -1268,7 +1268,7 @@ repairDependencyLoop(DumpableObject **loop,
* This should probably go somewhere else...
*/
static void
-describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
+describeDumpableObject(DumpableObject *obj, char *buf, Size bufsize)
{
switch (obj->objType)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index a141146e70..fc882138fb 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -5008,7 +5008,7 @@ do_watch(PQExpBuffer query_buf, double sleep)
char *title;
const char *pagerprog = NULL;
FILE *pagerpipe = NULL;
- int title_len;
+ Size title_len;
int res = 0;
#ifndef WIN32
sigset_t sigalrm_sigchld_sigint;
diff --git a/src/common/ip.c b/src/common/ip.c
index 6343f49a70..5f09be9195 100644
--- a/src/common/ip.c
+++ b/src/common/ip.c
@@ -42,7 +42,7 @@ static int getaddrinfo_unix(const char *path,
static int getnameinfo_unix(const struct sockaddr_un *sa, int salen,
char *node, int nodelen,
- char *service, int servicelen,
+ char *service, Size servicelen,
int flags);
@@ -227,7 +227,7 @@ getaddrinfo_unix(const char *path, const struct addrinfo *hintsp,
static int
getnameinfo_unix(const struct sockaddr_un *sa, int salen,
char *node, int nodelen,
- char *service, int servicelen,
+ char *service, Size servicelen,
int flags)
{
int ret;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index bd94bd4e6c..f1aa833911 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1533,7 +1533,7 @@ ecpg_build_params(struct statement *stmt)
if (stmt->command[position] == '?')
{
/* yes, replace with new style */
- int buffersize = sizeof(int) * CHAR_BIT * 10 / 3; /* a rough guess of the
+ size_t buffersize = sizeof(int) * CHAR_BIT * 10 / 3; /* a rough guess of the
* size we need */
if (!(tobeinserted = (char *) ecpg_alloc(buffersize, stmt->lineno)))
diff --git a/src/interfaces/ecpg/ecpglib/misc.c b/src/interfaces/ecpg/ecpglib/misc.c
index 1eef1ec044..d861d06993 100644
--- a/src/interfaces/ecpg/ecpglib/misc.c
+++ b/src/interfaces/ecpg/ecpglib/misc.c
@@ -267,7 +267,7 @@ ecpg_log(const char *format,...)
va_list ap;
struct sqlca_t *sqlca = ECPGget_sqlca();
const char *intl_format;
- int bufsize;
+ size_t bufsize;
char *fmt;
if (!simple_debug)
diff --git a/src/interfaces/ecpg/ecpglib/prepare.c b/src/interfaces/ecpg/ecpglib/prepare.c
index ea1146f520..5ff8feac7d 100644
--- a/src/interfaces/ecpg/ecpglib/prepare.c
+++ b/src/interfaces/ecpg/ecpglib/prepare.c
@@ -120,7 +120,7 @@ replace_variables(char **text, int lineno)
else
{
/* a rough guess of the size we need: */
- int buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
+ size_t buffersize = sizeof(int) * CHAR_BIT * 10 / 3;
int len;
char *buffer,
*newcopy;
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 746e9b4f1e..cdccd59c54 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -7313,7 +7313,7 @@ sslVerifyProtocolRange(const char *min, const char *max)
* there (which it isn't).
*/
bool
-pqGetHomeDirectory(char *buf, int bufsize)
+pqGetHomeDirectory(char *buf, size_t bufsize)
{
#ifndef WIN32
const char *home;
diff --git a/src/interfaces/libpq/fe-gssapi-common.c b/src/interfaces/libpq/fe-gssapi-common.c
index fa08526ee2..9f03d02c32 100644
--- a/src/interfaces/libpq/fe-gssapi-common.c
+++ b/src/interfaces/libpq/fe-gssapi-common.c
@@ -83,7 +83,7 @@ pg_GSS_load_servicename(PGconn *conn)
{
OM_uint32 maj_stat,
min_stat;
- int maxlen;
+ Size maxlen;
gss_buffer_desc temp_gbuf;
char *host;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c75ed63a2c..2d570017a3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -648,7 +648,7 @@ extern char *const pgresStatus[];
extern void pqDropConnection(PGconn *conn, bool flushInput);
extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
-extern bool pqGetHomeDirectory(char *buf, int bufsize);
+extern bool pqGetHomeDirectory(char *buf, size_t bufsize);
#ifdef ENABLE_THREAD_SAFETY
extern pgthreadlock_t pg_g_threadlock;
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index dda076847a..229165a684 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -1206,7 +1206,7 @@ static char *
get_alternative_expectfile(const char *expectfile, int i)
{
char *last_dot;
- int ssize = strlen(expectfile) + 2 + 1;
+ Size ssize = strlen(expectfile) + 2 + 1;
char *tmp;
char *s;
--
2.37.2
Aleksander Alekseev <aleksander@timescale.com> writes:
This leaves us one patch to deal with.
[ v4-0001-Pass-Size-size_t-as-a-2nd-argument-of-snprintf.patch ]
I looked at this and am inclined to reject it. None of these
places realistically need to deal with strings longer than
MAXPATHLEN or so, let alone multiple gigabytes. So it's just
code churn, creating backpatch hazards (admittedly not big ones)
for no real gain.
regards, tom lane