Add support for specifying tables in pg_createsubscriber.
Hi hackers,
Currently, pg_createsubscriber supports converting streaming
replication to logical replication for selected databases or all
databases. However, there is no provision to replicate only a few
selected tables. For such cases, users are forced to manually set up
logical replication using individual SQL commands (CREATE PUBLICATION,
CREATE SUBSCRIPTION, etc.), which can be time-consuming and
error-prone. Extending pg_createsubscriber to support table-level
replication would significantly improve the time taken to perform the
setup.
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument. It allows users to
selectively replicate specific tables within a database instead of
defaulting to all tables. The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.
Example usage:
./pg_createsubscriber \ --database db1 \ --table 'public.t1' \ --table
'public.t2(a,b) WHERE a > 100' \ --database db2 \ --table 'public.t3'
I conducted tests comparing the patched pg_createsubscriber with
standard logical replication under various scenarios to assess
performance and flexibility. All test results represent the average of
five runs.
Scenario pg_createsubscriber Logical Replication Improvement
Two databases
(postgres and
db1 each
having 100
tables), replicate
all 100 in
postgres, 50
tables in db1
(100MB/table)
total 15GB data 2m4.823s 7m23.294s 71.85%
One DB, 100
tables, replicate
50 tables
(200 MB/table)
total 10GB data 2m47.703s 4m58.003s 43.73%
One DB, 200
tables, replicate
100 tables
(100 MB/table)
total 10GB data 3m6.476s 4m35.130s 32.22%
One DB, 100
tables, replicate
50 tables
(100MB/table)
total 5GB data 1m54.384s 2m23.719s 20.42%
These results demonstrate that pg_createsubscriber consistently
outperforms standard logical replication by 20.42% for 5GB data to
71.85% for 15GB data, the time taken reduces as the data increases.
The attached test scripts were used for all experiments.
Scenario 1 (Logical replication setup involving 50 tables across 2
databases, each containing 100 tables with 100 MB of data per table):
pg_createsubscriber_setup_multi_db.sh was used for setup, followed by
pg_createsubscriber_test_multi_db.sh to measure performance. For
logical replication, the setup was done using
logical_replication_setup_multi_db.sh, with performance measured via
logical_replication_test_multi_db.sh.
Scenario 2 and 3:
The pg_createsubscriber_setup_single_db.sh (uncomment appropriate
scenario mentioned in comments) script was used, with configuration
changes specific to Scenario 2 and Scenario 3. In both cases,
pg_createsubscriber_test_single_db.sh (uncomment appropriate scenario
mentioned in comments) was used for measuring performance. Logical
replication followed the same pattern, using
logical_replication_setup_single_db.sh (uncomment appropriate scenario
mentioned in comments) and logical_replication_test_single_db.sh
(uncomment appropriate scenario mentioned in comments) for
measurement.
Scenario 4 (Logical replication setup on 50 tables from a database
containing 100 tables, each with 100 MB of data):
pg_createsubscriber_setup_single_db.sh (without modifications) was
used for setup, and pg_createsubscriber_test_single_db.sh (without
modifications) was used for performance measurement. Logical
replication used logical_replication_setup_single_db.sh (without
modifications) for setup and logical_replication_test_single_db.sh
(without modifications) for measurement.
Thoughts?
Thanks and regards,
Shubham Khanna.
Attachments:
v1-0001-Support-tables-via-pg_createsubscriber.patchapplication/octet-stream; name=v1-0001-Support-tables-via-pg_createsubscriber.patchDownload
From 94f3bf64e9e06a9a32ce99dc795c06465ecdeb44 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 26 Jun 2025 11:11:48 +0530
Subject: [PATCH v1] Support tables via pg_createsubscriber
This patch adds support for specifying tables to be included in logical
replication publications via pg_createsubscriber. Users can now pass multiple
'--database' and '--table' options to define which tables should be published
and subscribed for each database.
Features:
1. Supports per-database table mapping using multiple '--database'/'--table'
pairs.
2. Allows optional column lists and row filters.
3. If '--table' is omitted for a database, a 'FOR ALL TABLES' publication is
created.
4. Adds TAP tests to validate combinations of database and table arguments.
This improves fine-grained control over logical replication setup and aligns
pg_createsubscriber CLI design with other tools like vacuumdb and pg_restore.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 12 +
src/bin/pg_basebackup/pg_createsubscriber.c | 230 +++++++++++++++++-
.../t/040_pg_createsubscriber.pl | 83 +++++++
3 files changed, 322 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..f22a50b2c43 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -125,6 +125,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>-f <replaceable class="parameter">table</replaceable></option></term>
+ <term><option>--table=<replaceable class="parameter">table</replaceable></option></term>
+ <listitem>
+ <para>
+ Adds a table to be included in the publication for the most recently
+ specified database. Can be repeated multiple times. The syntax
+ supports optional column lists and WHERE clauses.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
<term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 025b893a41e..e8874e13f99 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -31,6 +31,46 @@
#define DEFAULT_SUB_PORT "50432"
#define OBJECTTYPE_PUBLICATIONS 0x0001
+static char *
+pg_strcasestr(const char *haystack, const char *needle)
+{
+ size_t needlelen;
+
+ if (!haystack || !needle)
+ return NULL;
+
+ needlelen = strlen(needle);
+
+ for (; *haystack; haystack++)
+ {
+ if (pg_strncasecmp(haystack, needle, needlelen) == 0)
+ return (char *) haystack;
+ }
+
+ return NULL;
+}
+
+typedef struct TableSpec
+{
+ char *spec;
+ char *schema_name;
+ char *table_name;
+ char *column_list_raw;
+ char *where_clause_raw;
+ struct TableSpec *next;
+} TableSpec;
+
+typedef struct TableListPerDB
+{
+ char *dbname;
+ TableSpec *tables;
+ struct TableListPerDB *next;
+} TableListPerDB;
+
+static TableListPerDB * dblist_head = NULL;
+static TableListPerDB * dblist_tail = NULL;
+static TableListPerDB * dblist_cur = NULL;
+
/* Command-line options */
struct CreateSubscriberOptions
{
@@ -61,6 +101,7 @@ struct LogicalRepInfo
bool made_replslot; /* replication slot was created */
bool made_publication; /* publication was created */
+ TableSpec *tables; /* list of tables to be subscribed */
};
/*
@@ -249,6 +290,7 @@ usage(void)
printf(_(" -a, --all create subscriptions for all databases except template\n"
" databases and databases that don't allow connections\n"));
printf(_(" -d, --database=DBNAME database in which to create a subscription\n"));
+ printf(_(" -f, --table table to subscribe to; can be specified multiple times\n"));
printf(_(" -D, --pgdata=DATADIR location for the subscriber data directory\n"));
printf(_(" -n, --dry-run dry run, just show what would be done\n"));
printf(_(" -p, --subscriber-port=PORT subscriber port number (default %s)\n"), DEFAULT_SUB_PORT);
@@ -505,6 +547,7 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
else
dbinfo[i].subname = NULL;
/* Other fields will be filled later */
+ dbinfo[i].tables = NULL;
pg_log_debug("publisher(%d): publication: %s ; replication slot: %s ; connection string: %s", i,
dbinfo[i].pubname ? dbinfo[i].pubname : "(auto)",
@@ -525,6 +568,20 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
i++;
}
+ for (int j = 0; j < num_dbs; j++)
+ {
+ const char *dbname = dbinfo[j].dbname;
+
+ for (TableListPerDB * cur = dblist_head; cur != NULL; cur = cur->next)
+ {
+ if (strcmp(cur->dbname, dbname) == 0)
+ {
+ dbinfo[j].tables = cur->tables;
+ break;
+ }
+ }
+ }
+
return dbinfo;
}
@@ -1645,11 +1702,74 @@ create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo)
pg_log_info("creating publication \"%s\" in database \"%s\"",
dbinfo->pubname, dbinfo->dbname);
- appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES",
- ipubname_esc);
+ if (dbinfo->tables == NULL)
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", ipubname_esc);
+ else
+ {
+ bool first = true;
+ TableSpec *tbl = dbinfo->tables;
+
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR TABLE ", ipubname_esc);
+ while (tbl)
+ {
+ char *escaped_schema = NULL;
+ char *escaped_table = NULL;
+
+ if (!tbl->table_name || strlen(tbl->table_name) == 0)
+ pg_fatal("table name cannot be null");
+
+ if (tbl->schema_name)
+ escaped_schema = PQescapeIdentifier(conn, tbl->schema_name, strlen(tbl->schema_name));
+ escaped_table = PQescapeIdentifier(conn, tbl->table_name, strlen(tbl->table_name));
+
+ appendPQExpBuffer(str, "%s", first ? "" : ", ");
+
+ if (escaped_schema)
+ appendPQExpBuffer(str, "%s.", escaped_schema);
+ appendPQExpBuffer(str, "%s", escaped_table);
+
+ if (tbl->column_list_raw && strlen(tbl->column_list_raw) > 0)
+ appendPQExpBuffer(str, " (%s)", tbl->column_list_raw);
+
+ if (tbl->where_clause_raw && strlen(tbl->where_clause_raw) > 0)
+ appendPQExpBuffer(str, " WHERE %s", tbl->where_clause_raw);
+
+ first = false;
+ tbl = tbl->next;
+
+ if (escaped_schema)
+ PQfreemem(escaped_schema);
+
+ if (escaped_table)
+ PQfreemem(escaped_table);
+ }
+ }
pg_log_debug("command is: %s", str->data);
+ if (dry_run)
+ {
+ res = PQexec(conn, "BEGIN");
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ pg_log_error("could not begin transaction: %s", PQerrorMessage(conn));
+ disconnect_database(conn, true);
+ }
+ PQclear(res);
+
+ res = PQexec(conn, str->data);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ pg_log_error("could not create publication \"%s\" in database \"%s\": %s",
+ dbinfo->pubname, dbinfo->dbname, PQresultErrorMessage(res));
+ disconnect_database(conn, true);
+ }
+ PQclear(res);
+
+ res = PQexec(conn, "ROLLBACK");
+ PQclear(res);
+ }
+
if (!dry_run)
{
res = PQexec(conn, str->data);
@@ -2022,6 +2142,7 @@ main(int argc, char **argv)
{
{"all", no_argument, NULL, 'a'},
{"database", required_argument, NULL, 'd'},
+ {"table", required_argument, NULL, 'f'},
{"pgdata", required_argument, NULL, 'D'},
{"dry-run", no_argument, NULL, 'n'},
{"subscriber-port", required_argument, NULL, 'p'},
@@ -2109,7 +2230,7 @@ main(int argc, char **argv)
get_restricted_token();
- while ((c = getopt_long(argc, argv, "ad:D:np:P:s:t:TU:v",
+ while ((c = getopt_long(argc, argv, "ad:f:D:np:P:s:t:TU:v",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2118,6 +2239,7 @@ main(int argc, char **argv)
opt.all_dbs = true;
break;
case 'd':
+ TableListPerDB * newdb;
if (!simple_string_list_member(&opt.database_names, optarg))
{
simple_string_list_append(&opt.database_names, optarg);
@@ -2125,6 +2247,108 @@ main(int argc, char **argv)
}
else
pg_fatal("database \"%s\" specified more than once for -d/--database", optarg);
+
+ newdb = pg_malloc0(sizeof(TableListPerDB));
+ newdb->dbname = pg_strdup(optarg);
+ newdb->tables = NULL;
+ newdb->next = NULL;
+ if (dblist_tail)
+ dblist_tail->next = newdb;
+ else
+ dblist_head = newdb;
+
+ dblist_tail = newdb;
+ dblist_cur = newdb;
+
+ break;
+ case 'f':
+ TableSpec * ts;
+ char *full_table_spec;
+ char *temp_ptr;
+ char *where_start = NULL;
+ char *col_list_start = NULL;
+ char *col_list_end = NULL;
+ char *table_part_end = NULL;
+
+ if (dblist_cur == NULL)
+ pg_fatal("option --table must follow a --database");
+
+ if (strchr(optarg, ';') || strstr(optarg, "--") || strstr(optarg, "/*"))
+ pg_fatal("invalid SQL control characters in --table argument: \"%s\"", optarg);
+
+ ts = pg_malloc0(sizeof(TableSpec));
+ full_table_spec = pg_strdup(optarg);
+
+ where_start = pg_strcasestr(full_table_spec, " WHERE ");
+ if (where_start)
+ {
+ *where_start = '\0';
+ where_start += strlen(" WHERE ");
+ while (*where_start == ' ')
+ where_start++;
+ ts->where_clause_raw = pg_strdup(where_start);
+ }
+
+ col_list_start = strchr(full_table_spec, '(');
+ if (col_list_start)
+ {
+ col_list_end = strrchr(full_table_spec, ')');
+ if (!col_list_end || col_list_end < col_list_start)
+ pg_fatal("malformed column list in --table argument: \"%s\"", optarg);
+
+ *col_list_start = '\0';
+ *col_list_end = '\0';
+ ts->column_list_raw = pg_strdup(col_list_start + 1);
+ table_part_end = col_list_start;
+ }
+ else
+ table_part_end = full_table_spec + strlen(full_table_spec);
+
+ temp_ptr = strrchr(full_table_spec, '.');
+ if (temp_ptr && temp_ptr < table_part_end)
+ {
+ *temp_ptr = '\0';
+ ts->schema_name = pg_strdup(full_table_spec);
+ ts->table_name = pg_strdup(temp_ptr + 1);
+ }
+ else
+ {
+ ts->schema_name = pg_strdup("public");
+ ts->table_name = pg_strdup(full_table_spec);
+ }
+
+ if (ts->table_name)
+ {
+ size_t len = strlen(ts->table_name);
+
+ while (len > 0 && isspace((unsigned char) ts->table_name[len - 1]))
+ ts->table_name[--len] = '\0';
+ }
+
+ if (ts->schema_name)
+ {
+ size_t len = strlen(ts->schema_name);
+
+ while (len > 0 && isspace((unsigned char) ts->schema_name[len - 1]))
+ ts->schema_name[--len] = '\0';
+ }
+
+ if (!ts->table_name || strlen(ts->table_name) == 0)
+ pg_fatal("table name cannot be empty in --table argument: \"%s\"", optarg);
+
+ ts->next = NULL;
+
+ if (dblist_cur->tables == NULL)
+ dblist_cur->tables = ts;
+ else
+ {
+ TableSpec *tail = dblist_cur->tables;
+
+ while (tail->next)
+ tail = tail->next;
+ tail->next = ts;
+ }
+ pg_free(full_table_spec);
break;
case 'D':
subscriber_dir = pg_strdup(optarg);
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..3bd49630868 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,92 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Drop existing publications on database db1.
+$node_p->safe_psql(
+ $db1, qq(
+ DROP PUBLICATION test_pub1;
+ DROP PUBLICATION test_pub2;
+ DROP PUBLICATION pub1;
+));
+
+# Drop existing publications on database db2.
+$node_p->safe_psql($db2, "DROP PUBLICATION pub2");
+
+# Test: Table-level publication creation
+$node_p->safe_psql($db1, "CREATE TABLE public.t1 (id int, val text)");
+$node_p->safe_psql($db1, "CREATE TABLE public.t2 (id int, val text)");
+$node_p->safe_psql($db2,
+ "CREATE TABLE public.t3 (id int, val text, extra int)");
+
+# Initialize node_s2 as a fresh standby of node_p for table-level
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber with table-level options
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--table' => 'public.t1 (id)',
+ '--table' => 'public.t2 (val)',
+ '--database' => $db2,
+ '--table' => 'public.t3 (id, extra)',
+ ],
+ 'pg_createsubscriber runs with table-level publication (existing nodes)');
+
+# Get the publication name created by pg_createsubscriber for db1
+my $pubname1 = $node_p->safe_psql(
+ $db1, qq(
+ SELECT pubname FROM pg_publication
+ WHERE pubname LIKE 'pg_createsubscriber_%'
+ ORDER BY pubname LIMIT 1
+));
+
+# Check publication tables for db1
+my $actual1 = $node_p->safe_psql(
+ $db1, qq(
+ SELECT pubname || '|public|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname1'
+ ORDER BY tablename
+));
+is($actual1, "$pubname1|public|t1\n$pubname1|public|t2",
+ 'single publication for both tables created successfully on database db1'
+);
+
+# Get the publication name created by pg_createsubscriber for db2
+my $pubname2 = $node_p->safe_psql(
+ $db2, qq(
+ SELECT pubname FROM pg_publication
+ WHERE pubname LIKE 'pg_createsubscriber_%'
+ ORDER BY pubname LIMIT 1
+));
+
+# Check publication tables for db2
+my $actual2 = $node_p->safe_psql(
+ $db2, qq(
+ SELECT pubname || '|public|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname2'
+ ORDER BY tablename
+));
+is($actual2, "$pubname2|public|t3",
+ 'single publication for t3 created successfully on database db2');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.41.0.windows.3
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.
Do we have another example which we consider the ordering of options? I'm unsure
for it. Does getopt_long() always return parsed options with the specified order?
The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.
Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.
Per document:
```
+ <term><option>-f <replaceable class="parameter">table</replaceable></option></term>
+ <term><option>--table=<replaceable class="parameter">table</replaceable></option></term>
```
I feel using "-f" is not suitable. Let's remove the shorten option now.
Best regards,
Hayato Kuroda
FUJITSU LIMITED
On Monday, July 21, 2025 1:31 PM Shubham Khanna <khannashubham1197@gmail.com> wrote:
Hi hackers,
Currently, pg_createsubscriber supports converting streaming
replication to logical replication for selected databases or all
databases. However, there is no provision to replicate only a few
selected tables. For such cases, users are forced to manually set up
logical replication using individual SQL commands (CREATE PUBLICATION,
CREATE SUBSCRIPTION, etc.), which can be time-consuming and
error-prone. Extending pg_createsubscriber to support table-level
replication would significantly improve the time taken to perform the
setup.
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument. It allows users to
selectively replicate specific tables within a database instead of
defaulting to all tables. The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.
Example usage:
./pg_createsubscriber \ --database db1 \ --table 'public.t1' \ --table
'public.t2(a,b) WHERE a > 100' \ --database db2 \ --table 'public.t3'I conducted tests comparing the patched pg_createsubscriber with
standard logical replication under various scenarios to assess
performance and flexibility. All test results represent the average of
five runs.Thoughts?
Aside from the interface discussion, I think this is an interesting feature in
general. I got some feedback from PGConf.dev this year that many people favor
using pg_createsubscriber to accelerate the initial table synchronization phase.
However, some users prefer subscribing to a subset of tables from the publisher,
whereas pg_createsubscriber currently subscribes all tables by default. This
necessitates additional adjustments to publications and subscriptions afterward.
So, this feature could streamline the process, reducing the steps users need to
take.
Best Regards,
Hou zj
On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> wrote:
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.Do we have another example which we consider the ordering of options? I'm
unsure
for it. Does getopt_long() always return parsed options with the specified
order?The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.
We have another example to consider: pg_amcheck, which allows users to specify
multiple databases. Following this precedent, it may be beneficial to adopt a
similar style in pg_createsubscriber. E.g., Users could specify tables using
database-qualified names, such as:
./pg_createsubscriber --database db1 --table 'db1.public.t1' --table
'db1.public.t2(a,b) WHERE a > 100' --database db2 --table 'db2.public.t3'
This approach enables the tool to internally categorize specified tables by
database and create publications accordingly.
Best Regards,
Hou zj
On 2025-08-01 Fr 4:03 AM, Zhijie Hou (Fujitsu) wrote:
On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu)<kuroda.hayato@fujitsu.com> wrote:
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.Do we have another example which we consider the ordering of options? I'm
unsure
for it. Does getopt_long() always return parsed options with the specified
order?The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.We have another example to consider: pg_amcheck, which allows users to specify
multiple databases.
I don't think that's quite the point, as I understand it. pg_amcheck
might allow you to have multiple --database arguments, but I don't think
it depends on the order of arguments. You didn't answer his question
about what getopt_long() does. I don't recall if it is free to mangle
the argument order.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
On Friday, August 1, 2025 8:56 PM Andrew Dunstan <andrew@dunslane.net> wrote:
On 2025-08-01 Fr 4:03 AM, Zhijie Hou (Fujitsu) wrote:
On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu)
mailto:kuroda.hayato@fujitsu.com wrote:Dear Shubham,
The attached patch introduces a new '--table' option that can be specified
after each '--database' argument.Do we have another example which we consider the ordering of options? I'm
unsure for it. Does getopt_long() always return parsed options with the
specified order?The syntax is like that used in 'vacuumdb' and supports multiple '--table'
arguments per database, including optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right? I'm afraid
that current API has too complex.We have another example to consider: pg_amcheck, which allows users to specify
multiple databases.I don't think that's quite the point, as I understand it. pg_amcheck might
allow you to have multiple --database arguments, but I don't think it depends
on the order of arguments. You didn't answer his question about what
getopt_long() does. I don't recall if it is free to mangle the argument order.
I think you might misunderstand my proposal. I am suggesting an alternative
interface style that employs database-qualified table names, which doesn't
depend on the order of options. This style is already used by pg_amcheck when
dealing with multiple database specifications. I referenced pg_amcheck as an
example. Please see below:
--
Following this precedent, it may be beneficial to adopt a similar style in
pg_createsubscriber. E.g., Users could specify tables using database-qualified
names, such as:
./pg_createsubscriber --database db1 --table 'db1.public.t1' --table
'db1.public.t2(a,b) WHERE a > 100' --database db2 --table 'db2.public.t3'
This approach enables the tool to internally categorize specified tables by
database and create publications accordingly.
--
Best Regards,
Hou zj
On 2025-08-01 Fr 11:03 AM, Zhijie Hou (Fujitsu) wrote:
On Friday, August 1, 2025 8:56 PM Andrew Dunstan<andrew@dunslane.net> wrote:
On 2025-08-01 Fr 4:03 AM, Zhijie Hou (Fujitsu) wrote:
On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu)
mailto:kuroda.hayato@fujitsu.com wrote:Dear Shubham,
The attached patch introduces a new '--table' option that can be specified
after each '--database' argument.Do we have another example which we consider the ordering of options? I'm
unsure for it. Does getopt_long() always return parsed options with the
specified order?The syntax is like that used in 'vacuumdb' and supports multiple '--table'
arguments per database, including optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right? I'm afraid
that current API has too complex.We have another example to consider: pg_amcheck, which allows users to specify
multiple databases.I don't think that's quite the point, as I understand it. pg_amcheck might
allow you to have multiple --database arguments, but I don't think it depends
on the order of arguments. You didn't answer his question about what
getopt_long() does. I don't recall if it is free to mangle the argument order.I think you might misunderstand my proposal. I am suggesting an alternative
interface style that employs database-qualified table names, which doesn't
depend on the order of options. This style is already used by pg_amcheck when
dealing with multiple database specifications. I referenced pg_amcheck as an
example.
I simple took your own description:
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.
Maybe I need some remedial English, but to me that "after" says that
argument order is significant.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
On Saturday, August 2, 2025 12:59 AM Andrew Dunstan <andrew@dunslane.net> wrote:
On 2025-08-01 Fr 11:03 AM, Zhijie Hou (Fujitsu) wrote:
On Friday, August 1, 2025 8:56 PM Andrew Dunstan mailto:andrew@dunslane.net
wrote:
We have another example to consider: pg_amcheck, which allows users to
specify multiple databases.I don't think that's quite the point, as I understand it. pg_amcheck might
allow you to have multiple --database arguments, but I don't think it depends
on the order of arguments. You didn't answer his question about what
getopt_long() does. I don't recall if it is free to mangle the argument order.I think you might misunderstand my proposal. I am suggesting an alternative
interface style that employs database-qualified table names, which doesn't
depend on the order of options. This style is already used by pg_amcheck when
dealing with multiple database specifications. I referenced pg_amcheck as an
example.I simple took your own description: The attached patch introduces a new
'--table' option that can be specified after each '--database' argument. Maybe I
need some remedial English, but to me that "after" says that argument order is
significant.
Allow me to clarify the situation. The description you referenced is the
original interface proposed by the author in the initial email. However, it was
found to be unstable due to its reliance on the argument order. In response to
the discussion, instead of supporting the original interface, I suggested an
alternative interface to consider, which is the one that does not depend on
argument order, as I mentioned in my previous email.
Best Regards,
Hou zj
On 2025-08-01 Fr 8:24 PM, Zhijie Hou (Fujitsu) wrote:
On Saturday, August 2, 2025 12:59 AM Andrew Dunstan <andrew@dunslane.net> wrote:
On 2025-08-01 Fr 11:03 AM, Zhijie Hou (Fujitsu) wrote:
On Friday, August 1, 2025 8:56 PM Andrew Dunstan mailto:andrew@dunslane.net
wrote:
We have another example to consider: pg_amcheck, which allows users to
specify multiple databases.I don't think that's quite the point, as I understand it. pg_amcheck might
allow you to have multiple --database arguments, but I don't think it depends
on the order of arguments. You didn't answer his question about what
getopt_long() does. I don't recall if it is free to mangle the argument order.I think you might misunderstand my proposal. I am suggesting an alternative
interface style that employs database-qualified table names, which doesn't
depend on the order of options. This style is already used by pg_amcheck when
dealing with multiple database specifications. I referenced pg_amcheck as an
example.I simple took your own description: The attached patch introduces a new
'--table' option that can be specified after each '--database' argument. Maybe I
need some remedial English, but to me that "after" says that argument order is
significant.Allow me to clarify the situation. The description you referenced is the
original interface proposed by the author in the initial email. However, it was
found to be unstable due to its reliance on the argument order. In response to
the discussion, instead of supporting the original interface, I suggested an
alternative interface to consider, which is the one that does not depend on
argument order, as I mentioned in my previous email.
Apologies, then, I misread the thread.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On Fri, 1 Aug 2025 at 13:33, Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> wrote:
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.Do we have another example which we consider the ordering of options? I'm
unsure
for it. Does getopt_long() always return parsed options with the specified
order?The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.We have another example to consider: pg_amcheck, which allows users to specify
multiple databases. Following this precedent, it may be beneficial to adopt a
similar style in pg_createsubscriber. E.g., Users could specify tables using
database-qualified names, such as:./pg_createsubscriber --database db1 --table 'db1.public.t1' --table
'db1.public.t2(a,b) WHERE a > 100' --database db2 --table 'db2.public.t3'
pg_amcheck allows specifying tables as a pattern, the below note is from [1]https://www.postgresql.org/docs/devel/app-pgamcheck.html:
Patterns may be unqualified, e.g. myrel*, or they may be
schema-qualified, e.g. myschema*.myrel* or database-qualified and
schema-qualified, e.g. mydb*.myschema*.myrel*. A database-qualified
pattern will add matching databases to the list of databases to be
checked.
In pg_createsubscriber will it be using the exact spec of pg_amcheck
or will the user have to give fully qualified names?
[1]: https://www.postgresql.org/docs/devel/app-pgamcheck.html
Regards,
Vignesh
On Wednesday, August 6, 2025 7:23 PM vignesh C <vignesh21@gmail.com> wrote:
On Fri, 1 Aug 2025 at 13:33, Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
wrote:On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.Do we have another example which we consider the ordering of
options? I'm unsure for it. Does getopt_long() always return parsed
options with the specified order?The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.We have another example to consider: pg_amcheck, which allows users to
specify multiple databases. Following this precedent, it may be
beneficial to adopt a similar style in pg_createsubscriber. E.g.,
Users could specify tables using database-qualified names, such as:./pg_createsubscriber --database db1 --table 'db1.public.t1' --table
'db1.public.t2(a,b) WHERE a > 100' --database db2 --table 'db2.public.t3'pg_amcheck allows specifying tables as a pattern, the below note is from [1]:
Patterns may be unqualified, e.g. myrel*, or they may be schema-qualified, e.g.
myschema*.myrel* or database-qualified and schema-qualified, e.g.
mydb*.myschema*.myrel*. A database-qualified pattern will add matching
databases to the list of databases to be checked.In pg_createsubscriber will it be using the exact spec of pg_amcheck or will the
user have to give fully qualified names?
Both options are acceptable to me. Fully qualified names might be more familiar
to users of publication DDLs, given that regex is not supported for these
statements. So, I personally think that if we want to start with something
simple, using fully qualified names is sensible, with the possibility to extend
this functionality later if needed.
Best Regards,
Hou zj
On Fri, 8 Aug 2025 at 13:47, Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
On Wednesday, August 6, 2025 7:23 PM vignesh C <vignesh21@gmail.com> wrote:
On Fri, 1 Aug 2025 at 13:33, Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
wrote:On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.Do we have another example which we consider the ordering of
options? I'm unsure for it. Does getopt_long() always return parsed
options with the specified order?The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.We have another example to consider: pg_amcheck, which allows users to
specify multiple databases. Following this precedent, it may be
beneficial to adopt a similar style in pg_createsubscriber. E.g.,
Users could specify tables using database-qualified names, such as:./pg_createsubscriber --database db1 --table 'db1.public.t1' --table
'db1.public.t2(a,b) WHERE a > 100' --database db2 --table 'db2.public.t3'pg_amcheck allows specifying tables as a pattern, the below note is from [1]:
Patterns may be unqualified, e.g. myrel*, or they may be schema-qualified, e.g.
myschema*.myrel* or database-qualified and schema-qualified, e.g.
mydb*.myschema*.myrel*. A database-qualified pattern will add matching
databases to the list of databases to be checked.In pg_createsubscriber will it be using the exact spec of pg_amcheck or will the
user have to give fully qualified names?Both options are acceptable to me. Fully qualified names might be more familiar
to users of publication DDLs, given that regex is not supported for these
statements. So, I personally think that if we want to start with something
simple, using fully qualified names is sensible, with the possibility to extend
this functionality later if needed.
+1 for implementing this way.
Regards,
Vignesh
On Fri, Aug 8, 2025 at 1:47 PM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
On Wednesday, August 6, 2025 7:23 PM vignesh C <vignesh21@gmail.com> wrote:
On Fri, 1 Aug 2025 at 13:33, Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
wrote:On Monday, July 28, 2025 1:07 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
Dear Shubham,
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument.Do we have another example which we consider the ordering of
options? I'm unsure for it. Does getopt_long() always return parsed
options with the specified order?The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.Vacuumdb nor pg_restore do not accept multiple --database, right?
I'm afraid that current API has too complex.We have another example to consider: pg_amcheck, which allows users to
specify multiple databases. Following this precedent, it may be
beneficial to adopt a similar style in pg_createsubscriber. E.g.,
Users could specify tables using database-qualified names, such as:./pg_createsubscriber --database db1 --table 'db1.public.t1' --table
'db1.public.t2(a,b) WHERE a > 100' --database db2 --table 'db2.public.t3'pg_amcheck allows specifying tables as a pattern, the below note is from [1]:
Patterns may be unqualified, e.g. myrel*, or they may be schema-qualified, e.g.
myschema*.myrel* or database-qualified and schema-qualified, e.g.
mydb*.myschema*.myrel*. A database-qualified pattern will add matching
databases to the list of databases to be checked.In pg_createsubscriber will it be using the exact spec of pg_amcheck or will the
user have to give fully qualified names?Both options are acceptable to me. Fully qualified names might be more familiar
to users of publication DDLs, given that regex is not supported for these
statements. So, I personally think that if we want to start with something
simple, using fully qualified names is sensible, with the possibility to extend
this functionality later if needed.
Thanks for the suggestion. I have implemented your approach and
incorporated the required changes into the attached patch.
Thanks and regards,
Shubham Khanna.
Attachments:
v2-0001-Support-tables-via-pg_createsubscriber.patchapplication/octet-stream; name=v2-0001-Support-tables-via-pg_createsubscriber.patchDownload
From 5cb30d2b2af6064684247d75e00b414f537a21b9 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 26 Jun 2025 11:11:48 +0530
Subject: [PATCH v2] Support tables via pg_createsubscriber
This patch adds support for specifying tables to be included in logical
replication publications via pg_createsubscriber. Users can now pass multiple
'--database' and '--table' options to define which tables should be published
and subscribed for each database.
Features:
1. Supports per-database table mapping using multiple '--database'/'--table'
pairs.
2. Allows optional column lists and row filters.
3. If '--table' is omitted for a database, a 'FOR ALL TABLES' publication is
created.
4. Adds TAP tests to validate combinations of database and table arguments.
This improves fine-grained control over logical replication setup and aligns
pg_createsubscriber CLI design with other tools like vacuumdb and pg_restore.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 11 ++
src/bin/pg_basebackup/pg_createsubscriber.c | 173 +++++++++++++++++-
.../t/040_pg_createsubscriber.pl | 80 ++++++++
3 files changed, 262 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..2b70fc8851f 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -321,6 +321,17 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--table=<replaceable class="parameter">table</replaceable></option></term>
+ <listitem>
+ <para>
+ Adds a table to be included in the publication for the most recently
+ specified database. Can be repeated multiple times. The syntax
+ supports optional column lists and WHERE clauses.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..7fe71ece6e9 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -31,6 +31,27 @@
#define DEFAULT_SUB_PORT "50432"
#define OBJECTTYPE_PUBLICATIONS 0x0001
+typedef struct TableSpec
+{
+ char *spec;
+ char *pattern_regex;
+ char *pattern_db_regex;
+ char *pattern_schema_regex;
+ char *pattern_table_regex;
+ struct TableSpec *next;
+} TableSpec;
+
+typedef struct TableListPerDB
+{
+ char *dbname;
+ TableSpec *tables;
+ struct TableListPerDB *next;
+} TableListPerDB;
+
+static TableListPerDB * dblist_head = NULL;
+static TableListPerDB * dblist_tail = NULL;
+static TableListPerDB * dblist_cur = NULL;
+
/* Command-line options */
struct CreateSubscriberOptions
{
@@ -61,6 +82,7 @@ struct LogicalRepInfo
bool made_replslot; /* replication slot was created */
bool made_publication; /* publication was created */
+ TableSpec *tables; /* list of tables to be subscribed */
};
/*
@@ -265,6 +287,7 @@ usage(void)
printf(_(" --publication=NAME publication name\n"));
printf(_(" --replication-slot=NAME replication slot name\n"));
printf(_(" --subscription=NAME subscription name\n"));
+ printf(_(" --table table to subscribe to; can be specified multiple times\n"));
printf(_(" -V, --version output version information, then exit\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
@@ -505,6 +528,7 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
else
dbinfo[i].subname = NULL;
/* Other fields will be filled later */
+ dbinfo[i].tables = NULL;
pg_log_debug("publisher(%d): publication: %s ; replication slot: %s ; connection string: %s", i,
dbinfo[i].pubname ? dbinfo[i].pubname : "(auto)",
@@ -525,6 +549,20 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
i++;
}
+ for (int j = 0; j < num_dbs; j++)
+ {
+ const char *dbname = dbinfo[j].dbname;
+
+ for (TableListPerDB * cur = dblist_head; cur != NULL; cur = cur->next)
+ {
+ if (strcmp(cur->dbname, dbname) == 0)
+ {
+ dbinfo[j].tables = cur->tables;
+ break;
+ }
+ }
+ }
+
return dbinfo;
}
@@ -1654,11 +1692,79 @@ create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo)
pg_log_info("creating publication \"%s\" in database \"%s\"",
dbinfo->pubname, dbinfo->dbname);
- appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES",
- ipubname_esc);
+ if (dbinfo->tables == NULL)
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", ipubname_esc);
+ else
+ {
+ bool first = true;
+
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR TABLE ", ipubname_esc);
+ for (TableSpec * tbl = dbinfo->tables; tbl != NULL; tbl = tbl->next)
+ {
+ const char *params[2] = {
+ tbl->pattern_schema_regex,
+ tbl->pattern_table_regex
+ };
+
+ PGresult *tres = PQexecParams(conn, "SELECT n.nspname, c.relname "
+ "FROM pg_class c "
+ "JOIN pg_namespace n ON n.oid = c.relnamespace "
+ "WHERE n.nspname ~ $1 "
+ "AND c.relname ~ $2 "
+ "AND c.relkind IN ('r','p') "
+ "ORDER BY 1, 2",
+ 2, NULL, params, NULL, NULL, 0);
+
+ if (PQresultStatus(tres) != PGRES_TUPLES_OK)
+ pg_fatal("could not fetch tables for pattern \"%s\": %s",
+ tbl->spec, PQerrorMessage(conn));
+
+ if (PQntuples(tres) == 0)
+ pg_fatal("no matching tables found for pattern \"%s\"", tbl->spec);
+
+ for (int i = 0; i < PQntuples(tres); i++)
+ {
+ char *escaped_schema = PQescapeIdentifier(conn, PQgetvalue(tres, i, 0),
+ strlen(PQgetvalue(tres, i, 0)));
+ char *escaped_table = PQescapeIdentifier(conn, PQgetvalue(tres, i, 1),
+ strlen(PQgetvalue(tres, i, 1)));
+
+ appendPQExpBuffer(str, "%s%s.%s", first ? "" : ", ",
+ escaped_schema, escaped_table);
+
+ PQfreemem(escaped_schema);
+ PQfreemem(escaped_table);
+ first = false;
+ }
+ PQclear(tres);
+ }
+ }
pg_log_debug("command is: %s", str->data);
+ if (dry_run)
+ {
+ res = PQexec(conn, "BEGIN");
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ pg_log_error("could not begin transaction: %s", PQerrorMessage(conn));
+ disconnect_database(conn, true);
+ }
+ PQclear(res);
+
+ res = PQexec(conn, str->data);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ pg_log_error("could not create publication \"%s\" in database \"%s\": %s",
+ dbinfo->pubname, dbinfo->dbname, PQresultErrorMessage(res));
+ disconnect_database(conn, true);
+ }
+ PQclear(res);
+
+ res = PQexec(conn, "ROLLBACK");
+ PQclear(res);
+ }
+
if (!dry_run)
{
res = PQexec(conn, str->data);
@@ -2047,6 +2153,7 @@ main(int argc, char **argv)
{"replication-slot", required_argument, NULL, 3},
{"subscription", required_argument, NULL, 4},
{"clean", required_argument, NULL, 5},
+ {"table", required_argument, NULL, 6},
{NULL, 0, NULL, 0}
};
@@ -2127,6 +2234,7 @@ main(int argc, char **argv)
opt.all_dbs = true;
break;
case 'd':
+ TableListPerDB * newdb;
if (!simple_string_list_member(&opt.database_names, optarg))
{
simple_string_list_append(&opt.database_names, optarg);
@@ -2134,6 +2242,18 @@ main(int argc, char **argv)
}
else
pg_fatal("database \"%s\" specified more than once for -d/--database", optarg);
+
+ newdb = pg_malloc0(sizeof(TableListPerDB));
+ newdb->dbname = pg_strdup(optarg);
+ newdb->tables = NULL;
+ newdb->next = NULL;
+ if (dblist_tail)
+ dblist_tail->next = newdb;
+ else
+ dblist_head = newdb;
+
+ dblist_tail = newdb;
+ dblist_cur = newdb;
break;
case 'D':
subscriber_dir = pg_strdup(optarg);
@@ -2200,6 +2320,55 @@ main(int argc, char **argv)
else
pg_fatal("object type \"%s\" specified more than once for --clean", optarg);
break;
+ case 6:
+ TableSpec * ts = pg_malloc0(sizeof(TableSpec));
+ PQExpBuffer dbbuf;
+ PQExpBuffer schemabuf;
+ PQExpBuffer namebuf;
+ int encoding;
+ int dotcnt = 0;
+
+ if (!dblist_cur)
+ pg_fatal("--table specified without a preceding --database");
+
+ ts->spec = pg_strdup(optarg);
+ dbbuf = createPQExpBuffer();
+ schemabuf = createPQExpBuffer();
+ namebuf = createPQExpBuffer();
+ encoding = pg_get_encoding_from_locale(NULL, false);
+
+ patternToSQLRegex(encoding, dbbuf, schemabuf, namebuf, optarg,
+ false, false, &dotcnt);
+ if (dotcnt == 2)
+ {
+ ts->pattern_db_regex = NULL;
+ ts->pattern_schema_regex = pg_strdup(schemabuf->data);
+ ts->pattern_table_regex = pg_strdup(namebuf->data);
+ }
+ else if (dotcnt == 1)
+ {
+ ts->pattern_db_regex = NULL;
+ ts->pattern_schema_regex = pg_strdup(dbbuf->data);
+ ts->pattern_table_regex = pg_strdup(schemabuf->data);
+ }
+ else
+ pg_fatal("invalid --table specification: %s", optarg);
+
+ destroyPQExpBuffer(dbbuf);
+ destroyPQExpBuffer(schemabuf);
+ destroyPQExpBuffer(namebuf);
+ ts->next = NULL;
+ if (!dblist_cur->tables)
+ dblist_cur->tables = ts;
+ else
+ {
+ TableSpec *tail = dblist_cur->tables;
+
+ while (tail->next)
+ tail = tail->next;
+ tail->next = ts;
+ }
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..d80c3f1470f 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,89 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Declare database names
+my $db3 = 'db3';
+my $db4 = 'db4';
+
+# Create databases
+$node_p->safe_psql('postgres', "CREATE DATABASE $db3");
+$node_p->safe_psql('postgres', "CREATE DATABASE $db4");
+
+# Test: Table-level publication creation
+$node_p->safe_psql($db3, "CREATE TABLE public.t1 (id int, val text)");
+$node_p->safe_psql($db3, "CREATE TABLE public.t2 (id int, val text)");
+$node_p->safe_psql($db4,
+ "CREATE TABLE public.t3 (id int, val text, extra int)");
+
+# Initialize node_s2 as a fresh standby of node_p for table-level
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber with table-level options
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db3),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db3,
+ '--table' => "$db3.public.t1",
+ '--table' => "$db3.public.t2",
+ '--database' => $db4,
+ '--table' => "$db4.public.t3",
+ ],
+ 'pg_createsubscriber runs with table-level publication (existing nodes)');
+
+# Get the publication name created by pg_createsubscriber for db3
+my $pubname1 = $node_p->safe_psql(
+ $db3, qq(
+ SELECT pubname FROM pg_publication
+ WHERE pubname LIKE 'pg_createsubscriber_%'
+ ORDER BY pubname LIMIT 1
+));
+
+# Check publication tables for db3
+my $actual1 = $node_p->safe_psql(
+ $db3, qq(
+ SELECT pubname || '|public|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname1'
+ ORDER BY tablename
+));
+is($actual1, "$pubname1|public|t1\n$pubname1|public|t2",
+ 'single publication for both tables created successfully on database db3'
+);
+
+# Get the publication name created by pg_createsubscriber for db4
+my $pubname2 = $node_p->safe_psql(
+ $db4, qq(
+ SELECT pubname FROM pg_publication
+ WHERE pubname LIKE 'pg_createsubscriber_%'
+ ORDER BY pubname LIMIT 1
+));
+
+# Check publication tables for db4
+my $actual2 = $node_p->safe_psql(
+ $db4, qq(
+ SELECT pubname || '|public|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname2'
+ ORDER BY tablename
+));
+is($actual2, "$pubname2|public|t3",
+ 'single publication for t3 created successfully on database db4');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
Some review comments for patch v2-0001
======
1. General - compile errors!
Patch applies OK, but I cannot build pg_createsubscriber. e.g.
pg_createsubscriber.c: In function ‘main’:
pg_createsubscriber.c:2237:5: error: a label can only be part of a
statement and a declaration is not a statement
TableListPerDB * newdb;
^
pg_createsubscriber.c:2324:5: error: a label can only be part of a
statement and a declaration is not a statement
TableSpec * ts = pg_malloc0(sizeof(TableSpec));
^
pg_createsubscriber.c:2325:5: error: expected expression before ‘PQExpBuffer’
PQExpBuffer dbbuf;
^
pg_createsubscriber.c:2326:5: warning: ISO C90 forbids mixed
declarations and code [-Wdeclaration-after-statement]
PQExpBuffer schemabuf;
^
pg_createsubscriber.c:2335:5: error: ‘dbbuf’ undeclared (first use in
this function)
dbbuf = createPQExpBuffer();
I'm not sure how this can be working for you (???). You cannot declare
variables without introducing a code block in the scope of the switch
case.
======
doc/src/sgml/ref/pg_createsubscriber.sgml
2.
--table is missing from synopsis
~~~
3.
+ <term><option>--table=<replaceable
class="parameter">table</replaceable></option></term>
+ <listitem>
+ <para>
+ Adds a table to be included in the publication for the most recently
+ specified database. Can be repeated multiple times. The syntax
+ supports optional column lists and WHERE clauses.
+ </para>
+ </listitem>
+ </varlistentry>
Lacks detail, so I can't tell how to use this. e.g:
- What does the syntax actually look like?
- code suggests you can specify DB, but this doc says it only applies
to the most recent DB
- how supports column lists and WHERE clause (needs examples)
- needs rules FOR ALL TABLES, etc.
- allowed in combination with --all?
- etc.
======
src/bin/pg_basebackup/pg_createsubscriber.c
4.
+typedef struct TableListPerDB
+{
+ char *dbname;
+ TableSpec *tables;
+ struct TableListPerDB *next;
+} TableListPerDB;
I didn't understand the need for this "per-DB" structure.
Later, you declare "TableSpec *tables;" within the "LogicalRepInfo"
structure (which is per-DB) so you already have the "per-DB" table
list right there. Even if you need to maintain some global static
list, then I imagine you could just put a 'dbname' member in
TableSpec. You don't need a whole new structure to do it.
~~~
create_publication:
5.
+ if (dbinfo->tables == NULL)
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", ipubname_esc);
+ else
+ {
+ bool first = true;
+
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR TABLE ", ipubname_esc);
+ for (TableSpec * tbl = dbinfo->tables; tbl != NULL; tbl = tbl->next)
What if '*' is specified for the table name? Should that cause a
"CREATE PUBLICATION ... FOR TABLES IN SCHEMA ..." instead of making a
publication with 100s or more tables in a FOR TABLES?
~~~
6.
+ for (int i = 0; i < PQntuples(tres); i++)
+ {
+ char *escaped_schema = PQescapeIdentifier(conn, PQgetvalue(tres, i, 0),
+ strlen(PQgetvalue(tres, i, 0)));
+ char *escaped_table = PQescapeIdentifier(conn, PQgetvalue(tres, i, 1),
+ strlen(PQgetvalue(tres, i, 1)));
+
+ appendPQExpBuffer(str, "%s%s.%s", first ? "" : ", ",
+ escaped_schema, escaped_table);
+
+ PQfreemem(escaped_schema);
+ PQfreemem(escaped_table);
+ first = false;
+ }
6a.
How about some other simple variables to avoid all the repeated PQgetvalue?
e.g.
char *sch = PQgetvalue(tres, i, 0);
char *tbl = PQgetvalue(tres, i, 1);
char *escaped_schema = PQescapeIdentifier(conn, sch, strlen(sch));
char *escaped_table = PQescapeIdentifier(conn, tbl, strlen(tbl));
~
6b.
Variable 'first' is redundant. Same as checking 'i == 0'.
~~~
7.
+ if (dry_run)
+ {
+ res = PQexec(conn, "BEGIN");
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
Would it be better to use if/else instead of:
if (dry_run)
if (!dry_run)
~~~
main:
8.
+ TableListPerDB * newdb;
if (!simple_string_list_member(&opt.database_names, optarg))
{
simple_string_list_append(&opt.database_names, optarg);
8a.
Compile error. Need {} scope to declare that variable!
~
8b.
This seems a strange muddle of 'd' which represents DATABASE and
TableListPerDB, which is a list of tables (not a database). I have
doubts that most of this linked list code is even necessary for the
'd' case.
~~~
9.
+ case 6:
+ TableSpec * ts = pg_malloc0(sizeof(TableSpec));
+ PQExpBuffer dbbuf;
Compile error. Need {} scope to declare that variable!
~~~
10.
+ if (dotcnt == 2)
+ {
+ ts->pattern_db_regex = NULL;
+ ts->pattern_schema_regex = pg_strdup(schemabuf->data);
+ ts->pattern_table_regex = pg_strdup(namebuf->data);
+ }
+ else if (dotcnt == 1)
+ {
+ ts->pattern_db_regex = NULL;
+ ts->pattern_schema_regex = pg_strdup(dbbuf->data);
+ ts->pattern_table_regex = pg_strdup(schemabuf->data);
+ }
+ else
+ pg_fatal("invalid --table specification: %s", optarg);
10a.
This code seems quite odd to me:
- The DB becomes the schema?
- The schema becomes the table?
Rather than fudging all the names of the --table parts, if you don't
know what they represent up-fron,t probably it is better to call them
just part1,part2,part3.
~~~
10b.
Why is pattern_db_regex always NULL? If it is always NULL why have it at all?
======
.../t/040_pg_createsubscriber.pl
11.
+# Test: Table-level publication creation
+$node_p->safe_psql($db3, "CREATE TABLE public.t1 (id int, val text)");
+$node_p->safe_psql($db3, "CREATE TABLE public.t2 (id int, val text)");
+$node_p->safe_psql($db4,
+ "CREATE TABLE public.t3 (id int, val text, extra int)");
+
IIUC, the schema name is part of the table syntax. So, you should
include test cases for different schemas.
~~~
12.
+# Run pg_createsubscriber with table-level options
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db3),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db3,
+ '--table' => "$db3.public.t1",
+ '--table' => "$db3.public.t2",
+ '--database' => $db4,
+ '--table' => "$db4.public.t3",
+ ],
+ 'pg_createsubscriber runs with table-level publication (existing nodes)');
12a.
This is not really testing the same as what the commit message
describes. e.g. what about a test case where --table does not mention
the db explicitly, so relies on the most recent.
~
12b.
What should happen if the explicitly named DB in --table is not the
same as the most recent --database, even though it is otherwise
correct?
e.g.
'--database' => $db3,
'--table' => "$db3.public.t1",
'--database' => $db4,
'--table' => "$db3.public.t2",
'--table' => "$db4.public.t3",
I quickly tried it and AFAICT this was silently accepted and then the
test failed because it gave unexpected results. It doesn't seem good
behaviour.
~
12c.
(related to 12b/12c).
I became suspicious that the DB name part of the --table option is
completely bogus. And it is. The test still passes OK even after I
write junk in the --table database part, like below.
command_ok(
[
'pg_createsubscriber',
'--verbose',
'--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
'--pgdata' => $node_s2->data_dir,
'--publisher-server' => $node_p->connstr($db3),
'--socketdir' => $node_s2->host,
'--subscriber-port' => $node_s2->port,
'--database' => $db3,
'--table' => "$db3.public.t1",
'--table' => "REALLY???.public.t2",
'--database' => $db4,
'--table' => "$db4.public.t3",
],
'pg_createsubscriber runs with table-level publication (existing nodes)');
~~~
13.
+# Get the publication name created by pg_createsubscriber for db3
+my $pubname1 = $node_p->safe_psql(
+ $db3, qq(
+ SELECT pubname FROM pg_publication
+ WHERE pubname LIKE 'pg_createsubscriber_%'
+ ORDER BY pubname LIMIT 1
+));
+
Why don't you just name the publication explicitly in the command?
Then you don't need any of this code to discover the publication name
here.
~~~
14.
+# Check publication tables for db3
+my $actual1 = $node_p->safe_psql(
+ $db3, qq(
+ SELECT pubname || '|public|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname1'
+ ORDER BY tablename
+));
+is($actual1, "$pubname1|public|t1\n$pubname1|public|t2",
+ 'single publication for both tables created successfully on database db3'
+);
What is the point of hardwiring the 'public' in the concatenated
string, and then verifying that it is still there in the result? Why
not hardwire 'banana' instead of 'public' -- it passes the test just
the same.
~~~
15.
+# Get the publication name created by pg_createsubscriber for db4
+my $pubname2 = $node_p->safe_psql(
+ $db4, qq(
+ SELECT pubname FROM pg_publication
+ WHERE pubname LIKE 'pg_createsubscriber_%'
+ ORDER BY pubname LIMIT 1
+));
+
(same as #13 before)
Why don't you simply name the publication explicitly in the command?
Then you don't need any of this code to discover the publication name
here.
~~~
16.
+# Check publication tables for db4
+my $actual2 = $node_p->safe_psql(
+ $db4, qq(
+ SELECT pubname || '|public|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname2'
+ ORDER BY tablename
+));
+is($actual2, "$pubname2|public|t3",
+ 'single publication for t3 created successfully on database db4');
+
(same as #14 before)
What is the point of the hardwired 'public'?
======
Kind Regards,
Peter Smith.
Fujitsu Australia
Hi Shubham,
The patch claims (e.g. in the PG docs and in the commit message) that
"Column lists" and "WHERE clause" are possible, but I don't see how
they can work. AFAICT the patch assumes everything to the right of the
rightmost dot (.) must be the relation name.
~~~
WHERE Clause
------------
e.g.
If I say something like this:
'--table' => "$db3.public.t1 WHERE (id != 1234)",
Gives error like:
2025-08-18 09:41:50.295 AEST client backend[17727]
040_pg_createsubscriber.pl LOG: execute <unnamed>: SELECT n.nspname,
c.relname FROM pg_class c JOIN pg_namespace n ON n.oid =
c.relnamespace WHERE n.nspname ~ $1 AND c.relname ~ $2 AND c.relkind
IN ('r','p') ORDER BY 1, 2
2025-08-18 09:41:50.295 AEST client backend[17727]
040_pg_createsubscriber.pl DETAIL: Parameters: $1 = '^(public)$', $2
= '^(t1 where (id != 1234))$'
~~~
Column Lists
------------
Same. These don't work either...
e.g.
--table' => "$db3.public.t1(id,val)",
Gives error like:
2025-08-18 09:53:20.338 AEST client backend[19785]
040_pg_createsubscriber.pl LOG: execute <unnamed>: SELECT n.nspname,
c.relname FROM pg_class c JOIN pg_namespace n ON n.oid =
c.relnamespace WHERE n.nspname ~ $1 AND c.relname ~ $2 AND c.relkind
IN ('r','p') ORDER BY 1, 2
2025-08-18 09:53:20.338 AEST client backend[19785]
040_pg_createsubscriber.pl DETAIL: Parameters: $1 = '^(public)$', $2
= '^(t1(id,val))$'
======
Kind Regards
Peter Smith.
Fujitsu Australia
On Fri, Aug 15, 2025 at 12:46 PM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
Some review comments for patch v2-0001
======
1. General - compile errors!Patch applies OK, but I cannot build pg_createsubscriber. e.g.
pg_createsubscriber.c: In function ‘main’:
pg_createsubscriber.c:2237:5: error: a label can only be part of a
statement and a declaration is not a statement
TableListPerDB * newdb;
^
pg_createsubscriber.c:2324:5: error: a label can only be part of a
statement and a declaration is not a statement
TableSpec * ts = pg_malloc0(sizeof(TableSpec));
^
pg_createsubscriber.c:2325:5: error: expected expression before ‘PQExpBuffer’
PQExpBuffer dbbuf;
^
pg_createsubscriber.c:2326:5: warning: ISO C90 forbids mixed
declarations and code [-Wdeclaration-after-statement]
PQExpBuffer schemabuf;
^
pg_createsubscriber.c:2335:5: error: ‘dbbuf’ undeclared (first use in
this function)
dbbuf = createPQExpBuffer();I'm not sure how this can be working for you (???). You cannot declare
variables without introducing a code block in the scope of the switch
case.
Fixed.
======
doc/src/sgml/ref/pg_createsubscriber.sgml2.
--table is missing from synopsis~~~
Fixed.
3. + <term><option>--table=<replaceable class="parameter">table</replaceable></option></term> + <listitem> + <para> + Adds a table to be included in the publication for the most recently + specified database. Can be repeated multiple times. The syntax + supports optional column lists and WHERE clauses. + </para> + </listitem> + </varlistentry>Lacks detail, so I can't tell how to use this. e.g:
- What does the syntax actually look like?
- code suggests you can specify DB, but this doc says it only applies
to the most recent DB
- how supports column lists and WHERE clause (needs examples)
- needs rules FOR ALL TABLES, etc.
- allowed in combination with --all?
- etc.
Fixed.
======
src/bin/pg_basebackup/pg_createsubscriber.c4. +typedef struct TableListPerDB +{ + char *dbname; + TableSpec *tables; + struct TableListPerDB *next; +} TableListPerDB;I didn't understand the need for this "per-DB" structure.
Later, you declare "TableSpec *tables;" within the "LogicalRepInfo"
structure (which is per-DB) so you already have the "per-DB" table
list right there. Even if you need to maintain some global static
list, then I imagine you could just put a 'dbname' member in
TableSpec. You don't need a whole new structure to do it.~~~
Removed the structure TableListPerDB as suggested.
create_publication:
5. + if (dbinfo->tables == NULL) + appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", ipubname_esc); + else + { + bool first = true; + + appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR TABLE ", ipubname_esc); + for (TableSpec * tbl = dbinfo->tables; tbl != NULL; tbl = tbl->next)What if '*' is specified for the table name? Should that cause a
"CREATE PUBLICATION ... FOR TABLES IN SCHEMA ..." instead of making a
publication with 100s or more tables in a FOR TABLES?~~~
When the user specifies '*' for the table name in pg_createsubscriber,
it makes sense to generate a SQL publication command using FOR TABLES
IN SCHEMA instead of enumerating all tables explicitly. This approach
is more efficient and scalable, especially when dealing with schemas
containing hundreds or more tables, and it ensures future tables added
to the schema are automatically included without needing to recreate
the publication.
Currently, the provided patches do not implement this behavior and
always enumerate tables in the FOR TABLE clause regardless of
wildcards. I acknowledge this and plan to address the handling of the
'*' wildcard and generate FOR TABLES IN SCHEMA publications in a
future update.
6. + for (int i = 0; i < PQntuples(tres); i++) + { + char *escaped_schema = PQescapeIdentifier(conn, PQgetvalue(tres, i, 0), + strlen(PQgetvalue(tres, i, 0))); + char *escaped_table = PQescapeIdentifier(conn, PQgetvalue(tres, i, 1), + strlen(PQgetvalue(tres, i, 1))); + + appendPQExpBuffer(str, "%s%s.%s", first ? "" : ", ", + escaped_schema, escaped_table); + + PQfreemem(escaped_schema); + PQfreemem(escaped_table); + first = false; + }6a.
How about some other simple variables to avoid all the repeated PQgetvalue?e.g.
char *sch = PQgetvalue(tres, i, 0);
char *tbl = PQgetvalue(tres, i, 1);
char *escaped_schema = PQescapeIdentifier(conn, sch, strlen(sch));
char *escaped_table = PQescapeIdentifier(conn, tbl, strlen(tbl));~
Fixed.
6b.
Variable 'first' is redundant. Same as checking 'i == 0'.~~~
Fixed.
7. + if (dry_run) + { + res = PQexec(conn, "BEGIN"); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + {Would it be better to use if/else instead of:
if (dry_run)
if (!dry_run)~~~
Fixed.
main:
8.
+ TableListPerDB * newdb;
if (!simple_string_list_member(&opt.database_names, optarg))
{
simple_string_list_append(&opt.database_names, optarg);8a.
Compile error. Need {} scope to declare that variable!~
Fixed.
8b.
This seems a strange muddle of 'd' which represents DATABASE and
TableListPerDB, which is a list of tables (not a database). I have
doubts that most of this linked list code is even necessary for the
'd' case.~~~
Fixed.
9. + case 6: + TableSpec * ts = pg_malloc0(sizeof(TableSpec)); + PQExpBuffer dbbuf;Compile error. Need {} scope to declare that variable!
~~~
Fixed.
10. + if (dotcnt == 2) + { + ts->pattern_db_regex = NULL; + ts->pattern_schema_regex = pg_strdup(schemabuf->data); + ts->pattern_table_regex = pg_strdup(namebuf->data); + } + else if (dotcnt == 1) + { + ts->pattern_db_regex = NULL; + ts->pattern_schema_regex = pg_strdup(dbbuf->data); + ts->pattern_table_regex = pg_strdup(schemabuf->data); + } + else + pg_fatal("invalid --table specification: %s", optarg);10a.
This code seems quite odd to me:
- The DB becomes the schema?
- The schema becomes the table?Rather than fudging all the names of the --table parts, if you don't
know what they represent up-fron,t probably it is better to call them
just part1,part2,part3.~~~
Fixed.
10b.
Why is pattern_db_regex always NULL? If it is always NULL why have it at all?
Removed.
======
.../t/040_pg_createsubscriber.pl11. +# Test: Table-level publication creation +$node_p->safe_psql($db3, "CREATE TABLE public.t1 (id int, val text)"); +$node_p->safe_psql($db3, "CREATE TABLE public.t2 (id int, val text)"); +$node_p->safe_psql($db4, + "CREATE TABLE public.t3 (id int, val text, extra int)"); +IIUC, the schema name is part of the table syntax. So, you should
include test cases for different schemas.~~~
Fixed.
12. +# Run pg_createsubscriber with table-level options +command_ok( + [ + 'pg_createsubscriber', + '--verbose', + '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default, + '--pgdata' => $node_s2->data_dir, + '--publisher-server' => $node_p->connstr($db3), + '--socketdir' => $node_s2->host, + '--subscriber-port' => $node_s2->port, + '--database' => $db3, + '--table' => "$db3.public.t1", + '--table' => "$db3.public.t2", + '--database' => $db4, + '--table' => "$db4.public.t3", + ], + 'pg_createsubscriber runs with table-level publication (existing nodes)');12a.
This is not really testing the same as what the commit message
describes. e.g. what about a test case where --table does not mention
the db explicitly, so relies on the most recent.~
12b.
What should happen if the explicitly named DB in --table is not the
same as the most recent --database, even though it is otherwise
correct?e.g.
'--database' => $db3,
'--table' => "$db3.public.t1",
'--database' => $db4,
'--table' => "$db3.public.t2",
'--table' => "$db4.public.t3",
I quickly tried it and AFAICT this was silently accepted and then the
test failed because it gave unexpected results. It doesn't seem good
behaviour.~
12c.
(related to 12b/12c).
I became suspicious that the DB name part of the --table option is
completely bogus. And it is. The test still passes OK even after I
write junk in the --table database part, like below.command_ok(
[
'pg_createsubscriber',
'--verbose',
'--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
'--pgdata' => $node_s2->data_dir,
'--publisher-server' => $node_p->connstr($db3),
'--socketdir' => $node_s2->host,
'--subscriber-port' => $node_s2->port,
'--database' => $db3,
'--table' => "$db3.public.t1",
'--table' => "REALLY???.public.t2",
'--database' => $db4,
'--table' => "$db4.public.t3",
],
'pg_createsubscriber runs with table-level publication (existing nodes)');
~~~
v3-0002 patch handles the comment 12.
13. +# Get the publication name created by pg_createsubscriber for db3 +my $pubname1 = $node_p->safe_psql( + $db3, qq( + SELECT pubname FROM pg_publication + WHERE pubname LIKE 'pg_createsubscriber_%' + ORDER BY pubname LIMIT 1 +)); +Why don't you just name the publication explicitly in the command?
Then you don't need any of this code to discover the publication name
here.~~~
Fixed.
14. +# Check publication tables for db3 +my $actual1 = $node_p->safe_psql( + $db3, qq( + SELECT pubname || '|public|' || tablename + FROM pg_publication_tables + WHERE pubname = '$pubname1' + ORDER BY tablename +)); +is($actual1, "$pubname1|public|t1\n$pubname1|public|t2", + 'single publication for both tables created successfully on database db3' +);What is the point of hardwiring the 'public' in the concatenated
string, and then verifying that it is still there in the result? Why
not hardwire 'banana' instead of 'public' -- it passes the test just
the same.~~~
Fixed.
15. +# Get the publication name created by pg_createsubscriber for db4 +my $pubname2 = $node_p->safe_psql( + $db4, qq( + SELECT pubname FROM pg_publication + WHERE pubname LIKE 'pg_createsubscriber_%' + ORDER BY pubname LIMIT 1 +)); +(same as #13 before)
Why don't you simply name the publication explicitly in the command?
Then you don't need any of this code to discover the publication name
here.~~~
Fixed.
16. +# Check publication tables for db4 +my $actual2 = $node_p->safe_psql( + $db4, qq( + SELECT pubname || '|public|' || tablename + FROM pg_publication_tables + WHERE pubname = '$pubname2' + ORDER BY tablename +)); +is($actual2, "$pubname2|public|t3", + 'single publication for t3 created successfully on database db4'); +(same as #14 before)
What is the point of the hardwired 'public'?
Fixed.
The attached patches contain the suggested changes. They also address
the comments given by Peter at [1]/messages/by-id/CAHut+PuG8Vd=MNbQyN-3D1nsfEatmcd5bG6+L-GnOyoR9tC_6w@mail.gmail.com.
[1]: /messages/by-id/CAHut+PuG8Vd=MNbQyN-3D1nsfEatmcd5bG6+L-GnOyoR9tC_6w@mail.gmail.com
Thanks and regards,
Shubham Khanna.
Attachments:
v3-0001-Support-tables-via-pg_createsubscriber.patchapplication/octet-stream; name=v3-0001-Support-tables-via-pg_createsubscriber.patchDownload
From 15678a86b82207bd91aa0b596e112ebaed1be915 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 26 Jun 2025 11:11:48 +0530
Subject: [PATCH v3 1/2] Support tables via pg_createsubscriber
This patch adds support for specifying tables to be included in logical
replication publications via pg_createsubscriber. Users can now pass multiple
'--database' and '--table' options to define which tables should be published
and subscribed for each database.
Features:
1. Supports per-database table mapping using multiple '--database'/'--table'
pairs.
2. Allows optional column lists and row filters.
3. If '--table' is omitted for a database, a 'FOR ALL TABLES' publication is
created.
4. Adds TAP tests to validate combinations of database and table arguments.
This improves fine-grained control over logical replication setup and aligns
pg_createsubscriber CLI design with other tools like vacuumdb and pg_restore.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 57 +++++
src/bin/pg_basebackup/pg_createsubscriber.c | 236 +++++++++++++++++-
.../t/040_pg_createsubscriber.pl | 83 ++++++
3 files changed, 366 insertions(+), 10 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..ddc8777e138 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -39,6 +39,10 @@ PostgreSQL documentation
<arg choice="plain"><option>--publisher-server</option></arg>
</group>
<replaceable>connstr</replaceable>
+ <group choice="req">
+ <arg choice="plain"><option>--table</option></arg>
+ </group>
+ <replaceable>table-name</replaceable>
</group>
</cmdsynopsis>
</refsynopsisdiv>
@@ -321,6 +325,59 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--table=<replaceable class="parameter">table</replaceable></option></term>
+ <listitem>
+ <para>
+ Adds one or more specific tables to the publication for the most recently
+ specified <option>--database</option>. This option can be given multiple
+ times to include additional tables.
+ </para>
+
+ <para>
+ The argument must be a fully qualified table name in one of the
+ following forms:
+ <itemizedlist><listitem><para><literal>schema.table</literal></para></listitem>
+ <listitem><para><literal>db.schema.table</literal></para></listitem></itemizedlist>
+ If the database name is provided, it must match the most recent
+ <option>--database</option> argument.
+ </para>
+
+ <para>
+ A table specification may also include an optional column list and/or
+ row filter:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>schema.table(col1, col2, ...)</literal> — publishes
+ only the specified columns.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>schema.table WHERE (predicate)</literal> — publishes
+ only rows that satisfy the given condition.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Both forms can be combined, e.g.
+ <literal>schema.table(col1, col2) WHERE (id > 100)</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ When <option>--table</option> is specified, only the listed tables are
+ included in the publication. It cannot be combined with
+ <option>--all</option> (which publishes all databases and all tables).
+ Within a database, if no <option>--table</option> options are given, all
+ tables are included by default.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-V</option></term>
<term><option>--version</option></term>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..0017a740b72 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -31,6 +31,21 @@
#define DEFAULT_SUB_PORT "50432"
#define OBJECTTYPE_PUBLICATIONS 0x0001
+typedef struct TableSpec
+{
+ char *spec;
+ char *dbname;
+ char *pattern_regex;
+ char *pattern_part1_regex;
+ char *pattern_part2_regex;
+ char *pattern_part3_regex;
+ struct TableSpec *next;
+} TableSpec;
+
+static TableSpec * table_list_head = NULL;
+static TableSpec * table_list_tail = NULL;
+static char *current_dbname = NULL;
+
/* Command-line options */
struct CreateSubscriberOptions
{
@@ -61,6 +76,7 @@ struct LogicalRepInfo
bool made_replslot; /* replication slot was created */
bool made_publication; /* publication was created */
+ TableSpec *tables; /* list of tables to be subscribed */
};
/*
@@ -161,7 +177,6 @@ enum WaitPMResult
POSTMASTER_STILL_STARTING
};
-
/*
* Cleanup objects that were created by pg_createsubscriber if there is an
* error.
@@ -265,6 +280,7 @@ usage(void)
printf(_(" --publication=NAME publication name\n"));
printf(_(" --replication-slot=NAME replication slot name\n"));
printf(_(" --subscription=NAME subscription name\n"));
+ printf(_(" --table table to subscribe to; can be specified multiple times\n"));
printf(_(" -V, --version output version information, then exit\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\nReport bugs to <%s>.\n"), PACKAGE_BUGREPORT);
@@ -505,6 +521,7 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
else
dbinfo[i].subname = NULL;
/* Other fields will be filled later */
+ dbinfo[i].tables = NULL;
pg_log_debug("publisher(%d): publication: %s ; replication slot: %s ; connection string: %s", i,
dbinfo[i].pubname ? dbinfo[i].pubname : "(auto)",
@@ -525,6 +542,40 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
i++;
}
+ for (i = 0; i < num_dbs; i++)
+ {
+ TableSpec *prev = NULL;
+ TableSpec *cur = table_list_head;
+ TableSpec *filtered_head = NULL;
+ TableSpec *filtered_tail = NULL;
+
+ while (cur != NULL)
+ {
+ TableSpec *next = cur->next;
+
+ if (strcmp(cur->dbname, dbinfo[i].dbname) == 0)
+ {
+ if (prev)
+ prev->next = next;
+ else
+ table_list_head = next;
+
+ cur->next = NULL;
+ if (!filtered_head)
+ filtered_head = filtered_tail = cur;
+ else
+ {
+ filtered_tail->next = cur;
+ filtered_tail = cur;
+ }
+ }
+ else
+ prev = cur;
+ cur = next;
+ }
+ dbinfo[i].tables = filtered_head;
+ }
+
return dbinfo;
}
@@ -1615,6 +1666,7 @@ create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo)
PGresult *res;
char *ipubname_esc;
char *spubname_esc;
+ bool first_table = true;
Assert(conn != NULL);
@@ -1654,12 +1706,80 @@ create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo)
pg_log_info("creating publication \"%s\" in database \"%s\"",
dbinfo->pubname, dbinfo->dbname);
- appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES",
- ipubname_esc);
+ if (dbinfo->tables == NULL)
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR ALL TABLES", ipubname_esc);
+ else
+ {
+ appendPQExpBuffer(str, "CREATE PUBLICATION %s FOR TABLE ", ipubname_esc);
+
+ for (TableSpec * tbl = dbinfo->tables; tbl != NULL; tbl = tbl->next)
+ {
+ const char *params[2] = {
+ tbl->pattern_part2_regex,
+ tbl->pattern_part3_regex
+ };
+
+ PGresult *tres = PQexecParams(conn, "SELECT n.nspname, c.relname "
+ "FROM pg_class c "
+ "JOIN pg_namespace n ON n.oid = c.relnamespace "
+ "WHERE n.nspname ~ $1 "
+ "AND c.relname ~ $2 "
+ "AND c.relkind IN ('r','p') "
+ "ORDER BY 1, 2",
+ 2, NULL, params, NULL, NULL, 0);
+
+ if (PQresultStatus(tres) != PGRES_TUPLES_OK)
+ pg_fatal("could not fetch tables for pattern \"%s\": %s",
+ tbl->spec, PQerrorMessage(conn));
+
+ if (PQntuples(tres) == 0)
+ pg_fatal("no matching tables found for pattern \"%s\"", tbl->spec);
+
+ for (int i = 0; i < PQntuples(tres); i++)
+ {
+ char *sch = PQgetvalue(tres, i, 0);
+ char *relname = PQgetvalue(tres, i, 1);
+ char *escaped_schema = PQescapeIdentifier(conn, sch, strlen(sch));
+ char *escaped_table = PQescapeIdentifier(conn, relname, strlen(relname));
+
+ appendPQExpBuffer(str, "%s%s.%s",
+ first_table ? "" : ", ",
+ escaped_schema, escaped_table);
+
+ first_table = false;
+
+ PQfreemem(escaped_schema);
+ PQfreemem(escaped_table);
+ }
+ PQclear(tres);
+ }
+ }
pg_log_debug("command is: %s", str->data);
- if (!dry_run)
+ if (dry_run)
+ {
+ res = PQexec(conn, "BEGIN");
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ pg_log_error("could not begin transaction: %s", PQerrorMessage(conn));
+ disconnect_database(conn, true);
+ }
+ PQclear(res);
+
+ res = PQexec(conn, str->data);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ pg_log_error("could not create publication \"%s\" in database \"%s\": %s",
+ dbinfo->pubname, dbinfo->dbname, PQresultErrorMessage(res));
+ disconnect_database(conn, true);
+ }
+ PQclear(res);
+
+ res = PQexec(conn, "ROLLBACK");
+ PQclear(res);
+ }
+ else
{
res = PQexec(conn, str->data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
@@ -2047,6 +2167,7 @@ main(int argc, char **argv)
{"replication-slot", required_argument, NULL, 3},
{"subscription", required_argument, NULL, 4},
{"clean", required_argument, NULL, 5},
+ {"table", required_argument, NULL, 6},
{NULL, 0, NULL, 0}
};
@@ -2127,14 +2248,20 @@ main(int argc, char **argv)
opt.all_dbs = true;
break;
case 'd':
- if (!simple_string_list_member(&opt.database_names, optarg))
{
- simple_string_list_append(&opt.database_names, optarg);
- num_dbs++;
+ if (current_dbname)
+ pg_free(current_dbname);
+ current_dbname = pg_strdup(optarg);
+
+ if (!simple_string_list_member(&opt.database_names, optarg))
+ {
+ simple_string_list_append(&opt.database_names, optarg);
+ num_dbs++;
+ }
+ else
+ pg_fatal("database \"%s\" specified more than once for -d/--database", optarg);
+ break;
}
- else
- pg_fatal("database \"%s\" specified more than once for -d/--database", optarg);
- break;
case 'D':
subscriber_dir = pg_strdup(optarg);
canonicalize_path(subscriber_dir);
@@ -2200,6 +2327,95 @@ main(int argc, char **argv)
else
pg_fatal("object type \"%s\" specified more than once for --clean", optarg);
break;
+ case 6:
+ {
+ char *copy_arg;
+ char *first_dot;
+ char *second_dot;
+ char *dbname_arg = NULL;
+ char *schema_table_part;
+ TableSpec *ts;
+ PQExpBuffer dbbuf;
+ PQExpBuffer schemabuf;
+ PQExpBuffer namebuf;
+ int encoding;
+ int dotcnt;
+
+ if (!current_dbname)
+ pg_fatal("--table specified without a preceding --database");
+
+ copy_arg = pg_strdup(optarg);
+
+ first_dot = strchr(copy_arg, '.');
+ if (first_dot != NULL)
+ second_dot = strchr(first_dot + 1, '.');
+ else
+ second_dot = NULL;
+
+ if (second_dot != NULL)
+ {
+ *first_dot = '\0';
+ dbname_arg = copy_arg;
+ schema_table_part = first_dot + 1;
+ }
+ else
+ {
+ dbname_arg = NULL;
+ schema_table_part = copy_arg;
+ }
+
+ if (dbname_arg != NULL && strcmp(dbname_arg, current_dbname) != 0)
+ pg_fatal("database name in --table argument \"%s\" does not match most recent --database \"%s\"",
+ dbname_arg, current_dbname);
+
+ ts = pg_malloc0(sizeof(TableSpec));
+ dbbuf = createPQExpBuffer();
+ schemabuf = createPQExpBuffer();
+ namebuf = createPQExpBuffer();
+ encoding = pg_get_encoding_from_locale(NULL, false);
+ dotcnt = 0;
+
+ ts->spec = pg_strdup(optarg);
+ ts->dbname = pg_strdup(current_dbname);
+
+ patternToSQLRegex(encoding, dbbuf, schemabuf, namebuf,
+ schema_table_part, false, false, &dotcnt);
+
+ if (dbname_arg != NULL)
+ dotcnt++;
+
+ if (dotcnt == 2)
+ {
+ ts->pattern_part1_regex = pg_strdup(dbbuf->data);
+ ts->pattern_part2_regex = pg_strdup(schemabuf->data);
+ ts->pattern_part3_regex = namebuf->len > 0 ? pg_strdup(namebuf->data) : NULL;
+ }
+ else if (dotcnt == 1)
+ {
+ ts->pattern_part1_regex = NULL;
+ ts->pattern_part2_regex = pg_strdup(dbbuf->data);
+ ts->pattern_part3_regex = NULL;
+ }
+ else
+ pg_fatal("invalid table specification \"%s\"", optarg);
+
+ destroyPQExpBuffer(dbbuf);
+ destroyPQExpBuffer(schemabuf);
+ destroyPQExpBuffer(namebuf);
+ pg_free(copy_arg);
+
+ ts->next = NULL;
+
+ if (!table_list_head)
+ table_list_head = table_list_tail = ts;
+ else
+ table_list_tail->next = ts;
+
+ table_list_tail = ts;
+
+ break;
+ }
+
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..62e75af4bb5 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,92 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Declare database names
+my $db3 = 'db3';
+my $db4 = 'db4';
+
+# Create databases
+$node_p->safe_psql('postgres', "CREATE DATABASE $db3");
+$node_p->safe_psql('postgres', "CREATE DATABASE $db4");
+
+# Create additional schemas
+$node_p->safe_psql($db3, "CREATE SCHEMA myschema");
+$node_p->safe_psql($db4, "CREATE SCHEMA otherschema");
+
+# Test: Table-level publication creation
+$node_p->safe_psql($db3, "CREATE TABLE public.t1 (id int, val text)");
+$node_p->safe_psql($db3, "CREATE TABLE public.t2 (id int, val text)");
+$node_p->safe_psql($db3, "CREATE TABLE myschema.t4 (id int, val text)");
+
+$node_p->safe_psql($db4,
+ "CREATE TABLE public.t3 (id int, val text, extra int)");
+$node_p->safe_psql($db4,
+ "CREATE TABLE otherschema.t5 (id serial primary key, info text)");
+
+# Create explicit publications
+my $pubname1 = 'pub1';
+my $pubname2 = 'pub2';
+
+# Initialize node_s2 as a fresh standby of node_p for table-level
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber with table-level options
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db3),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--publication' => $pubname1,
+ '--publication' => $pubname2,
+ '--database' => $db3,
+ '--table' => "$db3.public.t1",
+ '--table' => "$db3.public.t2",
+ '--table' => "$db3.myschema.t4",
+ '--database' => $db4,
+ '--table' => "$db4.public.t3",
+ '--table' => "$db4.otherschema.t5",
+ ],
+ 'pg_createsubscriber runs with table-level publication (existing nodes)');
+
+# Check publication tables for db3 with public schema first
+my $actual1 = $node_p->safe_psql(
+ $db3, qq(
+ SELECT pubname || '|' || schemaname || '|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname1'
+ ORDER BY schemaname, tablename
+ )
+);
+is( $actual1,
+ "$pubname1|myschema|t4\n$pubname1|public|t1\n$pubname1|public|t2",
+ 'publication includes tables in public and myschema schemas on db3');
+
+# Check publication tables for db4, with public schema first
+my $actual2 = $node_p->safe_psql(
+ $db4, qq(
+ SELECT pubname || '|' || schemaname || '|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname2'
+ ORDER BY schemaname, tablename
+ )
+);
+is( $actual2,
+ "$pubname2|otherschema|t5\n$pubname2|public|t3",
+ 'publication includes tables in public and otherschema schemas on db4');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.41.0.windows.3
v3-0002-Support-WHERE-clause-and-COLUMN-list-in-table-arg.patchapplication/octet-stream; name=v3-0002-Support-WHERE-clause-and-COLUMN-list-in-table-arg.patchDownload
From 61623a4d40bb7f477bd0db192fe08b727bf4e1b4 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 21 Aug 2025 13:44:43 +0530
Subject: [PATCH v3 2/2] Support WHERE clause and COLUMN list in --table
argument
This patch support the specification of both a WHERE clause (row filter) and a
column list in the table specification via pg_createsubscriber, and modify the
utility's name (for example, to a more descriptive or aligned name).
For eg:-
CREATE PUBLICATION pub FOR TABLE schema.table (col1, col2) WHERE
(predicate);
---
src/bin/pg_basebackup/pg_createsubscriber.c | 49 +++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 0017a740b72..ecfb032593d 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -35,6 +35,8 @@ typedef struct TableSpec
{
char *spec;
char *dbname;
+ char *att_names;
+ char *row_filter;
char *pattern_regex;
char *pattern_part1_regex;
char *pattern_part2_regex;
@@ -143,6 +145,7 @@ static void drop_existing_subscriptions(PGconn *conn, const char *subname,
const char *dbname);
static void get_publisher_databases(struct CreateSubscriberOptions *opt,
bool dbnamespecified);
+static char *pg_strcasestr(const char *haystack, const char *needle);
#define USEC_PER_SEC 1000000
#define WAIT_INTERVAL 1 /* 1 second */
@@ -177,6 +180,24 @@ enum WaitPMResult
POSTMASTER_STILL_STARTING
};
+char *
+pg_strcasestr(const char *haystack, const char *needle)
+{
+ if (!*needle)
+ return (char *) haystack;
+ for (; *haystack; haystack++)
+ {
+ const char *h = haystack;
+ const char *n = needle;
+
+ while (*h && *n && pg_tolower((unsigned char) *h) == pg_tolower((unsigned char) *n))
+ ++h, ++n;
+ if (!*n)
+ return (char *) haystack;
+ }
+ return NULL;
+}
+
/*
* Cleanup objects that were created by pg_createsubscriber if there is an
* error.
@@ -1746,6 +1767,11 @@ create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo)
first_table ? "" : ", ",
escaped_schema, escaped_table);
+ if (tbl->att_names && strlen(tbl->att_names) > 0)
+ appendPQExpBuffer(str, " ( %s )", tbl->att_names);
+
+ if (tbl->row_filter && strlen(tbl->row_filter) > 0)
+ appendPQExpBuffer(str, " WHERE %s", tbl->row_filter);
first_table = false;
PQfreemem(escaped_schema);
@@ -2334,6 +2360,8 @@ main(int argc, char **argv)
char *second_dot;
char *dbname_arg = NULL;
char *schema_table_part;
+ char *paren_start = NULL;
+ char *where_start = NULL;
TableSpec *ts;
PQExpBuffer dbbuf;
PQExpBuffer schemabuf;
@@ -2346,6 +2374,27 @@ main(int argc, char **argv)
copy_arg = pg_strdup(optarg);
+ where_start = pg_strcasestr(copy_arg, " where ");
+ if (where_start != NULL)
+ {
+ *where_start = '\0';
+ where_start += 7;
+ }
+
+ paren_start = strchr(copy_arg, '(');
+ if (paren_start != NULL)
+ {
+ char *paren_end = strrchr(paren_start, ')');
+
+ if (!paren_end)
+ pg_fatal("unmatched '(' in --table argument \"%s\"", optarg);
+
+ *paren_start = '\0';
+ *paren_end = '\0';
+
+ paren_start++;
+ }
+
first_dot = strchr(copy_arg, '.');
if (first_dot != NULL)
second_dot = strchr(first_dot + 1, '.');
--
2.41.0.windows.3
On Thu, Aug 21, 2025, at 6:08 AM, Shubham Khanna wrote:
Attachments:
* v3-0001-Support-tables-via-pg_createsubscriber.patch
* v3-0002-Support-WHERE-clause-and-COLUMN-list-in-table-arg.patch
+ <term><option>--table=<replaceable class="parameter">table</replaceable></option></term>
+ <listitem>
+ <para>
+ Adds one or more specific tables to the publication for the most recently
+ specified <option>--database</option>. This option can be given multiple
+ times to include additional tables.
+ </para>
I think it is a really bad idea to rely on the order of options to infer which
tables are from which database. The current design about publication and
subscription names imply order but it doesn't mean subscription name must be
the next option after the publication name.
Your explanation from the initial email:
The attached patch introduces a new '--table' option that can be
specified after each '--database' argument. It allows users to
selectively replicate specific tables within a database instead of
defaulting to all tables. The syntax is like that used in 'vacuumdb'
and supports multiple '--table' arguments per database, including
optional column lists and row filters.
vacuumdb doesn't accept multiple --table options if you specify multiple
databases.
$ vacuumdb -a -t public.pgbench_branches -t public.pgbench_tellers -d db1 -d db2
vacuumdb: error: cannot vacuum all databases and a specific one at the same time
However, it accepts multiple --table options if you specify --all options.
(Although, I think it needs better error handling.)
$ vacuumdb -a -t public.pgbench_branches
vacuumdb: vacuuming database "contrib_regression"
vacuumdb: error: query failed: ERROR: relation "public.pgbench_branches" does not exist
LINE 2: VALUES ('public.pgbench_branches'::pg_catalog.regclass, NU...
^
vacuumdb: detail: Query was: WITH listed_objects (object_oid, column_list) AS (
VALUES ('public.pgbench_branches'::pg_catalog.regclass, NULL::pg_catalog.text)
)
Let's recap the initial goal: pg_createsubscriber creates a new logical replica
from a physical standby server. Your proposal is extending the tool to create a
partial logical replica but doesn't mention what you would do with the other
part; that is garbage after the conversion. I'm not convinced that the current
proposal is solid as-is.
+ <para>
+ The argument must be a fully qualified table name in one of the
+ following forms:
+ <itemizedlist><listitem><para><literal>schema.table</literal></para></listitem>
+ <listitem><para><literal>db.schema.table</literal></para></listitem></itemizedlist>
+ If the database name is provided, it must match the most recent
+ <option>--database</option> argument.
+ </para>
Why do you want to include the database if you already specified it?
+ <para>
+ A table specification may also include an optional column list and/or
+ row filter:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>schema.table(col1, col2, ...)</literal> — publishes
+ only the specified columns.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>schema.table WHERE (predicate)</literal> — publishes
+ only rows that satisfy the given condition.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Both forms can be combined, e.g.
+ <literal>schema.table(col1, col2) WHERE (id > 100)</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
This is a bad idea for some cases. Let's say your setup involves a filter that
uses only 10% of the rows from a certain table. It is better to do a manual
setup. Besides that, expressions in options can open a can of worms. In the
column list case, there might be corner cases like if you have a constraint in
a certain column and that column was not included in the column list, the setup
will fail; there isn't a cheap way to detect such cases.
It seems this proposal doesn't serve a general purpose. It is copying a *whole*
cluster to use only a subset of tables. Your task with pg_createsubscriber is
more expensive than doing a manual logical replication setup. If you have 500
tables and want to replicate only 400 tables, it doesn't seem productive to
specify 400 -t options. There are some cases like a small set of big tables
that this feature makes sense. However, I'm wondering if a post script should
be used to adjust your setup. There might be cases that involves only dozens of
tables but my experience says it is rare. My expectation is that this feature
is useful for avoiding some specific tables. Hence, the copy of the whole
cluster is worthwhile.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Hi Shubham.
Some review comments for patch v3-0001.
======
Commit message
1.
Allows optional column lists and row filters.
~
Is this right? Aren't the column lists and row filters now separated
in patch 0002?
======
doc/src/sgml/ref/pg_createsubscriber.sgml
2. synopsis
pg_createsubscriber [option...] { -d | --database }dbname { -D |
--pgdata }datadir { -P | --publisher-server }connstr { --table
}table-name
2a.
The --table should follow the --database instead of being last.
~
2b.
The brackets {--table} don't seem useful
~
2c.
Since there can be multiple tables per database, I feel an ellipsis
(...) is needed too
~
3.
+ <varlistentry>
+ <term><option>--table=<replaceable
class="parameter">table</replaceable></option></term>
+ <listitem>
This belongs more naturally immediately following --database.
~~~
4.
+ <para>
+ The argument must be a fully qualified table name in one of the
+ following forms:
+ <itemizedlist><listitem><para><literal>schema.table</literal></para></listitem>
+ <listitem><para><literal>db.schema.table</literal></para></listitem></itemizedlist>
+ If the database name is provided, it must match the most recent
+ <option>--database</option> argument.
+ </para>
4a.
It would be nicer if the SGML were less compacted.
e.g.
<itemizedlist>
<listitem><para><literal>schema.table</literal></para></listitem>
<listitem><para><literal>db.schema.table</literal></para></listitem>
</itemizedlist>
~
4b.
Why do we insist that the user must explicitly give a schema name?
Can't a missing schema just imply "public" so user doesn't have to
type so much?
~
4c.
FUNDAMENTAL SPEC QUESTION....
I am confused why this patch allows specifying the 'db' name part of
--table when you also insist it must be identical to the most recent
--database name. With this rule, it seems unnecessary and merely means
more validation code is required.
I can understand having the 'db' name part might be useful if you
would also permit the --table to be specified anywhere on the command,
but insisting it must match the most recent --database name seems to
cancel out any reason for allowing it in the first place.
~~~
5.
+ <para>
+ A table specification may also include an optional column list and/or
+ row filter:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>schema.table(col1, col2, ...)</literal> — publishes
+ only the specified columns.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>schema.table WHERE (predicate)</literal> — publishes
+ only rows that satisfy the given condition.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Both forms can be combined, e.g.
+ <literal>schema.table(col1, col2) WHERE (id > 100)</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
AFAIK, support for this was separated into patch 0002. So these docs
should be in patch 0002, not here.
======
src/bin/pg_basebackup/pg_createsubscriber.c
6.
+typedef struct TableSpec
+{
+ char *spec;
+ char *dbname;
+ char *pattern_regex;
+ char *pattern_part1_regex;
+ char *pattern_part2_regex;
+ char *pattern_part3_regex;
+ struct TableSpec *next;
+} TableSpec;
+
6a.
Add a typedefs.list entry for this new typedef, and run pg_indent.
~
6b.
pattern_regex is unused?
~
6c.
pattern_part1_regex seems assigned, but it is otherwise unused (??). Needed?
~
6d.
I had previously given a review comment suggesting names like
part1/2/3 because previously, you were using members with different
meanings depending on the tablespec. But, now it seems all the dot
parse logic is re-written, so AFAICT you are always using part1 means
dbregex; part2 means schemaregex; part3 means tableregex ... So, the
"part" names are strange in the current impl - if you really do know
what they represent, then call them by their proper names.
~~~
7.
+static TableSpec * table_list_head = NULL;
+static TableSpec * table_list_tail = NULL;
+static char *current_dbname = NULL;
7a.
Why isn't 'current_dbname' just a local variable of main()?
~
7b.
I doubt the 'tail' var is needed. See a later review comment for details.
~~~
8.
-
/*
* Cleanup objects that were created by pg_createsubscriber if there is an
* error.
Whitespace change unrelated to this patch?
~~~
usage:
9.
printf(_(" --subscription=NAME subscription name\n"));
+ printf(_(" --table table to subscribe to;
can be specified multiple times\n"));
printf(_(" -V, --version output version
information, then exit\n"));
Or, should this say "table to publish". Maybe it amounts to the same
thing, but this patch modifies CREATE PUBLICATION, not CREATE
SUBSCRIPTION.
~~~
store_pub_sub_info:
10.
+ for (i = 0; i < num_dbs; i++)
+ {
+ TableSpec *prev = NULL;
+ TableSpec *cur = table_list_head;
+ TableSpec *filtered_head = NULL;
+ TableSpec *filtered_tail = NULL;
+
+ while (cur != NULL)
+ {
+ TableSpec *next = cur->next;
+
+ if (strcmp(cur->dbname, dbinfo[i].dbname) == 0)
+ {
+ if (prev)
+ prev->next = next;
+ else
+ table_list_head = next;
+
+ cur->next = NULL;
+ if (!filtered_head)
+ filtered_head = filtered_tail = cur;
+ else
+ {
+ filtered_tail->next = cur;
+ filtered_tail = cur;
+ }
+ }
+ else
+ prev = cur;
+ cur = next;
+ }
+ dbinfo[i].tables = filtered_head;
+ }
+
10a.
Is there a bug? I have not debugged this, but when you are assigning
filtered_tail = cur, who is ensuring that "filtered" list is
NULL-terminated? e.g. I suspect the cur->next might still point to
something inappropriate.
~
10b.
I doubt all that 'filered_head/tail' logic is needed at all. Can't you
just assign directly to the head of dbinfo[i].tables list as you
encounter appropriate tables for that db? The order may become
reversed, but does that matter?
e.g. something like this
cur->next = dbinfo[i].tables ? dbinfo[i].tables.next : NULL;
dbinfo[i].tables = cur;
~~~
create_publication:
11.
PGresult *res;
char *ipubname_esc;
char *spubname_esc;
+ bool first_table = true;
This 'first_table' variable is redundant. Just check i == 0 in the loop.
~~~
12.
+ appendPQExpBuffer(str, "%s%s.%s",
+ first_table ? "" : ", ",
+ escaped_schema, escaped_table);
SUGGESTION
if (i > 0)
appendPQExpBuffer(str, ", ");
appendPQExpBuffer(str, "%s.%s", escaped_schema, escaped_table);
~~~
13.
- if (!simple_string_list_member(&opt.database_names, optarg))
{
- simple_string_list_append(&opt.database_names, optarg);
- num_dbs++;
+ if (current_dbname)
+ pg_free(current_dbname);
+ current_dbname = pg_strdup(optarg);
+
+ if (!simple_string_list_member(&opt.database_names, optarg))
+ {
+ simple_string_list_append(&opt.database_names, optarg);
+ num_dbs++;
+ }
+ else
+ pg_fatal("database \"%s\" specified more than once for
-d/--database", optarg);
+ break;
}
- else
- pg_fatal("database \"%s\" specified more than once for
-d/--database", optarg);
- break;
13a.
The scope {} is not needed. You removed the previously declared variable.
~
13b.
The pfree might be overkill. I think a few little database name
strdups leaks are hardly going to be a problem. It's up to you.
~~~
14.
+ char *copy_arg;
+ char *first_dot;
+ char *second_dot;
+ char *dbname_arg = NULL;
+ char *schema_table_part;
+ TableSpec *ts;
Does /dbname_arg/dbname_part/make more sense here?
~~~
15.
+ if (first_dot != NULL)
+ second_dot = strchr(first_dot + 1, '.');
+ else
+ second_dot = NULL;
+
+
The 'else' is not needed if you had assigned NULL at the declaration.
~~~
16.
The logic seems quite brittle. e.g. Maybe you should only parse
looking for dots to the end of the copy_arg or the first space.
Otherwise, other "dots" in the table spec will mess up your logic.
e.g.
"db.sch.tbl" -> 2 dots
"sch.tbl WHERE (x > 1.0)" -> also 2 dots
Similarly, what you called 'schema_table_part' cannot be allowed to
extend beyond any space; otherwise, it can easily return an unexpected
dotcnt.
"db.sch.tbl WHERE (x > 1.0)" -> 3 dots
~~~
17.
+ patternToSQLRegex(encoding, dbbuf, schemabuf, namebuf,
+ schema_table_part, false, false, &dotcnt);
+
+ if (dbname_arg != NULL)
+ dotcnt++;
+
+ if (dotcnt == 2)
+ {
+ ts->pattern_part1_regex = pg_strdup(dbbuf->data);
+ ts->pattern_part2_regex = pg_strdup(schemabuf->data);
+ ts->pattern_part3_regex = namebuf->len > 0 ? pg_strdup(namebuf->data) : NULL;
+ }
+ else if (dotcnt == 1)
+ {
+ ts->pattern_part1_regex = NULL;
+ ts->pattern_part2_regex = pg_strdup(dbbuf->data);
+ ts->pattern_part3_regex = NULL;
+ }
+ else
+ pg_fatal("invalid table specification \"%s\"", optarg);
Something seems very strange here...
I haven't debugged this, but I imagine if "--table sch.tab" then
dotcnt will be 1.
But then
+ ts->pattern_part1_regex = NULL;
+ ts->pattern_part2_regex = pg_strdup(dbbuf->data);
+ ts->pattern_part3_regex = NULL;
The schema regex is assigned the dbbuf->data (???) I don't trust this
logic. I saw that there are no test cases where dbpart is omitted, so
maybe this is a lurking bug?
~~~
18.
+
+ if (!table_list_head)
+ table_list_head = table_list_tail = ts;
+ else
+ table_list_tail->next = ts;
+
+ table_list_tail = ts;
+
All this 'tail' stuff looks potentially unnecessary. e.g. I think you
could simplify all this just by adding to the front of the list.
SUGGESTION
ts->next = table_list_head;
table_list_head = ts;
======
.../t/040_pg_createsubscriber.pl
19.
+# Declare database names
+my $db3 = 'db3';
+my $db4 = 'db4';
+
What do these variables achieve? e.g. Why not just use hardcoded
strings 'db1' and 'db2'.
~~~
20.
+# Test: Table-level publication creation
+$node_p->safe_psql($db3, "CREATE TABLE public.t1 (id int, val text)");
+$node_p->safe_psql($db3, "CREATE TABLE public.t2 (id int, val text)");
+$node_p->safe_psql($db3, "CREATE TABLE myschema.t4 (id int, val text)");
You could combine all these.
~~~
21.
+$node_p->safe_psql($db4,
+ "CREATE TABLE public.t3 (id int, val text, extra int)");
+$node_p->safe_psql($db4,
+ "CREATE TABLE otherschema.t5 (id serial primary key, info text)");
You could combine these.
~~~
22.
+# Create explicit publications
+my $pubname1 = 'pub1';
+my $pubname2 = 'pub2';
+
What does creating these variables achieve? e.g. Why not just use
hardcoded strings 'pub1' and 'pub2'.
~~~
23.
+# Run pg_createsubscriber with table-level options
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db3),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--publication' => $pubname1,
+ '--publication' => $pubname2,
+ '--database' => $db3,
+ '--table' => "$db3.public.t1",
+ '--table' => "$db3.public.t2",
+ '--table' => "$db3.myschema.t4",
+ '--database' => $db4,
+ '--table' => "$db4.public.t3",
+ '--table' => "$db4.otherschema.t5",
+ ],
+ 'pg_createsubscriber runs with table-level publication (existing nodes)');
+
There is no test for a --table spec where the dbpart is omitted.
~~~
24.
+# Check publication tables for db3 with public schema first
What is the significance "with public schema first" in this comment?
~~
25.
+# Check publication tables for db4, with public schema first
+my $actual2 = $node_p->safe_psql(
+ $db4, qq(
+ SELECT pubname || '|' || schemaname || '|' || tablename
+ FROM pg_publication_tables
+ WHERE pubname = '$pubname2'
+ ORDER BY schemaname, tablename
+ )
+);
+is( $actual2,
+ "$pubname2|otherschema|t5\n$pubname2|public|t3",
+ 'publication includes tables in public and otherschema schemas on db4');
+
Ditto. What is the significance "with public schema first" in this comment?
======
Kind Regards,
Peter Smith.
Fujitsu Australia
Hi Shubham,
Some brief review comments about patch v3-0002.
======
Commit message
1.
This patch support the specification of both a WHERE clause (row filter) and a
column list in the table specification via pg_createsubscriber, and modify the
utility's name (for example, to a more descriptive or aligned name).
For eg:-
CREATE PUBLICATION pub FOR TABLE schema.table (col1, col2) WHERE
(predicate);
1a.
/support/supports/
~
1b.
"and modify the utility's name (for example, to a more descriptive or
aligned name)."
/modify/modifies/ but more importantly, what does this sentence mean?
======
GENERAL
2.
Should be some docs moved to this patch
~~~
3.
Missing test cases
======
src/bin/pg_basebackup/pg_createsubscriber.c
typedef struct TableSpec:
4.
char *dbname;
+ char *att_names;
+ char *row_filter;
char *pattern_regex;
4a.
FUNDAMENTAL DESIGN QUESTION
Why do we even want to distinguish these? IIUC, all you need to know
is "char *extra;" which can represent *anything* else the user may
tack onto the end of the table name. You don't even need to parse
it... e.g. You could let the "CREATE PUBLICATION" check the syntax.
~
4b.
Current patch is not even assigning these members (??)
~~~
5.
+char *
+pg_strcasestr(const char *haystack, const char *needle)
+{
+ if (!*needle)
+ return (char *) haystack;
+ for (; *haystack; haystack++)
+ {
+ const char *h = haystack;
+ const char *n = needle;
+
+ while (*h && *n && pg_tolower((unsigned char) *h) ==
pg_tolower((unsigned char) *n))
+ ++h, ++n;
+ if (!*n)
+ return (char *) haystack;
+ }
+ return NULL;
+}
+
Not needed.
~~~
create_publication:
6.
+ if (tbl->att_names && strlen(tbl->att_names) > 0)
+ appendPQExpBuffer(str, " ( %s )", tbl->att_names);
+
+ if (tbl->row_filter && strlen(tbl->row_filter) > 0)
+ appendPQExpBuffer(str, " WHERE %s", tbl->row_filter);
Overkill? I imagine this could all be replaced with something simple
without trying to distinguish between them.
if (tbl->extra)
appendPQExpBuffer(str, " %s", tbl->extra);
~~~
main:
7.
+ where_start = pg_strcasestr(copy_arg, " where ");
+ if (where_start != NULL)
+ {
+ *where_start = '\0';
+ where_start += 7;
+ }
+
+ paren_start = strchr(copy_arg, '(');
+ if (paren_start != NULL)
+ {
+ char *paren_end = strrchr(paren_start, ')');
+
+ if (!paren_end)
+ pg_fatal("unmatched '(' in --table argument \"%s\"", optarg);
+
+ *paren_start = '\0';
+ *paren_end = '\0';
+
+ paren_start++;
+ }
+
Overkill? IIUC, all you need to know is whether there's something
beyond the table name. Just looking for a space ' ' or a parenthesis
'(' delimiter could be enough, right?
e.g. Something like:
char *extra = strpbk(copy_arg, ' (');
if (extra)
ts->extra = extra + 1;
======
Kind Regards,
Peter Smith.
Fujitsu Australia
On Friday, August 22, 2025 11:19 AM Euler Taveira <euler@eulerto.com> wrote:
On Thu, Aug 21, 2025, at 6:08 AM, Shubham Khanna wrote:
Attachments:
* v3-0001-Support-tables-via-pg_createsubscriber.patch
* v3-0002-Support-WHERE-clause-and-COLUMN-list-in-table-arg.patch+ <term><option>--table=<replaceable class="parameter">table</replaceable></option></term> + <listitem> + <para> + Adds one or more specific tables to the publication for the most recently + specified <option>--database</option>. This option can be given multiple + times to include additional tables. + </para>I think it is a really bad idea to rely on the order of options to infer which tables
are from which database. The current design about publication and
subscription names imply order but it doesn't mean subscription name must
be the next option after the publication name.
The documentation appears incorrect and needs revision. The latest version no
longer depends on the option order; instead, it requires users to provide
database-qualified table names, such as -t "db1.sch1.tb1". This adjustment
allows the command to internally categorize tables by their target database.
Let's recap the initial goal: pg_createsubscriber creates a new logical replica
from a physical standby server. Your proposal is extending the tool to create a
partial logical replica but doesn't mention what you would do with the other
part; that is garbage after the conversion. I'm not convinced that the current
proposal is solid as-is.
I think we can explore extending the existing --clean option in a separate patch
to support table cleanup. This option is implemented in a way that allows adding
further cleanup objects later, so it should be easy to extend it for table.
Prior to this extension, it should be noted in the documentation that users are
required to clean up the tables themselves.
+ <para> + The argument must be a fully qualified table name in one of the + following forms: + <itemizedlist><listitem><para><literal>schema.table</literal></para></li stitem> + <listitem><para><literal>db.schema.table</literal></para></listitem></it emizedlist> + If the database name is provided, it must match the most recent + <option>--database</option> argument. + </para>Why do you want to include the database if you already specified it?
As mentioned earlier, this document needs to be corrected.
+ <para> + A table specification may also include an optional column list and/or + row filter: + <itemizedlist> + <listitem> + <para> + <literal>schema.table(col1, col2, ...)</literal> — publishes + only the specified columns. + </para> + </listitem> + <listitem> + <para> + <literal>schema.table WHERE (predicate)</literal> — publishes + only rows that satisfy the given condition. + </para> + </listitem> + <listitem> + <para> + Both forms can be combined, e.g. + <literal>schema.table(col1, col2) WHERE (id > 100)</literal>. + </para> + </listitem> + </itemizedlist> + </para>This is a bad idea for some cases. Let's say your setup involves a filter that uses
only 10% of the rows from a certain table. It is better to do a manual setup.
Besides that, expressions in options can open a can of worms. In the column
list case, there might be corner cases like if you have a constraint in a certain
column and that column was not included in the column list, the setup will fail;
there isn't a cheap way to detect such cases.
I agree that supporting row filter and column list is not straightforward, and
we can consider it separately and do not implement that in the first version.
It seems this proposal doesn't serve a general purpose. It is copying a *whole*
cluster to use only a subset of tables. Your task with pg_createsubscriber is
more expensive than doing a manual logical replication setup. If you have 500
tables and want to replicate only 400 tables, it doesn't seem productive to
specify 400 -t options.
Specifying multiple -t options should not be problematic, as users has already
done similar things for "FOR TABLE" publication DDLs. I think it's not hard
for user to convert FOR TABLE list to -t option list.
There are some cases like a small set of big tables that
this feature makes sense. However, I'm wondering if a post script should be
used to adjust your setup.
I think it's not very convenient for users to perform this conversion manually.
I've learned in PGConf.dev this year that some users avoid using
pg_createsubscriber because they are unsure of the standard steps required to
convert it into subset table replication. Automating this process would be
beneficial, enabling more users to use pg_createsubscriber and take advantage of
the rapid initial table synchronization.
There might be cases that involves only dozens of
tables but my experience says it is rare. My expectation is that this feature is
useful for avoiding some specific tables. Hence, the copy of the whole cluster is
worthwhile.
We could consider adding an --exclude-table option later if needed. However, as
mentioned earlier, I think specifying multiple -t options can address this use
case as well.
Best Regards,
Hou zj
On Fri, Aug 22, 2025, at 6:57 AM, Zhijie Hou (Fujitsu) wrote:
The documentation appears incorrect and needs revision. The latest version no
longer depends on the option order; instead, it requires users to provide
database-qualified table names, such as -t "db1.sch1.tb1". This adjustment
allows the command to internally categorize tables by their target database.
I don't like this design. There is no tool that uses 3 elements. It is also
confusing and redundant to have the database in the --database option and also
in the --table option.
I'm wondering if we allow using a specified publication is a better UI. If you
specify --publication and it exists on primary, use it. The current behavior is
a failure if the publication exists. It changes the current behavior but I
don't expect someone relying on this failure to abort the execution. Moreover,
the error message was added to allow only FOR ALL TABLES; the proposal is to
relax this restriction.
I think we can explore extending the existing --clean option in a separate patch
to support table cleanup. This option is implemented in a way that allows adding
further cleanup objects later, so it should be easy to extend it for table.
Prior to this extension, it should be noted in the documentation that users are
required to clean up the tables themselves.
I would say that these cleanup feature (starting with the cleanup databases) is
equally important as the feature that selects specific objects.
I agree that supporting row filter and column list is not straightforward, and
we can consider it separately and do not implement that in the first version.
The proposal above would allow it with no additional lines of code.
It seems this proposal doesn't serve a general purpose. It is copying a *whole*
cluster to use only a subset of tables. Your task with pg_createsubscriber is
more expensive than doing a manual logical replication setup. If you have 500
tables and want to replicate only 400 tables, it doesn't seem productive to
specify 400 -t options.Specifying multiple -t options should not be problematic, as users has already
done similar things for "FOR TABLE" publication DDLs. I think it's not hard
for user to convert FOR TABLE list to -t option list.
Of course it is. Shell limits the number of arguments.
There are some cases like a small set of big tables that
this feature makes sense. However, I'm wondering if a post script should be
used to adjust your setup.I think it's not very convenient for users to perform this conversion manually.
I've learned in PGConf.dev this year that some users avoid using
pg_createsubscriber because they are unsure of the standard steps required to
convert it into subset table replication. Automating this process would be
beneficial, enabling more users to use pg_createsubscriber and take advantage of
the rapid initial table synchronization.
You missed my point. I'm not talking about manually converting a physical
replica into a logical replica. I'm talking about the plain logical replication
setup (CREATE PUBLICATION, CREATE SUBSCRIPTION). IME this tool is beneficial
for large clusters that we want to replicate (almost) all tables.
--
Euler Taveira
EDB https://www.enterprisedb.com/
IIUC, the only purpose of the proposed '--table' spec is to allow some
users the ability to tweak the default "CREATE PUBLICATION p FOR ALL
TABLES;" that the 'pg_createsubscriber' tool would otherwise
construct.
But --table is introducing problems too: e.g.
- tricky option ordering rules (e.g. most recent --database if dbname
not specified?)
- too many --table may be needed; redundant repeating of databases and
schemas also seems very verbose.
- limitations for syntax (e.g. what about FOR TABLES IN SCHEMA?)
- limitations for future syntax (e.g. what about SEQUENCES?; what about EXCEPT?)
Can these problems disappear just by slightly changing the meaning of
the existing '--publication' option to allow specifying more than just
the publication name: e.g.
--publication = 'pub1' ==> "CREATE PUBLICATION pub1 FOR ALL TABLES;"
--publication = 'pub1 FOR TABLE t1,t2(c1,c2),t3' ==> "CREATE
PUBLICATION pub1 FOR TABLE t1,t2(c1,c2),t3;"
--publication = 'pub1 FOR TABLES IN SCHEMA s1' ==> "CREATE PUBLICATION
pub1 FOR TABLES IN SCHEMA s1;"
--publication = 'pub1 FOR ALL TABLES WITH
(publish_via_partition_root)' ==> "CREATE PUBLICATION pub1 FOR ALL
TABLES WITH (publish_via_partition_root);"
Here:
- no new ordering rules; --publication/--database rules are already
well-defined; nothing to do.
- no new options; minimal document changes
- no limitations on existing CREATE PUBLICATION syntax; minimal new code changes
- future proof for new CREATE PUBLICATION syntax; zero code changes
- expanding the '--publication' meaning could also be compatible with
Euler's idea [1]/messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com to reuse existing publications
~~~
This idea could be error-prone, but it's only for a small subset of
users who need more flexibility than the default pg_createsubscriber
provides. And while it might be typo-prone, so is --table (e.g.,
--table 'db.typo.doesnotexist' is currently allowed). Did you consider
this approach already? The thread seems to have started with --table
as the assumed solution without mentioning if other options were
discussed.
Thoughts?
======
[1]: /messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com
Kind Regards,
Peter Smith.
Fujitsu Australia.
On Friday, August 22, 2025 11:26 PM Euler Taveira <euler@eulerto.com> wrote:
On Fri, Aug 22, 2025, at 6:57 AM, Zhijie Hou (Fujitsu) wrote:
The documentation appears incorrect and needs revision. The latest
version no longer depends on the option order; instead, it requires
users to provide database-qualified table names, such as -t
"db1.sch1.tb1". This adjustment allows the command to internally categorizetables by their target database.
I don't like this design. There is no tool that uses 3 elements. It is also confusing
and redundant to have the database in the --database option and also in the
--table option.I'm wondering if we allow using a specified publication is a better UI. If you
specify --publication and it exists on primary, use it. The current behavior is a
failure if the publication exists. It changes the current behavior but I don't
expect someone relying on this failure to abort the execution. Moreover, the
error message was added to allow only FOR ALL TABLES; the proposal is to
relax this restriction.
I think allowing the use of an existing publication is a good idea. If we do not
want to modify the behavior of the current --publication option, introducing a
new option like --existing-publication might be prudent. With this change, it's
also necessary to implement checks to ensure column lists or row filters are not
used, as previously discussed. What do you think ?
I think we can explore extending the existing --clean option in a
separate patch to support table cleanup. This option is implemented in
a way that allows adding further cleanup objects later, so it should be easy toextend it for table.
Prior to this extension, it should be noted in the documentation that
users are required to clean up the tables themselves.I would say that these cleanup feature (starting with the cleanup databases) is
equally important as the feature that selects specific objects.I agree that supporting row filter and column list is not
straightforward, and we can consider it separately and do not implement thatin the first version.
The proposal above would allow it with no additional lines of code.
It seems this proposal doesn't serve a general purpose. It is copying
a *whole* cluster to use only a subset of tables. Your task with
pg_createsubscriber is more expensive than doing a manual logical
replication setup. If you have 500 tables and want to replicate only
400 tables, it doesn't seem productive to specify 400 -t options.Specifying multiple -t options should not be problematic, as users has
already done similar things for "FOR TABLE" publication DDLs. I think
it's not hard for user to convert FOR TABLE list to -t option list.Of course it is. Shell limits the number of arguments.
I initially thought that other commands had a similar limit, so did not worry about
that, but given that publication features may necessitate specifying a larger
number of tables, I agree that this limitation arises from using multiple -t
options.
There are some cases like a small set of big tables that this feature
makes sense. However, I'm wondering if a post script should be used
to adjust your setup.I think it's not very convenient for users to perform this conversion manually.
I've learned in PGConf.dev this year that some users avoid using
pg_createsubscriber because they are unsure of the standard steps
required to convert it into subset table replication. Automating this
process would be beneficial, enabling more users to use
pg_createsubscriber and take advantage of the rapid initial tablesynchronization.
You missed my point. I'm not talking about manually converting a physical
replica into a logical replica. I'm talking about the plain logical replication setup
(CREATE PUBLICATION, CREATE SUBSCRIPTION). IME this tool is beneficial
for large clusters that we want to replicate (almost) all tables.
I understand, but the initial synchronization in plain logical replication can
be slow in many cases, which has been a major complaint I received
recently. Using pg_createsubscriber can significantly improve performance in
those cases, even when subset of tables is published, particularly if the tables
are large or if the number of tables are huge. Of course, there are cases where
plain logical replication outperforms pg_createsubscriber. However, we could
also provide documentation with guidelines to assist users in choosing when to
use this new option in pg_createsubscriber.
Best Regards,
Hou zj
On Monday, August 25, 2025 8:08 AM Peter Smith <smithpb2250@gmail.com> wrote:
IIUC, the only purpose of the proposed '--table' spec is to allow some users the
ability to tweak the default "CREATE PUBLICATION p FOR ALL TABLES;" that
the 'pg_createsubscriber' tool would otherwise construct.But --table is introducing problems too: e.g.
- tricky option ordering rules (e.g. most recent --database if dbname not
specified?)
- too many --table may be needed; redundant repeating of databases and
schemas also seems very verbose.
- limitations for syntax (e.g. what about FOR TABLES IN SCHEMA?)
- limitations for future syntax (e.g. what about SEQUENCES?; what about
EXCEPT?)Can these problems disappear just by slightly changing the meaning of the
existing '--publication' option to allow specifying more than just the
publication name: e.g.
--publication = 'pub1' ==> "CREATE PUBLICATION pub1 FOR ALL TABLES;"
--publication = 'pub1 FOR TABLE t1,t2(c1,c2),t3' ==> "CREATE
PUBLICATION pub1 FOR TABLE t1,t2(c1,c2),t3;"
--publication = 'pub1 FOR TABLES IN SCHEMA s1' ==> "CREATE
PUBLICATION
pub1 FOR TABLES IN SCHEMA s1;"
--publication = 'pub1 FOR ALL TABLES WITH (publish_via_partition_root)'
==> "CREATE PUBLICATION pub1 FOR ALL TABLES WITH
(publish_via_partition_root);"
I think embedding commands directly in the option value poses SQL injection
risks, so is not very safe. We could directly allow users to use an existing
publication (as mentioned by Euler[1]/messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com).
[1]: /messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com
Best Regards,
Hou zj
On Mon, Aug 25, 2025 at 12:53 PM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
On Monday, August 25, 2025 8:08 AM Peter Smith <smithpb2250@gmail.com> wrote:
IIUC, the only purpose of the proposed '--table' spec is to allow some users the
ability to tweak the default "CREATE PUBLICATION p FOR ALL TABLES;" that
the 'pg_createsubscriber' tool would otherwise construct.But --table is introducing problems too: e.g.
- tricky option ordering rules (e.g. most recent --database if dbname not
specified?)
- too many --table may be needed; redundant repeating of databases and
schemas also seems very verbose.
- limitations for syntax (e.g. what about FOR TABLES IN SCHEMA?)
- limitations for future syntax (e.g. what about SEQUENCES?; what about
EXCEPT?)Can these problems disappear just by slightly changing the meaning of the
existing '--publication' option to allow specifying more than just the
publication name: e.g.
--publication = 'pub1' ==> "CREATE PUBLICATION pub1 FOR ALL TABLES;"
--publication = 'pub1 FOR TABLE t1,t2(c1,c2),t3' ==> "CREATE
PUBLICATION pub1 FOR TABLE t1,t2(c1,c2),t3;"
--publication = 'pub1 FOR TABLES IN SCHEMA s1' ==> "CREATE
PUBLICATION
pub1 FOR TABLES IN SCHEMA s1;"
--publication = 'pub1 FOR ALL TABLES WITH (publish_via_partition_root)'
==> "CREATE PUBLICATION pub1 FOR ALL TABLES WITH
(publish_via_partition_root);"I think embedding commands directly in the option value poses SQL injection
risks, so is not very safe. We could directly allow users to use an existing
publication (as mentioned by Euler[1]).[1] /messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com
I have incorporated the latest approach suggested by Hou-san at [1]/messages/by-id/TY4PR01MB169075A23F8D5EED4154A4AF3943EA@TY4PR01MB16907.jpnprd01.prod.outlook.com
and added a new --existing-publication option. This enhancement allows
users to specify existing publications instead of always creating new
ones. The attached patch includes the corresponding changes.
[1]: /messages/by-id/TY4PR01MB169075A23F8D5EED4154A4AF3943EA@TY4PR01MB16907.jpnprd01.prod.outlook.com
Thanks and regards,
Shubham Khanna.
Attachments:
v4-0001-Support-existing-publications-in-pg_createsubscri.patchapplication/octet-stream; name=v4-0001-Support-existing-publications-in-pg_createsubscri.patchDownload
From 4c5fe9ea67329b374407a6c61b8154a42462a623 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v4] Support existing publications in pg_createsubscriber
Add the --existing-publication option to pg_createsubscriber, allowing users to
specify existing publications instead of creating new ones.
This provides more flexibility when setting up logical replication,
particularly in scenarios where publications are managed separately or need to
be reused across multiple subscribers.
Key features:
1. New --existing-publication option accepts existing publication names.
2. Prevents mixing --publication and --existing-publication options.
3. Supports per-database specification like other options.
4. Includes validation that specified publications exist.
The number of existing publication names must match the number of database
names, following the same pattern as other pg_createsubscriber options.
This design aligns with the principle of other PostgreSQL tools that allow both
creation of new objects and reuse of existing ones.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 38 ++++++
src/bin/pg_basebackup/pg_createsubscriber.c | 117 ++++++++++++++++--
.../t/040_pg_createsubscriber.pl | 90 ++++++++++++++
3 files changed, 233 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..d1c92910a74 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -258,6 +258,44 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--existing-publication=<replaceable class="parameter">publication_name</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the name of an existing publication on the publisher to use
+ for logical replication. This option can be specified multiple times
+ to provide publication names for multiple databases when using multiple
+ <option>--database</option> options.
+ </para>
+ <para>
+ The specified publication must already exist on the publisher database.
+ When this option is used, <application>pg_createsubscriber</application>
+ will not create a new publication but will use the existing one.
+ During cleanup operations (including when using <option>--clean</option>),
+ existing publications specified with this option will not be dropped,
+ ensuring they remain available for other uses.
+ </para>
+ <para>
+ This option cannot be used together with <option>--publication</option>.
+ Use either <option>--publication</option> to create new publications
+ or <option>--existing-publication</option> to use existing ones, but
+ not both.
+ </para>
+ <para>
+ If neither <option>--publication</option> nor
+ <option>--existing-publication</option> is specified,
+ <application>pg_createsubscriber</application> will automatically
+ generate publication names and create new publications.
+ </para>
+ <para>
+ The number of existing publication names specified must equal the number
+ of database names when multiple databases are being configured. Each
+ existing publication will be used for its corresponding database in
+ the order specified.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--config-file=<replaceable class="parameter">filename</replaceable></option></term>
<listitem>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..3096b1f96f8 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -42,6 +42,8 @@ struct CreateSubscriberOptions
bool two_phase; /* enable-two-phase option */
SimpleStringList database_names; /* list of database names */
SimpleStringList pub_names; /* list of publication names */
+ SimpleStringList existing_pub_names; /* list of existing publication
+ * names */
SimpleStringList sub_names; /* list of subscription names */
SimpleStringList replslot_names; /* list of replication slot names */
int recovery_timeout; /* stop recovery after this time */
@@ -61,6 +63,8 @@ struct LogicalRepInfo
bool made_replslot; /* replication slot was created */
bool made_publication; /* publication was created */
+ bool is_existing_publication; /* true if --existing-publication
+ * was used */
};
/*
@@ -114,6 +118,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -161,7 +166,6 @@ enum WaitPMResult
POSTMASTER_STILL_STARTING
};
-
/*
* Cleanup objects that were created by pg_createsubscriber if there is an
* error.
@@ -202,7 +206,7 @@ cleanup_objects_atexit(void)
conn = connect_database(dbinfo->pubconninfo, false);
if (conn != NULL)
{
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->is_existing_publication)
drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
&dbinfo->made_publication);
if (dbinfo->made_replslot)
@@ -216,7 +220,7 @@ cleanup_objects_atexit(void)
* that some objects were left on primary and should be
* removed before trying again.
*/
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->is_existing_publication)
{
pg_log_warning("publication \"%s\" created in database \"%s\" on primary was left behind",
dbinfo->pubname,
@@ -263,6 +267,7 @@ usage(void)
printf(_(" --config-file=FILENAME use specified main server configuration\n"
" file when running target cluster\n"));
printf(_(" --publication=NAME publication name\n"));
+ printf(_(" --existing-publication=NAME use an existing publication name\n"));
printf(_(" --replication-slot=NAME replication slot name\n"));
printf(_(" --subscription=NAME subscription name\n"));
printf(_(" -V, --version output version information, then exit\n"));
@@ -466,14 +471,17 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
{
struct LogicalRepInfo *dbinfo;
SimpleStringListCell *pubcell = NULL;
+ SimpleStringListCell *existing_pub_cell = NULL;
SimpleStringListCell *subcell = NULL;
SimpleStringListCell *replslotcell = NULL;
int i = 0;
dbinfo = pg_malloc_array(struct LogicalRepInfo, num_dbs);
- if (num_pubs > 0)
+ if (opt->pub_names.head > 0)
pubcell = opt->pub_names.head;
+ if (opt->existing_pub_names.head != NULL)
+ existing_pub_cell = opt->existing_pub_names.head;
if (num_subs > 0)
subcell = opt->sub_names.head;
if (num_replslots > 0)
@@ -487,10 +495,21 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
conninfo = concat_conninfo_dbname(pub_base_conninfo, cell->val);
dbinfo[i].pubconninfo = conninfo;
dbinfo[i].dbname = cell->val;
- if (num_pubs > 0)
+ if (pubcell != NULL)
+ {
dbinfo[i].pubname = pubcell->val;
+ dbinfo[i].is_existing_publication = false;
+ }
+ else if (existing_pub_cell != NULL)
+ {
+ dbinfo[i].pubname = existing_pub_cell->val;
+ dbinfo[i].is_existing_publication = true;
+ }
else
+ {
dbinfo[i].pubname = NULL;
+ dbinfo[i].is_existing_publication = false;
+ }
if (num_replslots > 0)
dbinfo[i].replslotname = replslotcell->val;
else
@@ -506,8 +525,9 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
dbinfo[i].subname = NULL;
/* Other fields will be filled later */
- pg_log_debug("publisher(%d): publication: %s ; replication slot: %s ; connection string: %s", i,
+ pg_log_debug("publisher(%d): publication: %s (%s); replication slot: %s ; connection string: %s", i,
dbinfo[i].pubname ? dbinfo[i].pubname : "(auto)",
+ dbinfo[i].is_existing_publication ? "existing" : "new",
dbinfo[i].replslotname ? dbinfo[i].replslotname : "(auto)",
dbinfo[i].pubconninfo);
pg_log_debug("subscriber(%d): subscription: %s ; connection string: %s, two_phase: %s", i,
@@ -515,8 +535,10 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
dbinfo[i].subconninfo,
dbinfos.two_phase ? "true" : "false");
- if (num_pubs > 0)
+ if (pubcell != NULL)
pubcell = pubcell->next;
+ if (existing_pub_cell != NULL)
+ existing_pub_cell = existing_pub_cell->next;
if (num_subs > 0)
subcell = subcell->next;
if (num_replslots > 0)
@@ -753,6 +775,31 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Add function to check if publication exists.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists = false;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) == PGRES_TUPLES_OK && PQntuples(res) == 1)
+ exists = true;
+ else if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -780,22 +827,34 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
* no replication slot is specified. It follows the same rule as
* CREATE SUBSCRIPTION.
*/
- if (num_pubs == 0 || num_subs == 0 || num_replslots == 0)
+ if (dbinfo[i].pubname == NULL || dbinfo[i].subname == NULL || dbinfo[i].replslotname == NULL)
genname = generate_object_name(conn);
- if (num_pubs == 0)
+ if (dbinfo[i].pubname == NULL)
dbinfo[i].pubname = pg_strdup(genname);
if (num_subs == 0)
dbinfo[i].subname = pg_strdup(genname);
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
+ /* If using an existing publication, verify it exists. */
+ if (dbinfo[i].is_existing_publication)
+ {
+ if (!check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ pg_fatal("publication \"%s\" does not exist in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+
/*
* Create publication on publisher. This step should be executed
* *before* promoting the subscriber to avoid any transactions between
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (!dbinfo[i].is_existing_publication)
+ create_publication(conn, &dbinfo[i]);
+ else
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
/* Create replication slot on publisher */
if (lsn)
@@ -1770,9 +1829,12 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
* In dry-run mode, we don't create publications, but we still try to drop
* those to provide necessary information to the user.
*/
- if (!drop_all_pubs || dry_run)
+ else if (!dbinfo->is_existing_publication)
drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
&dbinfo->made_publication);
+ else
+ pg_log_info("not dropping existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
}
/*
@@ -2047,6 +2109,7 @@ main(int argc, char **argv)
{"replication-slot", required_argument, NULL, 3},
{"subscription", required_argument, NULL, 4},
{"clean", required_argument, NULL, 5},
+ {"existing-publication", required_argument, NULL, 6},
{NULL, 0, NULL, 0}
};
@@ -2058,6 +2121,7 @@ main(int argc, char **argv)
char *pub_base_conninfo;
char *sub_base_conninfo;
char *dbname_conninfo = NULL;
+ int total_pubs_configured = 0;
uint64 pub_sysid;
uint64 sub_sysid;
@@ -2200,6 +2264,15 @@ main(int argc, char **argv)
else
pg_fatal("object type \"%s\" specified more than once for --clean", optarg);
break;
+ case 6:
+ if (!simple_string_list_member(&opt.existing_pub_names, optarg))
+ {
+ simple_string_list_append(&opt.existing_pub_names, optarg);
+ num_pubs++;
+ }
+ else
+ pg_fatal("existing publication \"%s\" specified more than once for --existing-publication", optarg);
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -2214,12 +2287,14 @@ main(int argc, char **argv)
if (num_dbs > 0)
bad_switch = "--database";
- else if (num_pubs > 0)
+ else if (opt.pub_names.head > 0)
bad_switch = "--publication";
else if (num_replslots > 0)
bad_switch = "--replication-slot";
else if (num_subs > 0)
bad_switch = "--subscription";
+ else if (opt.existing_pub_names.head != NULL)
+ bad_switch = "--existing-publication";
if (bad_switch)
{
@@ -2319,6 +2394,13 @@ main(int argc, char **argv)
}
}
+ if (opt.pub_names.head > 0 && opt.existing_pub_names.head != NULL)
+ {
+ pg_log_error("options --publication and --existing-publication cannot be used together");
+ pg_log_error_hint("Specify either new publications to create or existing publications to use, but not both.");
+ exit(1);
+ }
+
/* Number of object names must match number of databases */
if (num_pubs > 0 && num_pubs != num_dbs)
{
@@ -2327,6 +2409,17 @@ main(int argc, char **argv)
num_pubs, num_dbs);
exit(1);
}
+
+ for (SimpleStringListCell *cell = opt.existing_pub_names.head; cell != NULL; cell = cell->next)
+ total_pubs_configured++;
+
+ if (total_pubs_configured > 0 && total_pubs_configured != num_dbs)
+ {
+ pg_log_error("wrong number of existing publication names specified");
+ pg_log_error_detail("The number of specified existing publication names (%d) must match the number of specified database names (%d).",
+ total_pubs_configured, num_dbs);
+ exit(1);
+ }
if (num_subs > 0 && num_subs != num_dbs)
{
pg_log_error("wrong number of subscription names specified");
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..c49a5ec2e88 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,99 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1, "CREATE PUBLICATION test_pub3 FOR TABLE tbl1");
+$node_p->safe_psql($db2, "CREATE PUBLICATION test_pub4 FOR TABLE tbl2");
+
+# Initialize node_s2 as a fresh standby of node_p for table-level
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber with invalid --existing-publication option
+# (conflict with --publication)
+command_fails_like(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--publication' => 'pub1',
+ '--database' => $db1,
+ '--existing-publication' => 'test_pub3',
+ ],
+ qr/options --publication and --existing-publication cannot be used together/,
+ 'pg_createsubscriber rejects --publication with --existing-publication');
+
+# Run pg_createsubscriber on node S with --existing-publication option.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--existing-publication' => 'test_pub3',
+ '--existing-publication' => 'test_pub4',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub3}
+{test_pub4}),
+ "subscriptions use the correct publications in $db1 and $db2");
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
My first impression of patch v4-0001 was that the new design is adding
unnecessary complexity by introducing a new --existing-publication
option, instead of just allowing reinterpretation of the --publication
by allowing it to name/reuse an existing publication.
e.g.
Q. What if there are multiple databases, but there's a publication on
only one of them? How can the user specify this, given that the rules
say there must be the same number of --existing-publication as
--database? The rules also say --existing-publication and
--publication cannot coexist. So what is this user supposed to do in
this scenario?
Q. What if there are multiple databases with publications, but the
user wants to use the existing publication for only one of those and
FOR ALL TABLES for the other one? The rules say there must be the same
number of --existing-publication as --database, so how can the user
achieve what they want?
~~
AFAICT, these problems don't happen if you allow a reinterpretation of
--publication switch like Euler had suggested [1]Euler - /messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com. The dry-run logs
should make it clear whether an existing publication will be reused or
not, so I'm not sure even that the validation code (checking if the
publication exists) is needed.
Finally, I think the implementation might be simpler too -- e.g. fewer
rules/docs needed, no option conflict code needed.
Anyway, maybe you disagree. But, even if you still decide an
--existing-publication option is needed, IMO the rules should be
relaxed to permit this to co-exist with the --publication option, so
that the scenarios described above can have solutions.
~
Below are some more review comments for patch v4-0001, but since I had
doubts about the basic design, I did not review it in much detail.
======
Commit message
1.
The number of existing publication names must match the number of database
names, following the same pattern as other pg_createsubscriber options.
~
This assumes that if one database has existing publications, then they
all will. That does not seem like a valid assumption.
~~~
2.
This design aligns with the principle of other PostgreSQL tools that
allow both creation of new objects and reuse of existing ones.
~
What tools? Can you provide examples? I'd like to check the design
alignment for myself.
======
doc/src/sgml/ref/pg_createsubscriber.sgml
3.
+ <para>
+ This option cannot be used together with <option>--publication</option>.
+ Use either <option>--publication</option> to create new publications
+ or <option>--existing-publication</option> to use existing ones, but
+ not both.
+ </para>
Why not? How else can a user say to use an existing publication for
one database when another database does not have any publications?
~~~
4.
+ <para>
+ The number of existing publication names specified must equal the number
+ of database names when multiple databases are being configured. Each
+ existing publication will be used for its corresponding database in
+ the order specified.
+ </para>
Ditto above. What about when the second database does not even have an
existing publication?
======
src/bin/pg_basebackup/pg_createsubscriber.c
5.
-
/*
* Cleanup objects that were created by pg_createsubscriber if there is an
Does this whitespace change belong be in this patch?
~~~
6a.
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->is_existing_publication)
6b.
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->is_existing_publication)
~
Why do you need to check both flags in these places? IIUC, it's the
publication can be "made" or "existing"; it cannot be both. Why not
just check "made"?
~~~
7.
+ if (PQresultStatus(res) == PGRES_TUPLES_OK && PQntuples(res) == 1)
+ exists = true;
+ else if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
This logic should not need to check PQresultStatus(res) multiple times.
======
.../t/040_pg_createsubscriber.pl
8.
+# Run pg_createsubscriber with invalid --existing-publication option
+# (conflict with --publication)
+command_fails_like(
Why not allow these to coexist?
======
[1]: Euler - /messages/by-id/30cc34eb-07a0-4b55-b4fe-6c526886b2c4@app.fastmail.com
Kind Regards,
Peter Smith
Fujitsu Australia
On Wednesday, September 3, 2025 9:58 AM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
My first impression of patch v4-0001 was that the new design is adding
unnecessary complexity by introducing a new --existing-publication
option, instead of just allowing reinterpretation of the --publication
by allowing it to name/reuse an existing publication.e.g.
Q. What if there are multiple databases, but there's a publication on
only one of them? How can the user specify this, given that the rules
say there must be the same number of --existing-publication as
--database? The rules also say --existing-publication and
--publication cannot coexist. So what is this user supposed to do in
this scenario?Q. What if there are multiple databases with publications, but the
user wants to use the existing publication for only one of those and
FOR ALL TABLES for the other one? The rules say there must be the same
number of --existing-publication as --database, so how can the user
achieve what they want?~~
AFAICT, these problems don't happen if you allow a reinterpretation of
--publication switch like Euler had suggested [1]. The dry-run logs
should make it clear whether an existing publication will be reused or
not, so I'm not sure even that the validation code (checking if the
publication exists) is needed.Finally, I think the implementation might be simpler too -- e.g. fewer
rules/docs needed, no option conflict code needed.
I'm hesitant to directly change the behavior of --publication, as it seems
unexpected for the command in the new version to silently use an existing
publication when it previously reported an ERROR. Although the number of users
relying on the ERROR may be small, the change still appears risky to me.
I think it's reasonable to introduce an option that allows users to preserve the
original behavior. If --existing-publication appears too complex, we might
consider adding a more straightforward option, such as --if-not-exists or
--use-existing-pub (name subject to further discussion). When this option is
specified, the command will not report an ERROR if a publication with the
specified --publication name already exists; instead, it will use the existing
publication to set up the publisher. If the option is not specified, the original
behavior will be maintained.
Best Regards,
Hou zj
On Wed, Sep 10, 2025 at 8:05 AM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
On Wednesday, September 3, 2025 9:58 AM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
My first impression of patch v4-0001 was that the new design is adding
unnecessary complexity by introducing a new --existing-publication
option, instead of just allowing reinterpretation of the --publication
by allowing it to name/reuse an existing publication.e.g.
Q. What if there are multiple databases, but there's a publication on
only one of them? How can the user specify this, given that the rules
say there must be the same number of --existing-publication as
--database? The rules also say --existing-publication and
--publication cannot coexist. So what is this user supposed to do in
this scenario?Q. What if there are multiple databases with publications, but the
user wants to use the existing publication for only one of those and
FOR ALL TABLES for the other one? The rules say there must be the same
number of --existing-publication as --database, so how can the user
achieve what they want?~~
AFAICT, these problems don't happen if you allow a reinterpretation of
--publication switch like Euler had suggested [1]. The dry-run logs
should make it clear whether an existing publication will be reused or
not, so I'm not sure even that the validation code (checking if the
publication exists) is needed.Finally, I think the implementation might be simpler too -- e.g. fewer
rules/docs needed, no option conflict code needed.I'm hesitant to directly change the behavior of --publication, as it seems
unexpected for the command in the new version to silently use an existing
publication when it previously reported an ERROR. Although the number of users
relying on the ERROR may be small, the change still appears risky to me.I think it's reasonable to introduce an option that allows users to preserve the
original behavior. If --existing-publication appears too complex, we might
consider adding a more straightforward option, such as --if-not-exists or
--use-existing-pub (name subject to further discussion). When this option is
specified, the command will not report an ERROR if a publication with the
specified --publication name already exists; instead, it will use the existing
publication to set up the publisher. If the option is not specified, the original
behavior will be maintained.
As suggested I introduced a new option --if-not-exists. When this
option is specified, if the given --publication already exists, it
will be reused instead of raising an ERROR. If the option is not
specified, the current behavior is preserved, ensuring backward
compatibility.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
Attachments:
v5-0001-Support-existing-publications-in-pg_createsubscri.patchapplication/octet-stream; name=v5-0001-Support-existing-publications-in-pg_createsubscri.patchDownload
From fabb1b5d1c35fc86cdfab8dcd7da43e2f572993b Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v5] Support existing publications in pg_createsubscriber
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add a new '--if-not-exists' option to pg_createsubscriber, allowing users to
reuse existing publications if they are already present, or create them if they
do not exist.
This simplifies publication handling while providing more flexibility when
setting up logical replication.
Key features:
1. New '--if-not-exists' flag changes the behavior of '--publication'.
2. If the publication exists, it is reused.
3. If it does not exist, it is created automatically.
4. Supports per-database specification, consistent with other options.
5. Avoids the complexity of option conflicts and count-matching rules.
6. Provides semantics consistent with SQL’s IF NOT EXISTS syntax.
This design streamlines the CLI, improves usability, and supports scenarios
where some publications are reused while others are created during subscriber
setup.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 31 ++++++
src/bin/pg_basebackup/pg_createsubscriber.c | 98 ++++++++++++++++---
.../t/040_pg_createsubscriber.pl | 80 +++++++++++++++
3 files changed, 197 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..e7656d764d3 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -258,6 +258,37 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--if-not-exists</option></term>
+ <listitem>
+ <para>
+ When this option is specified,
+ <application>pg_createsubscriber</application> will check if each
+ publication specified with <option>--publication</option>
+ already exists on the publisher. If a publication exists, it will be
+ reused. If a publication does not exist, it will be created
+ automatically with <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ This option provides flexibility for mixed scenarios where some
+ publications may already exist while others need to be created.
+ It eliminates the need to know in advance which publications exist on
+ the publisher.
+ </para>
+ <para>
+ When <option>--if-not-exists</option> is used, existing publications will
+ not be dropped during cleanup operations, ensuring they remain available
+ for other uses. Only publications that were created by
+ <application>pg_createsubscriber</application> will be cleaned up.
+ </para>
+ <para>
+ This option follows the same semantics as SQL
+ <literal>IF NOT EXISTS</literal> clauses, providing consistent behavior
+ with other PostgreSQL tools.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--config-file=<replaceable class="parameter">filename</replaceable></option></term>
<listitem>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..fc0972eca9d 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -42,6 +42,7 @@ struct CreateSubscriberOptions
bool two_phase; /* enable-two-phase option */
SimpleStringList database_names; /* list of database names */
SimpleStringList pub_names; /* list of publication names */
+ bool if_not_exists; /* --if-not-exists option */
SimpleStringList sub_names; /* list of subscription names */
SimpleStringList replslot_names; /* list of replication slot names */
int recovery_timeout; /* stop recovery after this time */
@@ -61,6 +62,8 @@ struct LogicalRepInfo
bool made_replslot; /* replication slot was created */
bool made_publication; /* publication was created */
+ bool publication_existed; /* publication existed before we
+ * started */
};
/*
@@ -93,7 +96,7 @@ static void modify_subscriber_sysid(const struct CreateSubscriberOptions *opt);
static bool server_is_in_recovery(PGconn *conn);
static char *generate_object_name(PGconn *conn);
static void check_publisher(const struct LogicalRepInfo *dbinfo);
-static char *setup_publisher(struct LogicalRepInfo *dbinfo);
+static char *setup_publisher(struct LogicalRepInfo *dbinfo, const struct CreateSubscriberOptions *opt);
static void check_subscriber(const struct LogicalRepInfo *dbinfo);
static void setup_subscriber(struct LogicalRepInfo *dbinfo,
const char *consistent_lsn);
@@ -114,6 +117,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -202,7 +206,7 @@ cleanup_objects_atexit(void)
conn = connect_database(dbinfo->pubconninfo, false);
if (conn != NULL)
{
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->publication_existed)
drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
&dbinfo->made_publication);
if (dbinfo->made_replslot)
@@ -216,7 +220,7 @@ cleanup_objects_atexit(void)
* that some objects were left on primary and should be
* removed before trying again.
*/
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->publication_existed)
{
pg_log_warning("publication \"%s\" created in database \"%s\" on primary was left behind",
dbinfo->pubname,
@@ -263,6 +267,7 @@ usage(void)
printf(_(" --config-file=FILENAME use specified main server configuration\n"
" file when running target cluster\n"));
printf(_(" --publication=NAME publication name\n"));
+ printf(_(" --if-not-exists reuse existing publications if they exist\n"));
printf(_(" --replication-slot=NAME replication slot name\n"));
printf(_(" --subscription=NAME subscription name\n"));
printf(_(" -V, --version output version information, then exit\n"));
@@ -497,6 +502,7 @@ store_pub_sub_info(const struct CreateSubscriberOptions *opt,
dbinfo[i].replslotname = NULL;
dbinfo[i].made_replslot = false;
dbinfo[i].made_publication = false;
+ dbinfo[i].publication_existed = false;
/* Fill subscriber attributes */
conninfo = concat_conninfo_dbname(sub_base_conninfo, cell->val);
dbinfo[i].subconninfo = conninfo;
@@ -753,6 +759,31 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Add function to check if publication exists.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists = false;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -760,7 +791,7 @@ generate_object_name(PGconn *conn)
* set_replication_progress).
*/
static char *
-setup_publisher(struct LogicalRepInfo *dbinfo)
+setup_publisher(struct LogicalRepInfo *dbinfo, const struct CreateSubscriberOptions *opt)
{
char *lsn = NULL;
@@ -780,22 +811,46 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
* no replication slot is specified. It follows the same rule as
* CREATE SUBSCRIPTION.
*/
- if (num_pubs == 0 || num_subs == 0 || num_replslots == 0)
+ if (dbinfo[i].pubname == NULL || dbinfo[i].subname == NULL || dbinfo[i].replslotname == NULL)
genname = generate_object_name(conn);
- if (num_pubs == 0)
+ if (dbinfo[i].pubname == NULL)
dbinfo[i].pubname = pg_strdup(genname);
- if (num_subs == 0)
+ if (dbinfo[i].subname == NULL)
dbinfo[i].subname = pg_strdup(genname);
- if (num_replslots == 0)
+ if (dbinfo[i].replslotname == NULL)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
+ /*
+ * Check if publication already exists when --if-not-exists is
+ * specified
+ */
+ if (opt->if_not_exists)
+ {
+ dbinfo[i].publication_existed = check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname);
+ if (dbinfo[i].publication_existed)
+ {
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+ }
+
/*
* Create publication on publisher. This step should be executed
* *before* promoting the subscriber to avoid any transactions between
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (opt->if_not_exists && dbinfo[i].publication_existed)
+ dbinfo[i].made_publication = false;
+ else
+ {
+ create_publication(conn, &dbinfo[i]);
+ if (opt->if_not_exists)
+ {
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1771,8 +1826,14 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
* those to provide necessary information to the user.
*/
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ if (!dbinfo->publication_existed)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ }
+ else
+ pg_log_info("not dropping existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
}
/*
@@ -2047,6 +2108,7 @@ main(int argc, char **argv)
{"replication-slot", required_argument, NULL, 3},
{"subscription", required_argument, NULL, 4},
{"clean", required_argument, NULL, 5},
+ {"if-not-exists", no_argument, NULL, 6},
{NULL, 0, NULL, 0}
};
@@ -2095,6 +2157,7 @@ main(int argc, char **argv)
opt.sub_port = DEFAULT_SUB_PORT;
opt.sub_username = NULL;
opt.two_phase = false;
+ opt.if_not_exists = false;
opt.database_names = (SimpleStringList)
{
0
@@ -2200,6 +2263,9 @@ main(int argc, char **argv)
else
pg_fatal("object type \"%s\" specified more than once for --clean", optarg);
break;
+ case 6:
+ opt.if_not_exists = true;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -2220,6 +2286,8 @@ main(int argc, char **argv)
bad_switch = "--replication-slot";
else if (num_subs > 0)
bad_switch = "--subscription";
+ else if (opt.if_not_exists)
+ bad_switch = "--if-not-exists";
if (bad_switch)
{
@@ -2319,6 +2387,12 @@ main(int argc, char **argv)
}
}
+ if (opt.if_not_exists && num_pubs == 0)
+ {
+ pg_log_error("--if-not-exists requires --publication to be specified");
+ exit(1);
+ }
+
/* Number of object names must match number of databases */
if (num_pubs > 0 && num_pubs != num_dbs)
{
@@ -2424,7 +2498,7 @@ main(int argc, char **argv)
stop_standby_server(subscriber_dir);
/* Create the required objects for each database on publisher */
- consistent_lsn = setup_publisher(dbinfos.dbinfo);
+ consistent_lsn = setup_publisher(dbinfos.dbinfo, &opt);
/* Write the required recovery parameters */
setup_recovery(dbinfos.dbinfo, subscriber_dir, consistent_lsn);
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..95e7a056a21 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,89 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 as a fresh standby of node_p for table-level
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber on node S2 with --if-not-exists option for mixed
+# scenario (one existing publication, one new publication)
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--if-not-exists',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications with --if-not-exists in $db1 and $db2"
+);
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
My first impression of this latest patch is that the new option name
is not particularly intuitive.
Below are some more review comments for patch v5-0001; any changes to
the name will propagate all through this with different member names
and usage, etc, so I did not repeat that same comment over and over.
======
Commit message
1.
Add a new '--if-not-exists' option to pg_createsubscriber, allowing users to
reuse existing publications if they are already present, or create them if they
do not exist.
~
IMO, this option name ("if-not-exists") gives the user no clues as to
its purpose.
I suppose you wanted the user to associate this option with a CREATE
PUBLICATION, so that they can deduce that it acts internally like a
"CREATE PUBLICATION pub ... IF NOT EXISTS" (if there was such a
thing), I don't see how a user is able to find any meaning at all from
this vague name. The only way they can know what this means is to read
all the documentation.
e.g.
- if WHAT doesn't exist?
- if <??> not exists, then do WHAT?
Alternatives like "--reuse-pub-if-exists" or "--reuse-existing-pubs"
might be easier to understand.
~~~
2.
Key features:
1. New '--if-not-exists' flag changes the behavior of '--publication'.
2. If the publication exists, it is reused.
3. If it does not exist, it is created automatically.
4. Supports per-database specification, consistent with other options.
5. Avoids the complexity of option conflicts and count-matching rules.
6. Provides semantics consistent with SQL’s IF NOT EXISTS syntax.
~
Some of those "features" seem wrong, and some seem inappropriate for
the commit message:
e.g. TBH, I doubt that "4. Supports per-database specification" can
work as claimed here. Supposing I have multiple databases, then I
cannot see how I can say "if-not-exists" for some databases but not
for others.
e.g. The "5. Avoids the complexity..." seems like a reason why an
alternative design was rejected; Why does this comment belong here?
e.g. The "6. Provides semantics..." seems like your internal
implementation justification, whereas IMO this patch should be more
focused on making any new command-line option more
intuitive/meaningful from the point of view of the user.
======
doc/src/sgml/ref/pg_createsubscriber.sgml
3.
+ <para>
+ This option provides flexibility for mixed scenarios where some
+ publications may already exist while others need to be created.
+ It eliminates the need to know in advance which publications exist on
+ the publisher.
+ </para>
Hmm. Is that statement ("It eliminates the need to know...") correct?
I feel it should say the total opposite of this.
For example, IMO now the user needs to be extra careful because if
they say --publication=pub1 --if-not-exists then they have to be 100%
sure if 'pub1' exists or does not exist, otherwise they might
accidentally be making pub1 FOR ALL TABLES when really they expected
they were reusing some existing publication 'pub1', or vice versa.
In fact, I think the docs should go further and *recommend* that the
user never uses this new option without firstly doing a --dry-run to
verify they are actually getting the publications that they assume
they are getting.
~~~
4.
+ <para>
+ This option follows the same semantics as SQL
+ <literal>IF NOT EXISTS</literal> clauses, providing consistent behavior
+ with other PostgreSQL tools.
+ </para>
Why even say this in the docs? What other "tools" are you referring to?
======
src/bin/pg_basebackup/pg_createsubscriber.c
5.
bool made_publication; /* publication was created */
+ bool publication_existed; /* publication existed before we
+ * started */
};
This new member feels redundant to me.
AFAIK, the publication will either exist already, OR it will be
created/made by pg_createsubscriber. You don't need 2 booleans to
represent that.
At most, maybe all you want is another local variable in the function
setup_publisher().
~~~
cleanup_objects_atexit:
(same comments as posted in my v4 review)
6a.
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->publication_existed)
Same as above review comment #5. It's either made or it's not made,
right? Why the extra boolean?
~
6b.
- if (dbinfo->made_publication)
+ if (dbinfo->made_publication && !dbinfo->publication_existed)
Ditto.
~~~
check_publication_exists:
7.
+/*
+ * Add function to check if publication exists.
+ */
Function comment should not say "Add function".
~
8.
+ PGresult *res;
+ bool exists = false;
+ char *query;
Redundant assignment?
~~~
setup_publisher:
9.
+ /*
+ * Check if publication already exists when --if-not-exists is
+ * specified
+ */
+ if (opt->if_not_exists)
+ {
+ dbinfo[i].publication_existed = check_publication_exists(conn,
dbinfo[i].pubname, dbinfo[i].dbname);
+ if (dbinfo[i].publication_existed)
+ {
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+ }
+
/*
* Create publication on publisher. This step should be executed
* *before* promoting the subscriber to avoid any transactions between
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (opt->if_not_exists && dbinfo[i].publication_existed)
+ dbinfo[i].made_publication = false;
+ else
+ {
+ create_publication(conn, &dbinfo[i]);
+ if (opt->if_not_exists)
+ {
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+ }
This all seems way more complicated than it needs to be. e.g. I doubt
that you need to be checking opt->if_not_exists 3 times.
Simpler logic might be more like below:
bool make_pub = true;
if (opt->if_not_exists && check_publication_exists(...))
{
pg_log_info("using existing...");
make_pub = false;
}
...
if (make_pub)
{
create_publication(conn, &dbinfo[i]);
pg_log_info("created publication...");
}
dbinfo[i].made_publication = make_pub;
======
.../t/040_pg_createsubscriber.pl
10.
+# Initialize node_s2 as a fresh standby of node_p for table-level
+# publication test.
I don't know that you should still be calling this a "table-level"
test. Maybe it's more like an "existing publication" test now?
~~~
11.
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications with --if-not-exists in
$db1 and $db2"
+);
Something seems broken. I deliberately caused an error to occur to see
what would happen, and the substitutions of $db1 and $db2 went crazy.
# Failed test 'subscriptions use the correct publications with
--if-not-exists in regression\"\
¬ !"\#$%&'()*+,-\\"\\\
and regression./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ'
# at t/040_pg_createsubscriber.pl line 614.
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Thu, Sep 11, 2025 at 8:02 AM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Peter,
Thank you for the review. I have addressed your comments in the latest patch.
My first impression of this latest patch is that the new option name
is not particularly intuitive.Below are some more review comments for patch v5-0001; any changes to
the name will propagate all through this with different member names
and usage, etc, so I did not repeat that same comment over and over.======
Commit message1.
Add a new '--if-not-exists' option to pg_createsubscriber, allowing users to
reuse existing publications if they are already present, or create them if they
do not exist.~
IMO, this option name ("if-not-exists") gives the user no clues as to
its purpose.I suppose you wanted the user to associate this option with a CREATE
PUBLICATION, so that they can deduce that it acts internally like a
"CREATE PUBLICATION pub ... IF NOT EXISTS" (if there was such a
thing), I don't see how a user is able to find any meaning at all from
this vague name. The only way they can know what this means is to read
all the documentation.e.g.
- if WHAT doesn't exist?
- if <??> not exists, then do WHAT?Alternatives like "--reuse-pub-if-exists" or "--reuse-existing-pubs"
might be easier to understand.
I agree that --if-not-exists may not be very intuitive, as it doesn’t
clearly convey the intended behavior without reading the
documentation.
For this version, I have renamed the option to
"--reuse-existing-publications", which I believe makes the purpose
clearer: if the specified publication already exists, it will be
reused; otherwise, a new one will be created. This avoids ambiguity
while still keeping the semantics aligned with the original intent.
~~~
2.
Key features:
1. New '--if-not-exists' flag changes the behavior of '--publication'.
2. If the publication exists, it is reused.
3. If it does not exist, it is created automatically.
4. Supports per-database specification, consistent with other options.
5. Avoids the complexity of option conflicts and count-matching rules.
6. Provides semantics consistent with SQL’s IF NOT EXISTS syntax.~
Some of those "features" seem wrong, and some seem inappropriate for
the commit message:e.g. TBH, I doubt that "4. Supports per-database specification" can
work as claimed here. Supposing I have multiple databases, then I
cannot see how I can say "if-not-exists" for some databases but not
for others.e.g. The "5. Avoids the complexity..." seems like a reason why an
alternative design was rejected; Why does this comment belong here?e.g. The "6. Provides semantics..." seems like your internal
implementation justification, whereas IMO this patch should be more
focused on making any new command-line option more
intuitive/meaningful from the point of view of the user.
Updated the commit message.
======
doc/src/sgml/ref/pg_createsubscriber.sgml3. + <para> + This option provides flexibility for mixed scenarios where some + publications may already exist while others need to be created. + It eliminates the need to know in advance which publications exist on + the publisher. + </para>Hmm. Is that statement ("It eliminates the need to know...") correct?
I feel it should say the total opposite of this.For example, IMO now the user needs to be extra careful because if
they say --publication=pub1 --if-not-exists then they have to be 100%
sure if 'pub1' exists or does not exist, otherwise they might
accidentally be making pub1 FOR ALL TABLES when really they expected
they were reusing some existing publication 'pub1', or vice versa.In fact, I think the docs should go further and *recommend* that the
user never uses this new option without firstly doing a --dry-run to
verify they are actually getting the publications that they assume
they are getting.
Fixed.
~~~
4. + <para> + This option follows the same semantics as SQL + <literal>IF NOT EXISTS</literal> clauses, providing consistent behavior + with other PostgreSQL tools. + </para>Why even say this in the docs? What other "tools" are you referring to?
Fixed.
======
src/bin/pg_basebackup/pg_createsubscriber.c5. bool made_publication; /* publication was created */ + bool publication_existed; /* publication existed before we + * started */ };This new member feels redundant to me.
AFAIK, the publication will either exist already, OR it will be
created/made by pg_createsubscriber. You don't need 2 booleans to
represent that.
At most, maybe all you want is another local variable in the function
setup_publisher().
Fixed.
~~~
cleanup_objects_atexit:
(same comments as posted in my v4 review)
Fixed.
6a. - if (dbinfo->made_publication) + if (dbinfo->made_publication && !dbinfo->publication_existed)Same as above review comment #5. It's either made or it's not made,
right? Why the extra boolean?~
6b. - if (dbinfo->made_publication) + if (dbinfo->made_publication && !dbinfo->publication_existed)Ditto.
Fixed.
~~~
check_publication_exists:
7. +/* + * Add function to check if publication exists. + */Function comment should not say "Add function".
Fixed.
~
8. + PGresult *res; + bool exists = false; + char *query;Redundant assignment?
Fixed.
~~~
setup_publisher:
9. + /* + * Check if publication already exists when --if-not-exists is + * specified + */ + if (opt->if_not_exists) + { + dbinfo[i].publication_existed = check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname); + if (dbinfo[i].publication_existed) + { + pg_log_info("using existing publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + } + } + /* * Create publication on publisher. This step should be executed * *before* promoting the subscriber to avoid any transactions between * consistent LSN and the new publication rows (such transactions * wouldn't see the new publication rows resulting in an error). */ - create_publication(conn, &dbinfo[i]); + if (opt->if_not_exists && dbinfo[i].publication_existed) + dbinfo[i].made_publication = false; + else + { + create_publication(conn, &dbinfo[i]); + if (opt->if_not_exists) + { + pg_log_info("created publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + } + }This all seems way more complicated than it needs to be. e.g. I doubt
that you need to be checking opt->if_not_exists 3 times.Simpler logic might be more like below:
bool make_pub = true;
if (opt->if_not_exists && check_publication_exists(...))
{
pg_log_info("using existing...");
make_pub = false;
}...
if (make_pub)
{
create_publication(conn, &dbinfo[i]);
pg_log_info("created publication...");
}dbinfo[i].made_publication = make_pub;
Fixed.
======
.../t/040_pg_createsubscriber.pl10. +# Initialize node_s2 as a fresh standby of node_p for table-level +# publication test.I don't know that you should still be calling this a "table-level"
test. Maybe it's more like an "existing publication" test now?
Fixed.
~~~
11. +is( $result, qq({test_pub_existing} +{test_pub_new}), + "subscriptions use the correct publications with --if-not-exists in $db1 and $db2" +);Something seems broken. I deliberately caused an error to occur to see
what would happen, and the substitutions of $db1 and $db2 went crazy.# Failed test 'subscriptions use the correct publications with
--if-not-exists in regression\"\¬ !"\#$%&'()*+,-\\"\\\
and regression./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ'
# at t/040_pg_createsubscriber.pl line 614.
Fixed.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna,
Attachments:
v6-0001-Support-existing-publications-in-pg_createsubscri.patchapplication/octet-stream; name=v6-0001-Support-existing-publications-in-pg_createsubscri.patchDownload
From 8525a8f0fd3ed6a52a7fe6ad62567aeb8c8028a7 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v6] Support existing publications in pg_createsubscriber
Add a new '--reuse-existing-publications' option to pg_createsubscriber,
allowing users to reuse existing publications instead of failing when they
already exist on the publisher.
Without this option, pg_createsubscriber fails if any specified publication
already exists. With this option, existing publications are reused and
non-existing publications are created automatically with FOR ALL TABLES.
This is useful in environments where publications may have been created by
previous operations or other processes, eliminating the need to know in advance
which publications exist on the publisher.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 37 ++++++++
src/bin/pg_basebackup/pg_createsubscriber.c | 90 ++++++++++++++++---
.../t/040_pg_createsubscriber.pl | 80 +++++++++++++++++
3 files changed, 197 insertions(+), 10 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..18716d66a61 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -258,6 +258,43 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--reuse-existing-publications</option></term>
+ <listitem>
+ <para>
+ When this option is specified,
+ <application>pg_createsubscriber</application> will check if each
+ publication specified with <option>--publication</option>already exists
+ on the publisher. If a publication exists, it will be reused without
+ modification. If a publication does not exist, it will be created
+ automatically with <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ This option requires you to understand exactly which publications exist
+ and how they are configured. If you reuse an existing publication,
+ it will be used as-is with its current table list, filters, and settings.
+ If you specify a publication that doesn't exist, it will be created with
+ <literal>FOR ALL TABLES</literal>, which may replicate more tables than
+ intended.
+ </para>
+ <para>
+ It is strongly recommended to use <option>--dry-run</option> first to
+ verify exactly which publications will be reused and which will be
+ created. The dry-run output will show you whether each specified
+ publication exists on the publisher and what action will be taken.
+ This verification step can prevent unexpected replication behavior and
+ data transfer.
+ </para>
+ <para>
+ When <option>--reuse-existing-publications</option> is used,
+ existing publications will not be dropped during cleanup operations,
+ ensuring they remain available for other uses. Only publications that
+ were created by <application>pg_createsubscriber</application>
+ will be cleaned up.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--config-file=<replaceable class="parameter">filename</replaceable></option></term>
<listitem>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..07aeac43641 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -42,6 +42,8 @@ struct CreateSubscriberOptions
bool two_phase; /* enable-two-phase option */
SimpleStringList database_names; /* list of database names */
SimpleStringList pub_names; /* list of publication names */
+ bool reuse_existing_pubs; /* --reuse-existing-publications
+ * option */
SimpleStringList sub_names; /* list of subscription names */
SimpleStringList replslot_names; /* list of replication slot names */
int recovery_timeout; /* stop recovery after this time */
@@ -93,7 +95,7 @@ static void modify_subscriber_sysid(const struct CreateSubscriberOptions *opt);
static bool server_is_in_recovery(PGconn *conn);
static char *generate_object_name(PGconn *conn);
static void check_publisher(const struct LogicalRepInfo *dbinfo);
-static char *setup_publisher(struct LogicalRepInfo *dbinfo);
+static char *setup_publisher(struct LogicalRepInfo *dbinfo, const struct CreateSubscriberOptions *opt);
static void check_subscriber(const struct LogicalRepInfo *dbinfo);
static void setup_subscriber(struct LogicalRepInfo *dbinfo,
const char *consistent_lsn);
@@ -114,6 +116,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -263,6 +266,7 @@ usage(void)
printf(_(" --config-file=FILENAME use specified main server configuration\n"
" file when running target cluster\n"));
printf(_(" --publication=NAME publication name\n"));
+ printf(_(" --reuse-existing-publications reuse existing publications if they exist\n"));
printf(_(" --replication-slot=NAME replication slot name\n"));
printf(_(" --subscription=NAME subscription name\n"));
printf(_(" -V, --version output version information, then exit\n"));
@@ -753,6 +757,32 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Check if a publication with the given name exists in the specified database.
+ * Returns true if it exists, false otherwise.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -760,7 +790,7 @@ generate_object_name(PGconn *conn)
* set_replication_progress).
*/
static char *
-setup_publisher(struct LogicalRepInfo *dbinfo)
+setup_publisher(struct LogicalRepInfo *dbinfo, const struct CreateSubscriberOptions *opt)
{
char *lsn = NULL;
@@ -770,6 +800,7 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
{
PGconn *conn;
char *genname = NULL;
+ bool make_pub = true;
conn = connect_database(dbinfo[i].pubconninfo, true);
@@ -780,22 +811,42 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
* no replication slot is specified. It follows the same rule as
* CREATE SUBSCRIPTION.
*/
- if (num_pubs == 0 || num_subs == 0 || num_replslots == 0)
+ if (dbinfo[i].pubname == NULL || dbinfo[i].subname == NULL || dbinfo[i].replslotname == NULL)
genname = generate_object_name(conn);
- if (num_pubs == 0)
+ if (dbinfo[i].pubname == NULL)
dbinfo[i].pubname = pg_strdup(genname);
- if (num_subs == 0)
+ if (dbinfo[i].subname == NULL)
dbinfo[i].subname = pg_strdup(genname);
- if (num_replslots == 0)
+ if (dbinfo[i].replslotname == NULL)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
+ /*
+ * Check if publication already exists when
+ * --reuse-existing-publications is specified
+ */
+ if (opt->reuse_existing_pubs && check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ make_pub = false;
+ }
+
/*
* Create publication on publisher. This step should be executed
* *before* promoting the subscriber to avoid any transactions between
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (make_pub)
+ {
+ create_publication(conn, &dbinfo[i]);
+ dbinfo[i].made_publication = true;
+ if (opt->reuse_existing_pubs)
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+ else
+ dbinfo[i].made_publication = false;
/* Create replication slot on publisher */
if (lsn)
@@ -1771,8 +1822,14 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
* those to provide necessary information to the user.
*/
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ if (!dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ }
+ else
+ pg_log_info("not dropping existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
}
/*
@@ -2047,6 +2104,7 @@ main(int argc, char **argv)
{"replication-slot", required_argument, NULL, 3},
{"subscription", required_argument, NULL, 4},
{"clean", required_argument, NULL, 5},
+ {"reuse-existing-publications", no_argument, NULL, 6},
{NULL, 0, NULL, 0}
};
@@ -2095,6 +2153,7 @@ main(int argc, char **argv)
opt.sub_port = DEFAULT_SUB_PORT;
opt.sub_username = NULL;
opt.two_phase = false;
+ opt.reuse_existing_pubs = false;
opt.database_names = (SimpleStringList)
{
0
@@ -2200,6 +2259,9 @@ main(int argc, char **argv)
else
pg_fatal("object type \"%s\" specified more than once for --clean", optarg);
break;
+ case 6:
+ opt.reuse_existing_pubs = true;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -2220,6 +2282,8 @@ main(int argc, char **argv)
bad_switch = "--replication-slot";
else if (num_subs > 0)
bad_switch = "--subscription";
+ else if (opt.reuse_existing_pubs)
+ bad_switch = "--reuse-existing-publications";
if (bad_switch)
{
@@ -2319,6 +2383,12 @@ main(int argc, char **argv)
}
}
+ if (opt.reuse_existing_pubs && num_pubs == 0)
+ {
+ pg_log_error("--reuse-existing-publications requires --publication to be specified");
+ exit(1);
+ }
+
/* Number of object names must match number of databases */
if (num_pubs > 0 && num_pubs != num_dbs)
{
@@ -2424,7 +2494,7 @@ main(int argc, char **argv)
stop_standby_server(subscriber_dir);
/* Create the required objects for each database on publisher */
- consistent_lsn = setup_publisher(dbinfos.dbinfo);
+ consistent_lsn = setup_publisher(dbinfos.dbinfo, &opt);
/* Write the required recovery parameters */
setup_recovery(dbinfos.dbinfo, subscriber_dir, consistent_lsn);
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..473ae7eac70 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,89 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber on node S2 with --reuse-existing-publications option for mixed
+# scenario (one existing publication, one new publication)
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--reuse-existing-publications',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications with --reuse-existing-publications"
+);
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.41.0.windows.3
On Tue, Sep 9, 2025, at 11:34 PM, Zhijie Hou (Fujitsu) wrote:
On Wednesday, September 3, 2025 9:58 AM Peter Smith
<smithpb2250@gmail.com> wrote:AFAICT, these problems don't happen if you allow a reinterpretation of
--publication switch like Euler had suggested [1]. The dry-run logs
should make it clear whether an existing publication will be reused or
not, so I'm not sure even that the validation code (checking if the
publication exists) is needed.Finally, I think the implementation might be simpler too -- e.g. fewer
rules/docs needed, no option conflict code needed.I'm hesitant to directly change the behavior of --publication, as it seems
unexpected for the command in the new version to silently use an existing
publication when it previously reported an ERROR. Although the number of users
relying on the ERROR may be small, the change still appears risky to me.
I don't buy this argument. Every feature that is not supported emits an error.
You can interpret it as (a) we don't support using existing publications until
v18 but we will start support it from now on or (b) it is a hard error that we
shouldn't allow, hence, stop now. I would say this case is (a) rather than (b).
The proposed extra option would do pg_createsubscriber behaves in a different
way if it is specified or not. That's not a good UI. It will cause confusion.
If you are not reading the manual, it isn't intuitive that you need to specify
an extra option until executing it. If you have a good hint message, the second
try can succeed.
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Thu, Sep 11, 2025 at 6:34 PM Euler Taveira <euler@eulerto.com> wrote:
On Tue, Sep 9, 2025, at 11:34 PM, Zhijie Hou (Fujitsu) wrote:
On Wednesday, September 3, 2025 9:58 AM Peter Smith
<smithpb2250@gmail.com> wrote:AFAICT, these problems don't happen if you allow a reinterpretation of
--publication switch like Euler had suggested [1]. The dry-run logs
should make it clear whether an existing publication will be reused or
not, so I'm not sure even that the validation code (checking if the
publication exists) is needed.Finally, I think the implementation might be simpler too -- e.g. fewer
rules/docs needed, no option conflict code needed.I'm hesitant to directly change the behavior of --publication, as it seems
unexpected for the command in the new version to silently use an existing
publication when it previously reported an ERROR. Although the number of users
relying on the ERROR may be small, the change still appears risky to me.I don't buy this argument. Every feature that is not supported emits an error.
You can interpret it as (a) we don't support using existing publications until
v18 but we will start support it from now on or (b) it is a hard error that we
shouldn't allow, hence, stop now. I would say this case is (a) rather than (b).
Yeah, I am also not sure that it is worth adding a new option to save
backward compatibility for this option. It seems intuitive to use
existing publications when they exist, though we should clearly
document it.
--
With Regards,
Amit Kapila.
On Fri, Sep 12, 2025, at 2:05 AM, Amit Kapila wrote:
Yeah, I am also not sure that it is worth adding a new option to save
backward compatibility for this option. It seems intuitive to use
existing publications when they exist, though we should clearly
document it.
That's why we have a "Migration to Version XY" in the release notes. ;)
--
Euler Taveira
EDB https://www.enterprisedb.com/
Hi Shubham,
IIUC the v6 will be rewritten to remove the new option, in favour of
just redefining the --publication option behaviour.
So, much of the current v6 will become obsolete. The below comment is
just for one piece of code that I thought will survive the rewrite.
======
src/bin/pg_basebackup/pg_createsubscriber.c
setup_publisher:
1.
+ /*
+ * Check if publication already exists when
+ * --reuse-existing-publications is specified
+ */
+ if (opt->reuse_existing_pubs && check_publication_exists(conn,
dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ make_pub = false;
+ }
+
/*
* Create publication on publisher. This step should be executed
* *before* promoting the subscriber to avoid any transactions between
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (make_pub)
+ {
+ create_publication(conn, &dbinfo[i]);
+ dbinfo[i].made_publication = true;
+ if (opt->reuse_existing_pubs)
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
+ else
+ dbinfo[i].made_publication = false;
I think there are still too many if/else here. This logic can be
simplified like below:
if (check_publication_exists(...))
{
pg_log_info("using existing publication...");
dbinfo[i].made_publication = false;
}
else
{
create_publication(conn, &dbinfo[i]);
pg_log_info("created publication ...");
dbinfo[i].made_publication = true;
}
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Mon, Sep 15, 2025 at 6:01 AM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
IIUC the v6 will be rewritten to remove the new option, in favour of
just redefining the --publication option behaviour.So, much of the current v6 will become obsolete. The below comment is
just for one piece of code that I thought will survive the rewrite.======
src/bin/pg_basebackup/pg_createsubscriber.csetup_publisher:
1. + /* + * Check if publication already exists when + * --reuse-existing-publications is specified + */ + if (opt->reuse_existing_pubs && check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname)) + { + pg_log_info("using existing publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + make_pub = false; + } + /* * Create publication on publisher. This step should be executed * *before* promoting the subscriber to avoid any transactions between * consistent LSN and the new publication rows (such transactions * wouldn't see the new publication rows resulting in an error). */ - create_publication(conn, &dbinfo[i]); + if (make_pub) + { + create_publication(conn, &dbinfo[i]); + dbinfo[i].made_publication = true; + if (opt->reuse_existing_pubs) + pg_log_info("created publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + } + else + dbinfo[i].made_publication = false;I think there are still too many if/else here. This logic can be
simplified like below:if (check_publication_exists(...))
{
pg_log_info("using existing publication...");
dbinfo[i].made_publication = false;
}
else
{
create_publication(conn, &dbinfo[i]);
pg_log_info("created publication ...");
dbinfo[i].made_publication = true;
}
I have now removed the extra option and reworked the patch so that the
behavior is handled directly through the --publication option. This
way, the command will either reuse an existing publication (if it
already exists) or create a new one, making it simpler and more
intuitive for users.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
Attachments:
v7-0001-Support-existing-publications-in-pg_createsubscri.patchapplication/octet-stream; name=v7-0001-Support-existing-publications-in-pg_createsubscri.patchDownload
From e842cc85f752539e6e238e05f252b7ab00c5abf8 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v7] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication
already existed. Now, existing publications are reused as-is, and
non-existing publications are created automatically with FOR ALL TABLES.
This change eliminates the need to know in advance which publications exist
on the publisher, making the tool more user-friendly. Users can specify
publication names and the tool will handle both existing and new publications
appropriately.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses. Only
publications created by pg_createsubscriber are cleaned up.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 22 ++++++
src/bin/pg_basebackup/pg_createsubscriber.c | 56 ++++++++++++--
.../t/040_pg_createsubscriber.pl | 77 +++++++++++++++++++
3 files changed, 148 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..db28956a441 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -285,6 +285,28 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused as-is with its current configuration, including its
+ table list, row filters, column filters, and all other settings.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ When reusing existing publications, you should understand their current
+ configuration. Existing publications are used exactly as configured,
+ which may replicate different tables than expected.
+ New publications created with <literal>FOR ALL TABLES</literal> will
+ replicate all tables in the database, which may be more than intended.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to see which publications will be reused
+ and which will be created before running the actual command.
+ When publications are reused, they will not be dropped during cleanup
+ operations, ensuring they remain available for other uses.
+ Only publications created by
+ <application>pg_createsubscriber</application> will be cleaned up.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..afdd7ba258e 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,32 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Check if a publication with the given name exists in the specified database.
+ * Returns true if it exists, false otherwise.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -780,13 +807,13 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
* no replication slot is specified. It follows the same rule as
* CREATE SUBSCRIPTION.
*/
- if (num_pubs == 0 || num_subs == 0 || num_replslots == 0)
+ if (dbinfo[i].pubname == NULL || dbinfo[i].subname == NULL || dbinfo[i].replslotname == NULL)
genname = generate_object_name(conn);
- if (num_pubs == 0)
+ if (dbinfo[i].pubname == NULL)
dbinfo[i].pubname = pg_strdup(genname);
- if (num_subs == 0)
+ if (dbinfo[i].subname == NULL)
dbinfo[i].subname = pg_strdup(genname);
- if (num_replslots == 0)
+ if (dbinfo[i].replslotname == NULL)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
/*
@@ -795,7 +822,19 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1771,8 +1810,11 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
* those to provide necessary information to the user.
*/
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ }
}
/*
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..1fb46042090 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,86 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber on node S2
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
Here are some v7 review comments.
======
Commit message
1.
This change eliminates the need to know in advance which publications exist
on the publisher, making the tool more user-friendly. Users can specify
publication names and the tool will handle both existing and new publications
appropriately.
~
I disagree with the "eliminates the need to know" part. This change
doesn't remove that responsibility from the user. They still need to
be aware of what the existing publications are so they don't end up
reusing a publication they did not intend to reuse.
~~~
2. <general>
It would be better if the commit message wording was consistent with
the wording in the docs.
======
doc/src/sgml/ref/pg_createsubscriber.sgml
3.
+ <para>
+ When reusing existing publications, you should understand their current
+ configuration. Existing publications are used exactly as configured,
+ which may replicate different tables than expected.
+ New publications created with <literal>FOR ALL TABLES</literal> will
+ replicate all tables in the database, which may be more than intended.
+ </para>
Is that paragraph needed? What does it say that was not already said
by the previous paragraph?
~~~
4.
+ <para>
+ Use <option>--dry-run</option> to see which publications will be reused
+ and which will be created before running the actual command.
+ When publications are reused, they will not be dropped during cleanup
+ operations, ensuring they remain available for other uses.
+ Only publications created by
+ <application>pg_createsubscriber</application> will be cleaned up.
+ </para>
There also needs to be more clarity about the location of the
publications that are getting "cleaned up". AFAIK the function
check_and_drop_publications() only cleans up for the target server,
but that does not seem at all obvious here.
======
src/bin/pg_basebackup/pg_createsubscriber.c
setup_publisher:
5.
- if (num_pubs == 0 || num_subs == 0 || num_replslots == 0)
+ if (dbinfo[i].pubname == NULL || dbinfo[i].subname == NULL ||
dbinfo[i].replslotname == NULL)
genname = generate_object_name(conn);
- if (num_pubs == 0)
+ if (dbinfo[i].pubname == NULL)
dbinfo[i].pubname = pg_strdup(genname);
- if (num_subs == 0)
+ if (dbinfo[i].subname == NULL)
dbinfo[i].subname = pg_strdup(genname);
- if (num_replslots == 0)
+ if (dbinfo[i].replslotname == NULL)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
~
Are these changes related to the --publication change, or are these
some other issue?
~~~
6.
* consistent LSN and the new publication rows (such transactions
* wouldn't see the new publication rows resulting in an error).
*/
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
The comment preceding this if/else is only talking about "Create the
publications..", but it should be more like "Reuse existing or create
new publications...". Alternatively, move the comments within the if
and else.
~~~
check_and_drop_publications:
7.
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ }
I find this function logic confusing. In particular, why is the
"made_publication" flag only checked for dry_run?
This function comment says it will "drop all pre-existing
publications." but doesn't that contradict your commit message and
docs that said statements like "When publications are reused, they are
never dropped during cleanup operations"?
======
.../t/040_pg_createsubscriber.pl
8.
IMO one of the most important things for the user is that they must be
able to know exactly which publications will be reused, and which
publications will be created as FOR ALL TABLES. So, there should be a
test to verify that the --dry_run option emits all the necessary
logging so the user can tell that.
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Tue, Sep 16, 2025 at 7:20 AM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
Here are some v7 review comments.
======
Commit message1.
This change eliminates the need to know in advance which publications exist
on the publisher, making the tool more user-friendly. Users can specify
publication names and the tool will handle both existing and new publications
appropriately.~
I disagree with the "eliminates the need to know" part. This change
doesn't remove that responsibility from the user. They still need to
be aware of what the existing publications are so they don't end up
reusing a publication they did not intend to reuse.~~~
2. <general>
It would be better if the commit message wording was consistent with
the wording in the docs.
Updated the commit message as suggested.
======
doc/src/sgml/ref/pg_createsubscriber.sgml3. + <para> + When reusing existing publications, you should understand their current + configuration. Existing publications are used exactly as configured, + which may replicate different tables than expected. + New publications created with <literal>FOR ALL TABLES</literal> will + replicate all tables in the database, which may be more than intended. + </para>Is that paragraph needed? What does it say that was not already said
by the previous paragraph?
Removed.
~~~
4. + <para> + Use <option>--dry-run</option> to see which publications will be reused + and which will be created before running the actual command. + When publications are reused, they will not be dropped during cleanup + operations, ensuring they remain available for other uses. + Only publications created by + <application>pg_createsubscriber</application> will be cleaned up. + </para>There also needs to be more clarity about the location of the
publications that are getting "cleaned up". AFAIK the function
check_and_drop_publications() only cleans up for the target server,
but that does not seem at all obvious here.
Fixed.
======
src/bin/pg_basebackup/pg_createsubscriber.csetup_publisher:
5. - if (num_pubs == 0 || num_subs == 0 || num_replslots == 0) + if (dbinfo[i].pubname == NULL || dbinfo[i].subname == NULL || dbinfo[i].replslotname == NULL) genname = generate_object_name(conn); - if (num_pubs == 0) + if (dbinfo[i].pubname == NULL) dbinfo[i].pubname = pg_strdup(genname); - if (num_subs == 0) + if (dbinfo[i].subname == NULL) dbinfo[i].subname = pg_strdup(genname); - if (num_replslots == 0) + if (dbinfo[i].replslotname == NULL) dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname); ~Are these changes related to the --publication change, or are these
some other issue?
No, these changes are not related to the --publication modification.
They were unrelated adjustments, and I have now reverted them back to
match the behavior in HEAD.
~~~
6. * consistent LSN and the new publication rows (such transactions * wouldn't see the new publication rows resulting in an error). */ - create_publication(conn, &dbinfo[i]); + if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))The comment preceding this if/else is only talking about "Create the
publications..", but it should be more like "Reuse existing or create
new publications...". Alternatively, move the comments within the if
and else.
Fixed.
~~~
check_and_drop_publications:
7. if (!drop_all_pubs || dry_run) - drop_publication(conn, dbinfo->pubname, dbinfo->dbname, - &dbinfo->made_publication); + { + if (dbinfo->made_publication) + drop_publication(conn, dbinfo->pubname, dbinfo->dbname, + &dbinfo->made_publication); + }I find this function logic confusing. In particular, why is the
"made_publication" flag only checked for dry_run?This function comment says it will "drop all pre-existing
publications." but doesn't that contradict your commit message and
docs that said statements like "When publications are reused, they are
never dropped during cleanup operations"?
Fixed.
======
.../t/040_pg_createsubscriber.pl8.
IMO one of the most important things for the user is that they must be
able to know exactly which publications will be reused, and which
publications will be created as FOR ALL TABLES. So, there should be a
test to verify that the --dry_run option emits all the necessary
logging so the user can tell that.
Fixed.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
Attachments:
v8-0001-Support-existing-publications-in-pg_createsubscri.patchapplication/octet-stream; name=v8-0001-Support-existing-publications-in-pg_createsubscri.patchDownload
From 96b89e2ec34a8a948cc771a30c62d95b1c8112df Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v8] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are createdcautomatically with
FOR ALL TABLES.
This change provides flexibility when working with mixed scenarios of existing
and new publications. Users should verify that existing publications have the
desired configuration before reusing them, and can use --dry-run to see which
publications will be reused and which will be created.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses.
Only publications created by pg_createsubscriber are cleaned up.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 17 +++
src/bin/pg_basebackup/pg_createsubscriber.c | 61 +++++++--
.../t/040_pg_createsubscriber.pl | 116 ++++++++++++++++++
3 files changed, 184 insertions(+), 10 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..3d057013a97 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -285,6 +285,23 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused as-is with its current configuration, including its
+ table list, row filters, column filters, and all other settings.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to see which publications will be reused
+ and which will be created before running the actual command.
+ When publications are reused, they will not be dropped during cleanup
+ operations, ensuring they remain available for other uses.
+ Only publications created by
+ <application>pg_createsubscriber</application> on the target server will
+ be cleaned up if the operation fails. Publications on the publisher
+ server are never modified or dropped by cleanup operations.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..c8155959ef4 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,32 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Check if a publication with the given name exists in the specified database.
+ * Returns true if it exists, false otherwise.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -789,13 +816,27 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
- /*
- * Create publication on publisher. This step should be executed
- * *before* promoting the subscriber to avoid any transactions between
- * consistent LSN and the new publication rows (such transactions
- * wouldn't see the new publication rows resulting in an error).
- */
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1767,10 +1808,10 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
}
/*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
+ * Only drop publications that were created by pg_createsubscriber during
+ * this operation. Pre-existing publications are preserved.
*/
- if (!drop_all_pubs || dry_run)
+ if (dbinfo->made_publication)
drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
&dbinfo->made_publication);
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..9d9c7259598 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,125 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+# Run pg_createsubscriber on node S2 with --dry-run
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber --dry-run on node S2');
+
+like(
+ $stderr,
+ qr/using existing publication "test_pub_existing"/,
+ 'dry-run logs reuse of existing publication');
+like(
+ $stderr,
+ qr/created publication "test_pub_new"/,
+ 'dry-run logs creation of new publication');
+
+# Verify the existing publication is still there and unchanged
+my $existing_pub_count = $node_p->safe_psql($db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'"
+);
+is($existing_pub_count, '1',
+ 'existing publication remains unchanged after dry-run');
+
+# Verify no actual publications were created during dry-run
+my $pub_count_after_dry_run = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($pub_count_after_dry_run, '0',
+ 'dry-run did not actually create publications');
+
+# Run pg_createsubscriber on node S2 without --dry-run
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
Here are some v8 review comments.
======
Commit message
1.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are createdcautomatically with
FOR ALL TABLES.
~
typo: "createdcautomatically"
======
doc/src/sgml/ref/pg_createsubscriber.sgml
2.
+ <para>
+ Use <option>--dry-run</option> to see which publications will be reused
+ and which will be created before running the actual command.
+ When publications are reused, they will not be dropped during cleanup
+ operations, ensuring they remain available for other uses.
+ Only publications created by
+ <application>pg_createsubscriber</application> on the target server will
+ be cleaned up if the operation fails. Publications on the publisher
+ server are never modified or dropped by cleanup operations.
+ </para>
I still find all this confusing, for the following reasons:
* The "dry-run" advice is OK, but the "cleanup of existing pubs" seems
like a totally different topic which has nothing much to do with the
--publication option. I'm wondering if you need to talk about cleaning
existing pubs, maybe that info belongs in the "Notes" part of this
documentation.
* I don't understand how does saying "existing pubs will not be
dropped" reconcile with the --clean=publication option which says it
will drop all publications? They seem contradictory.
======
src/bin/pg_basebackup/pg_createsubscriber.c
check_and_drop_publications:
3.
/*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
+ * Only drop publications that were created by pg_createsubscriber during
+ * this operation. Pre-existing publications are preserved.
*/
- if (!drop_all_pubs || dry_run)
+ if (dbinfo->made_publication)
drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
&dbinfo->made_publication);
3a.
Sorry, but I still have the same question that I had in my previous v7
review. This function logic will already remove all pre-existing
publications when the 'drop_all_pubs' variable is true. It seems
contrary to the commit message that says "When publications are
reused, they are never dropped during cleanup operations, ensuring
pre-existing publications remain available for other uses."
~
3b.
Kind of similar to the previous review -- If the 'drop_all_pubs'
variable is true, then AFAICT this code attempts to drop again one of
the publications that was already dropped in the earlier part of this
function?
~
3c.
If this drop logic is broken wrt the intended cleanup rules then it
also means more/better --clean option tests are needed, otherwise how
was this passing the tests?
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Tue, Sep 16, 2025 at 2:17 PM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
Here are some v8 review comments.
======
Commit message1.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are createdcautomatically with
FOR ALL TABLES.~
typo: "createdcautomatically"
Fixed.
======
doc/src/sgml/ref/pg_createsubscriber.sgml2. + <para> + Use <option>--dry-run</option> to see which publications will be reused + and which will be created before running the actual command. + When publications are reused, they will not be dropped during cleanup + operations, ensuring they remain available for other uses. + Only publications created by + <application>pg_createsubscriber</application> on the target server will + be cleaned up if the operation fails. Publications on the publisher + server are never modified or dropped by cleanup operations. + </para>I still find all this confusing, for the following reasons:
* The "dry-run" advice is OK, but the "cleanup of existing pubs" seems
like a totally different topic which has nothing much to do with the
--publication option. I'm wondering if you need to talk about cleaning
existing pubs, maybe that info belongs in the "Notes" part of this
documentation.* I don't understand how does saying "existing pubs will not be
dropped" reconcile with the --clean=publication option which says it
will drop all publications? They seem contradictory.
I have now moved the paragraph about publications being preserved or
dropped under the description of the --clean option instead, where it
fits more naturally. This way, the --publication section focuses only
on creation/reuse semantics, while the --clean section clearly
documents the cleanup behavior.
======
src/bin/pg_basebackup/pg_createsubscriber.ccheck_and_drop_publications:
3. /* - * In dry-run mode, we don't create publications, but we still try to drop - * those to provide necessary information to the user. + * Only drop publications that were created by pg_createsubscriber during + * this operation. Pre-existing publications are preserved. */ - if (!drop_all_pubs || dry_run) + if (dbinfo->made_publication) drop_publication(conn, dbinfo->pubname, dbinfo->dbname, &dbinfo->made_publication);3a.
Sorry, but I still have the same question that I had in my previous v7
review. This function logic will already remove all pre-existing
publications when the 'drop_all_pubs' variable is true. It seems
contrary to the commit message that says "When publications are
reused, they are never dropped during cleanup operations, ensuring
pre-existing publications remain available for other uses."
Fixed. The logic has been updated so that pre-existing publications
are not dropped, consistent with the commit message.
~
3b.
Kind of similar to the previous review -- If the 'drop_all_pubs'
variable is true, then AFAICT this code attempts to drop again one of
the publications that was already dropped in the earlier part of this
function?
Fixed. The redundant drop call has been removed.
~
3c.
If this drop logic is broken wrt the intended cleanup rules then it
also means more/better --clean option tests are needed, otherwise how
was this passing the tests?
I have added a dedicated test case for the --clean option to cover
these scenarios and verify correct cleanup behavior.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
Attachments:
v9-0001-Support-existing-publications-in-pg_createsubscri.patchapplication/octet-stream; name=v9-0001-Support-existing-publications-in-pg_createsubscri.patchDownload
From 515e55c870b67771d2779359ec0a90855c8c5fcd Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v9] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are created automatically with
FOR ALL TABLES.
This change provides flexibility when working with mixed scenarios of existing
and new publications. Users should verify that existing publications have the
desired configuration before reusing them, and can use --dry-run to see which
publications will be reused and which will be created.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses.
Only publications created by pg_createsubscriber are cleaned up.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 19 +++
src/bin/pg_basebackup/pg_createsubscriber.c | 77 +++++++--
.../t/040_pg_createsubscriber.pl | 153 ++++++++++++++++++
3 files changed, 235 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..730805689db 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -246,6 +246,14 @@ PostgreSQL documentation
other publications replicated from the source server to be dropped as
well.
</para>
+ <para>
+ When publications are reused, they will not be dropped during cleanup
+ operations, ensuring they remain available for other uses.
+ Only publications created by
+ <application>pg_createsubscriber</application> on the target server
+ will be cleaned up if the operation fails. Publications on the
+ publisher server are never modified or dropped by cleanup operations.
+ </para>
</listitem>
</itemizedlist>
</para>
@@ -285,6 +293,17 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused as-is with its current configuration, including its
+ table list, row filters, column filters, and all other settings.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to see which publications will be reused
+ and which will be created before running the actual command.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..cf547a76ce0 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,32 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Check if a publication with the given name exists in the specified database.
+ * Returns true if it exists, false otherwise.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -789,13 +816,27 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
- /*
- * Create publication on publisher. This step should be executed
- * *before* promoting the subscriber to avoid any transactions between
- * consistent LSN and the new publication rows (such transactions
- * wouldn't see the new publication rows resulting in an error).
- */
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1760,19 +1801,27 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
/* Drop each publication */
for (int i = 0; i < PQntuples(res); i++)
- drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
- &dbinfo->made_publication);
-
+ {
+ if (dbinfo->made_publication)
+ drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
+ &dbinfo->made_publication);
+ }
PQclear(res);
}
/*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
+ * Only drop publications that were created by pg_createsubscriber during
+ * this operation. Pre-existing publications are preserved.
*/
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserving existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
}
/*
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..7a69060a379 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,162 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 and node_s3 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3');
+$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+
+# Run pg_createsubscriber on node S2 with --dry-run
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber --dry-run on node S2');
+
+like(
+ $stderr,
+ qr/using existing publication "test_pub_existing"/,
+ 'dry-run logs reuse of existing publication');
+like(
+ $stderr,
+ qr/created publication "test_pub_new"/,
+ 'dry-run logs creation of new publication');
+
+# Verify the existing publication is still there and unchanged
+my $existing_pub_count = $node_p->safe_psql($db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'"
+);
+is($existing_pub_count, '1',
+ 'existing publication remains unchanged after dry-run');
+
+# Verify no actual publications were created during dry-run
+my $pub_count_after_dry_run = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($pub_count_after_dry_run, '0',
+ 'dry-run did not actually create publications');
+
+# Run pg_createsubscriber on node S2 without --dry-run
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
+$node_s3->start;
+$node_s3->stop;
+
+# Run pg_createsubscriber on node S3 without --dry-run and --clean option
+# to verify that the existing publications are preserved.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber on node S3');
+
+# Start subscriber
+$node_s3->start;
+
+# Confirm publication created by pg_createsubscriber is removed
+is( $node_s3->safe_psql(
+ $db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new';"
+ ),
+ '0',
+ 'publication created by pg_createsubscriber have been removed');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
+$node_s3->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
Here are some v9 review comments.
======
doc/src/sgml/ref/pg_createsubscriber.sgml
--clean:
1.
+ <para>
+ When publications are reused, they will not be dropped during cleanup
+ operations, ensuring they remain available for other uses.
+ Only publications created by
+ <application>pg_createsubscriber</application> on the target server
+ will be cleaned up if the operation fails. Publications on the
+ publisher server are never modified or dropped by cleanup operations.
+ </para>
1a.
OK, I understand now that you want the --clean switch to behave the
same as before and remove all publications, except now it will *not*
drop any publications that are being reused.
I can imagine how that might be convenient to keep those publications
around if the subscriber ends up being promoted to a primary node.
OTOH, it seems a bit peculiar that the behaviour of the --clean option
is now depending on the other --publication option.
I wonder what others think about this feature/quirk? e.g. maybe you
need to introduce a --clean=unused_publications?
~
1b.
Anyway, even if this behaviour is what you wanted, that text
describing --clean needs rewording.
Before this paragraph, the docs still say --clean=publications:
"...causes all other publications replicated from the source server to
be dropped as well."
Which is then immediately contradicted when you wrote:
"When publications are reused, they will not be dropped"
~~~
--publication:
2.
+ <para>
+ Use <option>--dry-run</option> to see which publications will be reused
+ and which will be created before running the actual command.
+ </para>
It seemed a bit strange to say "the actual command" because IMO it's
always an actual command regardless of the options.
SUGGEST:
Use --dry-run to safely preview which publications will be reused and
which will be created.
======
src/bin/pg_basebackup/pg_createsubscriber.c
check_and_drop_publications:
3.
The function comment has not been updated with the new rules. It still
says, "Additionally, if requested, drop all pre-existing
publications."
~~~
4.
/* Drop each publication */
for (int i = 0; i < PQntuples(res); i++)
- drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
- &dbinfo->made_publication);
-
+ {
+ if (dbinfo->made_publication)
+ drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
+ &dbinfo->made_publication);
+ }
PQclear(res);
}
/*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
+ * Only drop publications that were created by pg_createsubscriber during
+ * this operation. Pre-existing publications are preserved.
*/
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserving existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
4a.
I always find it difficult to follow the logic of this function.
AFAICT the "dry_mode" logic is already handled within the
drop_publication(), so check_and_drop_publications() can be simplified
by refactoring it:
CURRENT
if (drop_all_pubs)
{
...
}
if (!drop_all_pub || dry_mode)
{
...
}
SUGGEST
if (drop_all_pubs)
{
...
}
else
{
...
}
~
4b.
Why is "preserving existing publication" logged in a --dry-run only?
Shouldn't you see the same logs regardless of --dry-run? That's kind
of the whole point of a dry run, right?
======
.../t/040_pg_createsubscriber.pl
5.
+my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3');
+$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+
Should this be done later, where it is needed, combined with the
node_s3->start/stop?
~~~
6.
+# Run pg_createsubscriber on node S3 without --dry-run and --clean option
+# to verify that the existing publications are preserved.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber on node S3');
6a.
That comment wording "without --dry-run and --clean option" is
confusing because it sounds like "without --dry-run" and "without
--clean"
~
6b.
There are other untested combinations like --dry-run with --clean,
which would give more confidence about the logic of function
check_and_drop_publications().
Perhaps this test could have been written using --dry-run, and you
could be checking the logs for the expected "drop" or "preserving"
messages.
~~~
7.
+# Confirm publication created by pg_createsubscriber is removed
+is( $node_s3->safe_psql(
+ $db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new';"
+ ),
+ '0',
+ 'publication created by pg_createsubscriber have been removed');
+
Hmm. Here you are testing the new publication was deleted, so nowhere
is testing what the earlier comment said it was doing "...verify that
the existing publications are preserved.".
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Thu, Sep 18, 2025 at 5:23 AM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
Here are some v9 review comments.
======
doc/src/sgml/ref/pg_createsubscriber.sgml--clean:
1. + <para> + When publications are reused, they will not be dropped during cleanup + operations, ensuring they remain available for other uses. + Only publications created by + <application>pg_createsubscriber</application> on the target server + will be cleaned up if the operation fails. Publications on the + publisher server are never modified or dropped by cleanup operations. + </para>1a.
OK, I understand now that you want the --clean switch to behave the
same as before and remove all publications, except now it will *not*
drop any publications that are being reused.I can imagine how that might be convenient to keep those publications
around if the subscriber ends up being promoted to a primary node.
OTOH, it seems a bit peculiar that the behaviour of the --clean option
is now depending on the other --publication option.I wonder what others think about this feature/quirk? e.g. maybe you
need to introduce a --clean=unused_publications?~
1b.
Anyway, even if this behaviour is what you wanted, that text
describing --clean needs rewording.Before this paragraph, the docs still say --clean=publications:
"...causes all other publications replicated from the source server to
be dropped as well."Which is then immediately contradicted when you wrote:
"When publications are reused, they will not be dropped"
Removed this contradictory paragraph.
~~~
--publication:
2. + <para> + Use <option>--dry-run</option> to see which publications will be reused + and which will be created before running the actual command. + </para>It seemed a bit strange to say "the actual command" because IMO it's
always an actual command regardless of the options.SUGGEST:
Use --dry-run to safely preview which publications will be reused and
which will be created.
Fixed.
======
src/bin/pg_basebackup/pg_createsubscriber.ccheck_and_drop_publications:
3.
The function comment has not been updated with the new rules. It still
says, "Additionally, if requested, drop all pre-existing
publications."
Fixed.
~~~
4. /* Drop each publication */ for (int i = 0; i < PQntuples(res); i++) - drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname, - &dbinfo->made_publication); - + { + if (dbinfo->made_publication) + drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname, + &dbinfo->made_publication); + } PQclear(res); }/* - * In dry-run mode, we don't create publications, but we still try to drop - * those to provide necessary information to the user. + * Only drop publications that were created by pg_createsubscriber during + * this operation. Pre-existing publications are preserved. */ if (!drop_all_pubs || dry_run) - drop_publication(conn, dbinfo->pubname, dbinfo->dbname, - &dbinfo->made_publication); + { + if (dbinfo->made_publication) + drop_publication(conn, dbinfo->pubname, dbinfo->dbname, + &dbinfo->made_publication); + else + pg_log_info("preserving existing publication \"%s\" in database \"%s\"", + dbinfo->pubname, dbinfo->dbname); + }4a.
I always find it difficult to follow the logic of this function.
AFAICT the "dry_mode" logic is already handled within the
drop_publication(), so check_and_drop_publications() can be simplified
by refactoring it:CURRENT
if (drop_all_pubs)
{
...
}
if (!drop_all_pub || dry_mode)
{
...
}SUGGEST
if (drop_all_pubs)
{
...
}
else
{
...
}~
Fixed.
4b.
Why is "preserving existing publication" logged in a --dry-run only?
Shouldn't you see the same logs regardless of --dry-run? That's kind
of the whole point of a dry run, right?
Fixed.
======
.../t/040_pg_createsubscriber.pl5. +my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3'); +$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1); +Should this be done later, where it is needed, combined with the
node_s3->start/stop?
Fixed.
~~~
6. +# Run pg_createsubscriber on node S3 without --dry-run and --clean option +# to verify that the existing publications are preserved. +command_ok( + [ + 'pg_createsubscriber', + '--verbose', '--verbose', + '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default, + '--pgdata' => $node_s3->data_dir, + '--publisher-server' => $node_p->connstr($db1), + '--socketdir' => $node_s3->host, + '--subscriber-port' => $node_s3->port, + '--database' => $db1, + '--database' => $db2, + '--publication' => 'test_pub_existing', + '--publication' => 'test_pub_new', + '--clean' => 'publications', + ], + 'run pg_createsubscriber on node S3');6a.
That comment wording "without --dry-run and --clean option" is
confusing because it sounds like "without --dry-run" and "without
--clean"
Fixed.
~
6b.
There are other untested combinations like --dry-run with --clean,
which would give more confidence about the logic of function
check_and_drop_publications().Perhaps this test could have been written using --dry-run, and you
could be checking the logs for the expected "drop" or "preserving"
messages.
Added a test case with --dry-run and --clean=publications option.
~~~
7. +# Confirm publication created by pg_createsubscriber is removed +is( $node_s3->safe_psql( + $db1, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new';" + ), + '0', + 'publication created by pg_createsubscriber have been removed'); +Hmm. Here you are testing the new publication was deleted, so nowhere
is testing what the earlier comment said it was doing "...verify that
the existing publications are preserved.".
Fixed.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
Attachments:
v10-0001-Support-existing-publications-in-pg_createsubscr.patchapplication/octet-stream; name=v10-0001-Support-existing-publications-in-pg_createsubscr.patchDownload
From 50327da73a6e29cd90a2f84af1b652a3004a9fa7 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v10] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are created automatically with
FOR ALL TABLES.
This change provides flexibility when working with mixed scenarios of existing
and new publications. Users should verify that existing publications have the
desired configuration before reusing them, and can use --dry-run to see which
publications will be reused and which will be created.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses.
Only publications created by pg_createsubscriber are cleaned up.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 11 +
src/bin/pg_basebackup/pg_createsubscriber.c | 83 +++++--
.../t/040_pg_createsubscriber.pl | 202 ++++++++++++++++++
3 files changed, 278 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..2b9d74f07bf 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -285,6 +285,17 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused as-is with its current configuration, including its
+ table list, row filters, column filters, and all other settings.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to safely preview which publications will
+ be reused and which will be created.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..c1e7e5f5ace 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,32 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Check if a publication with the given name exists in the specified database.
+ * Returns true if it exists, false otherwise.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -789,13 +816,27 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
- /*
- * Create publication on publisher. This step should be executed
- * *before* promoting the subscriber to avoid any transactions between
- * consistent LSN and the new publication rows (such transactions
- * wouldn't see the new publication rows resulting in an error).
- */
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1732,8 +1773,10 @@ drop_publication(PGconn *conn, const char *pubname, const char *dbname,
* Since the publications were created before the consistent LSN, they
* remain on the subscriber even after the physical replica is
* promoted. Remove these publications from the subscriber because
- * they have no use. Additionally, if requested, drop all pre-existing
- * publications.
+ * they have no use. If --clean=publications is specified, drop all existing
+ * publications in the database. Otherwise, only drop publications that were
+ * created by pg_createsubscriber during this operation, preserving any
+ * pre-existing publications.
*/
static void
check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
@@ -1762,17 +1805,21 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
for (int i = 0; i < PQntuples(res); i++)
drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
&dbinfo->made_publication);
-
PQclear(res);
}
-
- /*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
- */
- if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ else
+ {
+ /*
+ * Only drop publications that were created by pg_createsubscriber
+ * during this operation. Pre-existing publications are preserved.
+ */
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserving existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
}
/*
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..850f29e6b42 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,211 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 and node_s3 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3');
+$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s3->start;
+$node_s3->stop;
+
+# Run pg_createsubscriber on node S2 with --dry-run
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber --dry-run on node S2');
+
+like(
+ $stderr,
+ qr/using existing publication "test_pub_existing"/,
+ 'dry-run logs reuse of existing publication');
+like(
+ $stderr,
+ qr/created publication "test_pub_new"/,
+ 'dry-run logs creation of new publication');
+
+# Verify the existing publication is still there and unchanged
+my $existing_pub_count = $node_p->safe_psql($db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'"
+);
+is($existing_pub_count, '1',
+ 'existing publication remains unchanged after dry-run');
+
+# Verify no actual publications were created during dry-run
+my $pub_count_after_dry_run = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($pub_count_after_dry_run, '0',
+ 'dry-run did not actually create publications');
+
+# Run pg_createsubscriber on node S2 without --dry-run
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
+# Run pg_createsubscriber on node S3 with --dry-run
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber --dry-run --clean=publications on node S3');
+
+like(
+ $stderr,
+ qr/dropping publication "test_pub_existing"/,
+ 'dry-run with --clean shows existing publication would be dropped');
+
+like(
+ $stderr,
+ qr/dropping publication "test_pub_new"/,
+ 'dry-run with --clean shows pg_createsubscriber publication would be dropped'
+);
+
+# Verify nothing was actually changed
+my $existing_pub_still_exists = $node_p->safe_psql($db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'"
+);
+is($existing_pub_still_exists, '1',
+ 'existing publication still exists after dry-run with --clean');
+
+my $new_pub_still_exists = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($new_pub_still_exists, '1',
+ 'pg_createsubscriber publication still exists after dry-run with --clean'
+);
+
+# Run pg_createsubscriber on node S3 with --clean option to verify that the
+# existing publications are preserved.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber on node S3');
+
+# Start subscriber
+$node_s3->start;
+
+# Confirm ALL publications were removed (both existing and new)
+is( $node_s3->safe_psql(
+ $db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing';"
+ ),
+ '0',
+ 'pre-existing publication was removed by --clean=publications');
+
+is( $node_s3->safe_psql(
+ $db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new';"
+ ),
+ '0',
+ 'publication created by pg_createsubscriber was removed by --clean=publications'
+);
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
+$node_s3->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
Hi Shubham,
Here are some v10 review comments.
======
1. GENERAL
A couple of my review comments below (#4, #7) are about the tense of
the info messages; My comments are trying to introduce some
consistency in the patch but unfortunately I see there are already
lots of existing messages where the tense is a muddle of past/present
(e.g. "creating"/"could not create" instead of "created"/"could not
create" or "creating"/"cannot create". Perhaps it's better to just
ignore my pg_log_info comments and do whatever seems right for this
patch, but also make another thread to fix the tense for all the
messages in one go.
======
Commit Message
2.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses.
Only publications created by pg_createsubscriber are cleaned up.
~
AFAICT, the implemented behaviour has flip-flopped again regarding
--clean; I think now --clean=publications is unchanged from master
(e.g. --clean=publication will drop all publications).
That's OK, but you need to ensure the commit message is saying the
same thing. e.g. currently it says "cleanup operations" never drop
reused publications, but what about "--clean=publications" -- that's a
cleanup operation, isn't it?
======
src/bin/pg_basebackup/pg_createsubscriber.c
check_publication_exists:
3.
+/*
+ * Check if a publication with the given name exists in the specified database.
+ * Returns true if it exists, false otherwise.
+ */
It seems a verbose way of saying:
Return whether a specified publication exists in the specified database.
~~~
check_and_drop_publications:
4.
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("using existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("created publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
+ }
The tense of these messages seems inconsistent, and is also different
from another nearby message: "create replication slot..."
So, should these pg_log_info change to match? For example:
"using existing publication" --> "use existing..."
"created publication" --> "create publication..."
~~~
check_and_drop_publications:
5.
* Since the publications were created before the consistent LSN, they
* remain on the subscriber even after the physical replica is
* promoted. Remove these publications from the subscriber because
- * they have no use. Additionally, if requested, drop all pre-existing
- * publications.
+ * they have no use. If --clean=publications is specified, drop all existing
+ * publications in the database. Otherwise, only drop publications that were
+ * created by pg_createsubscriber during this operation, preserving any
+ * pre-existing publications.
*/
This function comment seems overkill now that this function has
evolved. e.g. That whole first part (before "If --clean...") seems
like it would be better just as a comment within the function body
'else' block rather than needing to say anything in the function
comment.
~~~
6.
for (int i = 0; i < PQntuples(res); i++)
drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
&dbinfo->made_publication);
-
PQclear(res);
This whitespace change is not needed for the patch.
~~~
7.
+ else
+ {
+ /*
+ * Only drop publications that were created by pg_createsubscriber
+ * during this operation. Pre-existing publications are preserved.
+ */
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserving existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
For consistency with earlier review comment, maybe the message should
be reworded:
"preserving existing publication" --> "preserve existing publication"
======
.../t/040_pg_createsubscriber.pl
8.
+# Run pg_createsubscriber on node S2 without --dry-run
+command_ok(
and later...
+# Run pg_createsubscriber on node S2 without --dry-run
+command_ok(
~
These are not very informative comments. It should say more about the
purpose -- e.g. you are specifying --publication to reuse one
publication and create another ... to demonstrate that yada yada...
~~~
9.
+# Verify the existing publication is still there and unchanged
+my $existing_pub_count = $node_p->safe_psql($db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'"
+);
+is($existing_pub_count, '1',
+ 'existing publication remains unchanged after dry-run');
+
+# Verify no actual publications were created during dry-run
+my $pub_count_after_dry_run = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($pub_count_after_dry_run, '0',
+ 'dry-run did not actually create publications');
+
Instead of 2 SQLs to check whether something exists and something else
does not exist, can't you just have 1 SELECT to fetch all publication
names, then you can deduce the same thing from the result.
~~~
10.
+# Run pg_createsubscriber on node S3 with --dry-run
+($stdout, $stderr) = run_command(
Again, it's not a very informative comment. It should say more about
the purpose -- e.g. you are specifying --publication and
--clean=publications at the same time ... to demonstrate yada yada...
~~~
11.
+# Verify nothing was actually changed
+my $existing_pub_still_exists = $node_p->safe_psql($db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'"
+);
+is($existing_pub_still_exists, '1',
+ 'existing publication still exists after dry-run with --clean');
+
+my $new_pub_still_exists = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($new_pub_still_exists, '1',
+ 'pg_createsubscriber publication still exists after dry-run with --clean'
+);
+
Isn't this another example of something that is easily verified with a
single SQL instead of checking both publications separately?
~~~
12.
# Run pg_createsubscriber on node S3 with --clean option to verify that the
# existing publications are preserved.
command_ok(
Is that comment correct? I don't think so because --clean=publications
is supposed to drop all publications, right?
~~~
13.
+# Confirm ALL publications were removed (both existing and new)
+is( $node_s3->safe_psql(
+ $db1,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing';"
+ ),
+ '0',
+ 'pre-existing publication was removed by --clean=publications');
+
+is( $node_s3->safe_psql(
+ $db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new';"
+ ),
+ '0',
+ 'publication created by pg_createsubscriber was removed by
--clean=publications'
+);
+
Are 2x SELECT needed here? Can't you just have a single select to
discover that there are zero publications?
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Tue, Sep 23, 2025 at 1:22 PM Peter Smith <smithpb2250@gmail.com> wrote:
Hi Shubham,
Here are some v10 review comments.
======
1. GENERALA couple of my review comments below (#4, #7) are about the tense of
the info messages; My comments are trying to introduce some
consistency in the patch but unfortunately I see there are already
lots of existing messages where the tense is a muddle of past/present
(e.g. "creating"/"could not create" instead of "created"/"could not
create" or "creating"/"cannot create". Perhaps it's better to just
ignore my pg_log_info comments and do whatever seems right for this
patch, but also make another thread to fix the tense for all the
messages in one go.
As of now, I’ve addressed your comments related to the pg_log_info
messages. If you still feel there are more cases that should be
corrected, please let me know and I can start a separate thread to
handle tense consistency across all such messages.
======
Commit Message2.
When publications are reused, they are never dropped during cleanup operations,
ensuring pre-existing publications remain available for other uses.
Only publications created by pg_createsubscriber are cleaned up.~
AFAICT, the implemented behaviour has flip-flopped again regarding
--clean; I think now --clean=publications is unchanged from master
(e.g. --clean=publication will drop all publications).That's OK, but you need to ensure the commit message is saying the
same thing. e.g. currently it says "cleanup operations" never drop
reused publications, but what about "--clean=publications" -- that's a
cleanup operation, isn't it?
Fixed.
======
src/bin/pg_basebackup/pg_createsubscriber.ccheck_publication_exists:
3. +/* + * Check if a publication with the given name exists in the specified database. + * Returns true if it exists, false otherwise. + */It seems a verbose way of saying:
Return whether a specified publication exists in the specified database.
Fixed.
~~~
check_and_drop_publications:
4. + if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname)) + { + /* Reuse existing publication on publisher. */ + pg_log_info("using existing publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + dbinfo[i].made_publication = false; + } + else + { + /* + * Create publication on publisher. This step should be executed + * *before* promoting the subscriber to avoid any transactions + * between consistent LSN and the new publication rows (such + * transactions wouldn't see the new publication rows resulting in + * an error). + */ + create_publication(conn, &dbinfo[i]); + pg_log_info("created publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + dbinfo[i].made_publication = true; + }The tense of these messages seems inconsistent, and is also different
from another nearby message: "create replication slot..."So, should these pg_log_info change to match? For example:
"using existing publication" --> "use existing..."
"created publication" --> "create publication..."
Fixed.
~~~
check_and_drop_publications:
5. * Since the publications were created before the consistent LSN, they * remain on the subscriber even after the physical replica is * promoted. Remove these publications from the subscriber because - * they have no use. Additionally, if requested, drop all pre-existing - * publications. + * they have no use. If --clean=publications is specified, drop all existing + * publications in the database. Otherwise, only drop publications that were + * created by pg_createsubscriber during this operation, preserving any + * pre-existing publications. */This function comment seems overkill now that this function has
evolved. e.g. That whole first part (before "If --clean...") seems
like it would be better just as a comment within the function body
'else' block rather than needing to say anything in the function
comment.
Fixed.
~~~
6.
for (int i = 0; i < PQntuples(res); i++)
drop_publication(conn, PQgetvalue(res, i, 0), dbinfo->dbname,
&dbinfo->made_publication);
-
PQclear(res);
This whitespace change is not needed for the patch.
Fixed.
~~~
7. + else + { + /* + * Only drop publications that were created by pg_createsubscriber + * during this operation. Pre-existing publications are preserved. + */ + if (dbinfo->made_publication) + drop_publication(conn, dbinfo->pubname, dbinfo->dbname, + &dbinfo->made_publication); + else + pg_log_info("preserving existing publication \"%s\" in database \"%s\"", + dbinfo->pubname, dbinfo->dbname); + }For consistency with earlier review comment, maybe the message should
be reworded:
"preserving existing publication" --> "preserve existing publication"
Fixed.
======
.../t/040_pg_createsubscriber.pl8. +# Run pg_createsubscriber on node S2 without --dry-run +command_ok(and later...
+# Run pg_createsubscriber on node S2 without --dry-run +command_ok(~
These are not very informative comments. It should say more about the
purpose -- e.g. you are specifying --publication to reuse one
publication and create another ... to demonstrate that yada yada...
Fixed.
~~~
9. +# Verify the existing publication is still there and unchanged +my $existing_pub_count = $node_p->safe_psql($db1, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'" +); +is($existing_pub_count, '1', + 'existing publication remains unchanged after dry-run'); + +# Verify no actual publications were created during dry-run +my $pub_count_after_dry_run = $node_p->safe_psql($db2, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'"); +is($pub_count_after_dry_run, '0', + 'dry-run did not actually create publications'); +Instead of 2 SQLs to check whether something exists and something else
does not exist, can't you just have 1 SELECT to fetch all publication
names, then you can deduce the same thing from the result.
In the attached patch, I have removed that test case. It’s not really
valid to combine multiple switches in this way, since using
--publication together with --clean=publications is effectively the
same as running pg_createsubscriber twice (once with each switch).
That makes the extra combination test redundant, so I have dropped it.
~~~
10. +# Run pg_createsubscriber on node S3 with --dry-run +($stdout, $stderr) = run_command(Again, it's not a very informative comment. It should say more about
the purpose -- e.g. you are specifying --publication and
--clean=publications at the same time ... to demonstrate yada yada...
Fixed.
~~~
11. +# Verify nothing was actually changed +my $existing_pub_still_exists = $node_p->safe_psql($db1, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing'" +); +is($existing_pub_still_exists, '1', + 'existing publication still exists after dry-run with --clean'); + +my $new_pub_still_exists = $node_p->safe_psql($db2, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'"); +is($new_pub_still_exists, '1', + 'pg_createsubscriber publication still exists after dry-run with --clean' +); +Isn't this another example of something that is easily verified with a
single SQL instead of checking both publications separately?
Fixed.
~~~
12.
# Run pg_createsubscriber on node S3 with --clean option to verify that the
# existing publications are preserved.
command_ok(Is that comment correct? I don't think so because --clean=publications
is supposed to drop all publications, right?
Fixed.
~~~
13. +# Confirm ALL publications were removed (both existing and new) +is( $node_s3->safe_psql( + $db1, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_existing';" + ), + '0', + 'pre-existing publication was removed by --clean=publications'); + +is( $node_s3->safe_psql( + $db2, + "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new';" + ), + '0', + 'publication created by pg_createsubscriber was removed by --clean=publications' +); +Are 2x SELECT needed here? Can't you just have a single select to
discover that there are zero publications?
Fixed.
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
Attachments:
v11-0001-Support-existing-publications-in-pg_createsubscr.patchapplication/octet-stream; name=v11-0001-Support-existing-publications-in-pg_createsubscr.patchDownload
From 106a9a39ec7bfb74ec79d5eee06b7f02f6396221 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v11] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are created automatically with
FOR ALL TABLES.
This change provides flexibility when working with mixed scenarios of existing
and new publications. Users should verify that existing publications have the
desired configuration before reusing them, and can use --dry-run to see which
publications will be reused and which will be created.
Only publications created by pg_createsubscriber are cleaned up during error
cleanup operations. Pre-existing publications are preserved unless
'--clean=publications' is explicitly specified, which drops all publications.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 11 ++
src/bin/pg_basebackup/pg_createsubscriber.c | 86 +++++++---
.../t/040_pg_createsubscriber.pl | 162 ++++++++++++++++++
3 files changed, 239 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..2b9d74f07bf 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -285,6 +285,17 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused as-is with its current configuration, including its
+ table list, row filters, column filters, and all other settings.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to safely preview which publications will
+ be reused and which will be created.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..e7753d2ccf9 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,31 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Return whether a specified publication exists in the specified database.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -789,13 +815,27 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
- /*
- * Create publication on publisher. This step should be executed
- * *before* promoting the subscriber to avoid any transactions between
- * consistent LSN and the new publication rows (such transactions
- * wouldn't see the new publication rows resulting in an error).
- */
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("use existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("create publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1729,11 +1769,9 @@ drop_publication(PGconn *conn, const char *pubname, const char *dbname,
/*
* Retrieve and drop the publications.
*
- * Since the publications were created before the consistent LSN, they
- * remain on the subscriber even after the physical replica is
- * promoted. Remove these publications from the subscriber because
- * they have no use. Additionally, if requested, drop all pre-existing
- * publications.
+ * If --clean=publications is specified, drop all existing
+ * publications in the database. Otherwise, only drop publications that were
+ * created by pg_createsubscriber.
*/
static void
check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
@@ -1765,14 +1803,22 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
PQclear(res);
}
-
- /*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
- */
- if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ else
+ {
+ /*
+ * Since the publications were created before the consistent LSN, they
+ * remain on the subscriber even after the physical replica is
+ * promoted. Only drop publications that were created by
+ * pg_createsubscriber during this operation. Pre-existing
+ * publications are preserved.
+ */
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserve existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
}
/*
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..895610067b4 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,171 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 and node_s3 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3');
+$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s3->start;
+$node_s3->stop;
+
+# Test publication reuse and creation behavior with --dry-run.
+# This should reuse existing 'test_pub_existing' and create new 'test_pub_new',
+# demonstrating mixed publication handling without actual changes.
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber --dry-run on node S2');
+
+like(
+ $stderr,
+ qr/use existing publication "test_pub_existing"/,
+ 'dry-run logs reuse of existing publication');
+like(
+ $stderr,
+ qr/create publication "test_pub_new"/,
+ 'dry-run logs creation of new publication');
+
+# Verify dry-run did not modify publisher state
+my $pub_names_db1 = $node_p->safe_psql($db1,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is( $pub_names_db1, qq(pub1
+test_pub1
+test_pub2
+test_pub_existing),
+ "existing publication remains unchanged after dry-run");
+
+my $pub_names_db2 = $node_p->safe_psql($db2,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($pub_names_db2, 'pub2',
+ "dry-run did not actually create publications in db2");
+
+# Run pg_createsubscriber to test publication reuse and creation behavior.
+# This should reuse the existing 'test_pub_existing' publication in db1 and
+# create a new 'test_pub_new' publication in db2, demonstrating how the tool
+# handles mixed existing/new publication scenarios.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
+# Run pg_createsubscriber with --clean=publications to test that ALL
+# publications (both pre-existing and pg_createsubscriber-created) are dropped
+# when the --clean=publications option is used.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber on node S3');
+
+# Start subscriber
+$node_s3->start;
+
+# Confirm ALL publications were removed by --clean=publications
+my $remaining_pubs_db1 = $node_s3->safe_psql($db1,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($remaining_pubs_db1, '',
+ 'all publications were removed from db1 by --clean=publications');
+
+my $remaining_pubs_db2 = $node_s3->safe_psql($db2,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($remaining_pubs_db2, '',
+ 'all publications were removed from db2 by --clean=publications');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
+$node_s3->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.41.0.windows.3
Hi Shubham.
On Wed, Sep 24, 2025 at 4:37 PM Shubham Khanna
<khannashubham1197@gmail.com> wrote:
...
In the attached patch, I have removed that test case. It’s not really
valid to combine multiple switches in this way, since using
--publication together with --clean=publications is effectively the
same as running pg_createsubscriber twice (once with each switch).
That makes the extra combination test redundant, so I have dropped it.
I disagree with the "it's not really valid to combine..." part. IMO,
it should be perfectly valid to mix multiple switches if the tool
accepts them; otherwise, the tool ought to give an error for any
invalid combinations. OTOH, I agree that there is no point in
burdening this patch with redundant test cases.
//////////
Here are some v11 review comments.
======
src/sgml/ref/pg_createsubscriber.sgml
1.
+ <para>
+ If a publication with the specified name already exists on the
publisher,
+ it will be reused as-is with its current configuration, including its
+ table list, row filters, column filters, and all other settings.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
Below is a minor rewording suggestion for the first sentence by AI,
which I felt was a small improvement.
SUGGESTION:
If a publication with the specified name already exists on the
publisher, it will be reused with its current configuration, including
its table list, row filters, and column filters.
======
.../t/040_pg_createsubscriber.pl
2.
+# Confirm ALL publications were removed by --clean=publications
+my $remaining_pubs_db1 = $node_s3->safe_psql($db1,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($remaining_pubs_db1, '',
+ 'all publications were removed from db1 by --clean=publications');
+
+my $remaining_pubs_db2 = $node_s3->safe_psql($db2,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($remaining_pubs_db2, '',
+ 'all publications were removed from db2 by --clean=publications');
+
Since you are expecting no results, the "ORDER BY pubname" clauses are
redundant here. In hindsight, since you expect zero results, just
"SELECT COUNT(*) FROM pg_publications;" would also be ok.
======
Kind Regards,
Peter Smith
Fujitsu Australia
On Sep 24, 2025, at 14:37, Shubham Khanna <khannashubham1197@gmail.com> wrote:
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
<v11-0001-Support-existing-publications-in-pg_createsubscr.patch>
1.
```
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserve existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
```
Should we preserve “|| dry_run”? Because based on the old comment, in dry-run mode, even if we don’t create publications, we still want to inform the user.
2.
```
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("create publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = true;
```
dbinfo[i].made_publication = true; seems a redundant as create_publication() has done the assignment.
3.
```
@@ -1729,11 +1769,9 @@ drop_publication(PGconn *conn, const char *pubname, const char *dbname,
/*
* Retrieve and drop the publications.
*
- * Since the publications were created before the consistent LSN, they
- * remain on the subscriber even after the physical replica is
- * promoted. Remove these publications from the subscriber because
- * they have no use. Additionally, if requested, drop all pre-existing
- * publications.
+ * If --clean=publications is specified, drop all existing
+ * publications in the database. Otherwise, only drop publications that were
+ * created by pg_createsubscriber.
*/
static void
check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
```
The old comment clearly stated that deleting publication on target server, the updated comment loses that important information.
Regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Thu, Sep 25, 2025 at 9:02 AM Chao Li <li.evan.chao@gmail.com> wrote:
On Sep 24, 2025, at 14:37, Shubham Khanna <khannashubham1197@gmail.com> wrote:
The attached patch contains the suggested changes.
Thanks and regards,
Shubham Khanna.
<v11-0001-Support-existing-publications-in-pg_createsubscr.patch>1. ``` + if (dbinfo->made_publication) + drop_publication(conn, dbinfo->pubname, dbinfo->dbname, + &dbinfo->made_publication); + else + pg_log_info("preserve existing publication \"%s\" in database \"%s\"", + dbinfo->pubname, dbinfo->dbname); + } ```Should we preserve “|| dry_run”? Because based on the old comment, in dry-run mode, even if we don’t create publications, we still want to inform the user.
We don’t need to add an explicit "|| dry_run" here, since the
made_publication flag already accounts for that case. In dry-run mode,
no publications are actually created, so made_publication is never
set. This ensures we still hit the “preserve existing publication …”
branch and inform the user accordingly.
2. ``` + create_publication(conn, &dbinfo[i]); + pg_log_info("create publication \"%s\" in database \"%s\"", + dbinfo[i].pubname, dbinfo[i].dbname); + dbinfo[i].made_publication = true; ```dbinfo[i].made_publication = true; seems a redundant as create_publication() has done the assignment.
I have removed the redundant dbinfo[i].made_publication = true;, since
create_publication() already sets that flag.
3. ``` @@ -1729,11 +1769,9 @@ drop_publication(PGconn *conn, const char *pubname, const char *dbname, /* * Retrieve and drop the publications. * - * Since the publications were created before the consistent LSN, they - * remain on the subscriber even after the physical replica is - * promoted. Remove these publications from the subscriber because - * they have no use. Additionally, if requested, drop all pre-existing - * publications. + * If --clean=publications is specified, drop all existing + * publications in the database. Otherwise, only drop publications that were + * created by pg_createsubscriber. */ static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo) ```The old comment clearly stated that deleting publication on target server, the updated comment loses that important information.
I have adjusted the comments so that the function header now contains
the general description, while the else block comment explains the
subscriber (target server) context that was missing earlier. This way,
the header stays concise, and the important detail about where the
publications are being dropped is still preserved in the right place.
The attached patch contains the suggested changes. It also contains
the fix for Peter's comments at [1]/messages/by-id/CAHut+PsCQxWoPh-UXBUWu=6Pc6GuEQ4wnHZtDOwUnZN=krMxvQ@mail.gmail.com.
[1]: /messages/by-id/CAHut+PsCQxWoPh-UXBUWu=6Pc6GuEQ4wnHZtDOwUnZN=krMxvQ@mail.gmail.com
Thanks and regards,
Shubham Khanna.
Attachments:
v12-0001-Support-existing-publications-in-pg_createsubscr.patchapplication/octet-stream; name=v12-0001-Support-existing-publications-in-pg_createsubscr.patchDownload
From 68fe72a314c61c6f0afa0d894f951b0b485111e4 Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v12] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are created automatically with
FOR ALL TABLES.
This change provides flexibility when working with mixed scenarios of existing
and new publications. Users should verify that existing publications have the
desired configuration before reusing them, and can use --dry-run to see which
publications will be reused and which will be created.
Only publications created by pg_createsubscriber are cleaned up during error
cleanup operations. Pre-existing publications are preserved unless
'--clean=publications' is explicitly specified, which drops all publications.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 11 ++
src/bin/pg_basebackup/pg_createsubscriber.c | 83 ++++++---
.../t/040_pg_createsubscriber.pl | 162 ++++++++++++++++++
3 files changed, 235 insertions(+), 21 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..75e2edc83fd 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -285,6 +285,17 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused with its current configuration, including its table
+ list, row filters, and column filters.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to safely preview which publications will
+ be reused and which will be created.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..a503af921a1 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,31 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Return whether a specified publication exists in the specified database.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -789,13 +815,26 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
- /*
- * Create publication on publisher. This step should be executed
- * *before* promoting the subscriber to avoid any transactions between
- * consistent LSN and the new publication rows (such transactions
- * wouldn't see the new publication rows resulting in an error).
- */
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("use existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("create publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1728,12 +1767,6 @@ drop_publication(PGconn *conn, const char *pubname, const char *dbname,
/*
* Retrieve and drop the publications.
- *
- * Since the publications were created before the consistent LSN, they
- * remain on the subscriber even after the physical replica is
- * promoted. Remove these publications from the subscriber because
- * they have no use. Additionally, if requested, drop all pre-existing
- * publications.
*/
static void
check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
@@ -1765,14 +1798,22 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
PQclear(res);
}
-
- /*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
- */
- if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ else
+ {
+ /*
+ * Since the publications were created before the consistent LSN, they
+ * remain on the subscriber even after the physical replica is
+ * promoted. Only drop publications that were created by
+ * pg_createsubscriber during this operation. Pre-existing
+ * publications are preserved.
+ */
+ if (dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else
+ pg_log_info("preserve existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
}
/*
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..c4f10fdee7b 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,171 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 and node_s3 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3');
+$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s3->start;
+$node_s3->stop;
+
+# Test publication reuse and creation behavior with --dry-run.
+# This should reuse existing 'test_pub_existing' and create new 'test_pub_new',
+# demonstrating mixed publication handling without actual changes.
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber --dry-run on node S2');
+
+like(
+ $stderr,
+ qr/use existing publication "test_pub_existing"/,
+ 'dry-run logs reuse of existing publication');
+like(
+ $stderr,
+ qr/create publication "test_pub_new"/,
+ 'dry-run logs creation of new publication');
+
+# Verify dry-run did not modify publisher state
+my $pub_names_db1 = $node_p->safe_psql($db1,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is( $pub_names_db1, qq(pub1
+test_pub1
+test_pub2
+test_pub_existing),
+ "existing publication remains unchanged after dry-run");
+
+my $pub_names_db2 = $node_p->safe_psql($db2,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($pub_names_db2, 'pub2',
+ "dry-run did not actually create publications in db2");
+
+# Run pg_createsubscriber to test publication reuse and creation behavior.
+# This should reuse the existing 'test_pub_existing' publication in db1 and
+# create a new 'test_pub_new' publication in db2, demonstrating how the tool
+# handles mixed existing/new publication scenarios.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
+# Run pg_createsubscriber with --clean=publications to test that ALL
+# publications (both pre-existing and pg_createsubscriber-created) are dropped
+# when the --clean=publications option is used.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber on node S3');
+
+# Start subscriber
+$node_s3->start;
+
+# Confirm ALL publications were removed by --clean=publications
+my $pub_count_db1 =
+ $node_s3->safe_psql($db1, "SELECT COUNT(*) FROM pg_publication");
+is($pub_count_db1, '0',
+ 'all publications were removed from db1 by --clean=publications');
+
+my $pub_count_db2 =
+ $node_s3->safe_psql($db2, "SELECT COUNT(*) FROM pg_publication");
+is($pub_count_db2, '0',
+ 'all publications were removed from db2 by --clean=publications');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
+$node_s3->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.41.0.windows.3
On Sep 25, 2025, at 15:07, Shubham Khanna <khannashubham1197@gmail.com> wrote:
1. ``` + if (dbinfo->made_publication) + drop_publication(conn, dbinfo->pubname, dbinfo->dbname, + &dbinfo->made_publication); + else + pg_log_info("preserve existing publication \"%s\" in database \"%s\"", + dbinfo->pubname, dbinfo->dbname); + } ```Should we preserve “|| dry_run”? Because based on the old comment, in dry-run mode, even if we don’t create publications, we still want to inform the user.
We don’t need to add an explicit "|| dry_run" here, since the
made_publication flag already accounts for that case. In dry-run mode,
no publications are actually created, so made_publication is never
set. This ensures we still hit the “preserve existing publication …”
branch and inform the user accordingly.
I doubt that. Looking the code in create_publication():
if (!dry_run)
{
res = PQexec(conn, str->data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
pg_log_error("could not create publication \"%s\" in database \"%s\": %s",
dbinfo->pubname, dbinfo->dbname, PQresultErrorMessage(res));
disconnect_database(conn, true);
}
PQclear(res);
}
/* For cleanup purposes */
dbinfo->made_publication = true;
made_publication will always be set regardless of dry_run.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Thu, Sep 25, 2025 at 1:15 PM Chao Li <li.evan.chao@gmail.com> wrote:
On Sep 25, 2025, at 15:07, Shubham Khanna <khannashubham1197@gmail.com> wrote:
1. ``` + if (dbinfo->made_publication) + drop_publication(conn, dbinfo->pubname, dbinfo->dbname, + &dbinfo->made_publication); + else + pg_log_info("preserve existing publication \"%s\" in database \"%s\"", + dbinfo->pubname, dbinfo->dbname); + } ```Should we preserve “|| dry_run”? Because based on the old comment, in dry-run mode, even if we don’t create publications, we still want to inform the user.
We don’t need to add an explicit "|| dry_run" here, since the
made_publication flag already accounts for that case. In dry-run mode,
no publications are actually created, so made_publication is never
set. This ensures we still hit the “preserve existing publication …”
branch and inform the user accordingly.I doubt that. Looking the code in create_publication():
if (!dry_run)
{
res = PQexec(conn, str->data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
pg_log_error("could not create publication \"%s\" in database \"%s\": %s",
dbinfo->pubname, dbinfo->dbname, PQresultErrorMessage(res));
disconnect_database(conn, true);
}
PQclear(res);
}/* For cleanup purposes */
dbinfo->made_publication = true;made_publication will always be set regardless of dry_run.
You’re right — I made a mistake in my earlier explanation.
made_publication is always set in create_publication(), regardless of
dry-run. I have double-checked the dry-run output across the cases,
and from what I can see the messages are being logged correctly.
If you think there’s a specific combination where the dry-run logging
isn’t behaving as expected, could you point me to it? From my testing
it looks fine, but I want to be sure I’m not missing a corner case.
Thanks and regards,
Shubham Khanna.
On Sep 25, 2025, at 16:18, Shubham Khanna <khannashubham1197@gmail.com> wrote:
made_publication will always be set regardless of dry_run.
You’re right — I made a mistake in my earlier explanation.
made_publication is always set in create_publication(), regardless of
dry-run. I have double-checked the dry-run output across the cases,
and from what I can see the messages are being logged correctly.If you think there’s a specific combination where the dry-run logging
isn’t behaving as expected, could you point me to it? From my testing
it looks fine, but I want to be sure I’m not missing a corner case.
I think, here you code has a logic difference from the old code:
* With the old code, even if drop_all_pubs, as long as dry_run, it will still run drop_publication().
* With your code, if drop_all_pubs, then never run drop_publication(), because you moved the logic into “else”.
To be honest, I am not 100% sure which is correct, I am just pointing out the difference.
Regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Thu, Sep 25, 2025 at 6:36 PM Chao Li <li.evan.chao@gmail.com> wrote:
On Sep 25, 2025, at 16:18, Shubham Khanna <khannashubham1197@gmail.com> wrote:
made_publication will always be set regardless of dry_run.
You’re right — I made a mistake in my earlier explanation.
made_publication is always set in create_publication(), regardless of
dry-run. I have double-checked the dry-run output across the cases,
and from what I can see the messages are being logged correctly.If you think there’s a specific combination where the dry-run logging
isn’t behaving as expected, could you point me to it? From my testing
it looks fine, but I want to be sure I’m not missing a corner case.I think, here you code has a logic difference from the old code:
* With the old code, even if drop_all_pubs, as long as dry_run, it will still run drop_publication().
* With your code, if drop_all_pubs, then never run drop_publication(), because you moved the logic into “else”.To be honest, I am not 100% sure which is correct, I am just pointing out the difference.
Hi Shubham.
Chao is correct - that logic has changed slightly now. That stems from
my suggestion a couple of versions back to rewrite this as an if/else.
At that time, I thought there was a risk of a
double-drop/double-logging happening for the scenario of
'drop_all_pubs' in a 'dry_run'. In hindsight, it looks like that was
not possible because the 'drop_all_pubs' code can only drop
publications that it *finds*, but for 'dry_run', it's not going to
find the newly "made" publications because although
create_publication() was called, they were not actually created. I did
not recognise that was the intended meaning of the original code
comment.
~~
Even if the patch reverts to the original condition, there still seem
to be some quirks to be worked out:
* The original explanatory comment could have been better:
BEFORE
In dry-run mode, we don't create publications, but we still try to
drop those to provide necessary information to the user.
AFTER
In dry-run mode, create_publication() and drop_publication() do not
actually create or drop anything -- they only do logging. So, we still
need to call drop_publication() to log information to the user.
* I'm not sure if that "preserve existing" logging should be there or
not? What exactly is it for? If you reinstate the original
"(!drop_all_pubs || dry_run)" condition, then it seems possible to log
"preserve existing" also for 'drop_all_pubs', but that is contrary to
the docs. Is this just a leftover from a few versions back, when
'drop_all_pubs' would not drop everything?
* It is a bit concerning that although this function appears slightly
broken (e.g. causing wrong logging), the tests cannot detect it.
~~
The bottom line is, I think you'll need to make a matrix of every
possible combination of made=Y/N; drop_all_pub=Y/N; dry_run=Y/N; etc
and then decide exactly what logging you want for each. I don't know
if it is possible to automate such testing -- it might be overkill --
but at least the expected logging can be posted in this thread, so the
code can be reviewed properly.
======
Kind Regards,
Peter Smith.
Fujitsu Australia
On Fri, Sep 26, 2025 at 6:06 AM Peter Smith <smithpb2250@gmail.com> wrote:
On Thu, Sep 25, 2025 at 6:36 PM Chao Li <li.evan.chao@gmail.com> wrote:
On Sep 25, 2025, at 16:18, Shubham Khanna <khannashubham1197@gmail.com> wrote:
made_publication will always be set regardless of dry_run.
You’re right — I made a mistake in my earlier explanation.
made_publication is always set in create_publication(), regardless of
dry-run. I have double-checked the dry-run output across the cases,
and from what I can see the messages are being logged correctly.If you think there’s a specific combination where the dry-run logging
isn’t behaving as expected, could you point me to it? From my testing
it looks fine, but I want to be sure I’m not missing a corner case.I think, here you code has a logic difference from the old code:
* With the old code, even if drop_all_pubs, as long as dry_run, it will still run drop_publication().
* With your code, if drop_all_pubs, then never run drop_publication(), because you moved the logic into “else”.To be honest, I am not 100% sure which is correct, I am just pointing out the difference.
Hi Shubham.
Chao is correct - that logic has changed slightly now. That stems from
my suggestion a couple of versions back to rewrite this as an if/else.
At that time, I thought there was a risk of a
double-drop/double-logging happening for the scenario of
'drop_all_pubs' in a 'dry_run'. In hindsight, it looks like that was
not possible because the 'drop_all_pubs' code can only drop
publications that it *finds*, but for 'dry_run', it's not going to
find the newly "made" publications because although
create_publication() was called, they were not actually created. I did
not recognise that was the intended meaning of the original code
comment.~~
Even if the patch reverts to the original condition, there still seem
to be some quirks to be worked out:* The original explanatory comment could have been better:
BEFORE
In dry-run mode, we don't create publications, but we still try to
drop those to provide necessary information to the user.
AFTER
In dry-run mode, create_publication() and drop_publication() do not
actually create or drop anything -- they only do logging. So, we still
need to call drop_publication() to log information to the user.* I'm not sure if that "preserve existing" logging should be there or
not? What exactly is it for? If you reinstate the original
"(!drop_all_pubs || dry_run)" condition, then it seems possible to log
"preserve existing" also for 'drop_all_pubs', but that is contrary to
the docs. Is this just a leftover from a few versions back, when
'drop_all_pubs' would not drop everything?* It is a bit concerning that although this function appears slightly
broken (e.g. causing wrong logging), the tests cannot detect it.~~
The bottom line is, I think you'll need to make a matrix of every
possible combination of made=Y/N; drop_all_pub=Y/N; dry_run=Y/N; etc
and then decide exactly what logging you want for each. I don't know
if it is possible to automate such testing -- it might be overkill --
but at least the expected logging can be posted in this thread, so the
code can be reviewed properly.
Hi Peter, Chao,
Thanks for the detailed observations. I went back and prepared a
logging matrix for the different combinations of made_publication,
drop_all_pubs, and dry_run. This highlights where the behavior
diverges from the old code.
Summary of findings:
- When --clean=publications is used together with --dry-run, the code
correctly logs "dropping all existing publications", but it fails to
log the individual drop_publication() messages (e.g., "dropping
publication pubX").
- This affects both user-created (--publication=...) and auto-created
publications.
- Most other cases (new pub only, existing pub only, auto pub only)
behave as expected.
- New bug discovered: In the existing pub + clean case, the logs show
both "dropping publication pub1" and "preserve existing publication
pub1". This is contradictory and comes from the original code path
falling through to the “preserve” branch even when drop_all_pubs=true.
The restructuring into if/else caused the missing individual
drop_publication() logs in dry-run mode when drop_all_pubs=true. The
original condition also had a flaw: in the existing pub + clean case,
it could log both drop and preserve for the same publication.
I have updated the conditions so that:
- drop_publication() is always invoked in dry-run for correct logging.
- The “preserve existing” log is suppressed when drop_all_pubs=true,
eliminating the contradictory messages.
The attached patch contains the changes, and the attached image shows
the complete logging matrix for reference.
Thanks and regards,
Shubham Khanna.
Attachments:
v13-0001-Support-existing-publications-in-pg_createsubscr.patchapplication/x-patch; name=v13-0001-Support-existing-publications-in-pg_createsubscr.patchDownload
From ef2f8d373e04ebea1f8d057529d1e11642df755b Mon Sep 17 00:00:00 2001
From: Khanna <Shubham.Khanna@fujitsu.com>
Date: Thu, 28 Aug 2025 22:26:04 +0530
Subject: [PATCH v13] Support existing publications in pg_createsubscriber
Allow pg_createsubscriber to reuse existing publications instead of failing
when they already exist on the publisher.
Previously, pg_createsubscriber would fail if any specified publication already
existed. Now, existing publications are reused as-is with their current
configuration, and non-existing publications are created automatically with
FOR ALL TABLES.
This change provides flexibility when working with mixed scenarios of existing
and new publications. Users should verify that existing publications have the
desired configuration before reusing them, and can use --dry-run to see which
publications will be reused and which will be created.
Only publications created by pg_createsubscriber are cleaned up during error
cleanup operations. Pre-existing publications are preserved unless
'--clean=publications' is explicitly specified, which drops all publications.
---
doc/src/sgml/ref/pg_createsubscriber.sgml | 11 ++
src/bin/pg_basebackup/pg_createsubscriber.c | 80 +++++++--
.../t/040_pg_createsubscriber.pl | 162 ++++++++++++++++++
3 files changed, 234 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index bb9cc72576c..75e2edc83fd 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -285,6 +285,17 @@ PostgreSQL documentation
a generated name is assigned to the publication name. This option cannot
be used together with <option>--all</option>.
</para>
+ <para>
+ If a publication with the specified name already exists on the publisher,
+ it will be reused with its current configuration, including its table
+ list, row filters, and column filters.
+ If a publication does not exist, it will be created automatically with
+ <literal>FOR ALL TABLES</literal>.
+ </para>
+ <para>
+ Use <option>--dry-run</option> to safely preview which publications will
+ be reused and which will be created.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 3986882f042..ef9e70ceae7 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -114,6 +114,7 @@ static void stop_standby_server(const char *datadir);
static void wait_for_end_recovery(const char *conninfo,
const struct CreateSubscriberOptions *opt);
static void create_publication(PGconn *conn, struct LogicalRepInfo *dbinfo);
+static bool check_publication_exists(PGconn *conn, const char *pubname, const char *dbname);
static void drop_publication(PGconn *conn, const char *pubname,
const char *dbname, bool *made_publication);
static void check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo);
@@ -753,6 +754,31 @@ generate_object_name(PGconn *conn)
return objname;
}
+/*
+ * Return whether a specified publication exists in the specified database.
+ */
+static bool
+check_publication_exists(PGconn *conn, const char *pubname, const char *dbname)
+{
+ PGresult *res;
+ bool exists;
+ char *query;
+
+ query = psprintf("SELECT 1 FROM pg_publication WHERE pubname = %s",
+ PQescapeLiteral(conn, pubname, strlen(pubname)));
+ res = PQexec(conn, query);
+
+ if (PQresultStatus(res) != PGRES_TUPLES_OK)
+ pg_fatal("could not check for publication \"%s\" in database \"%s\": %s",
+ pubname, dbname, PQerrorMessage(conn));
+
+ exists = (PQntuples(res) == 1);
+
+ PQclear(res);
+ pg_free(query);
+ return exists;
+}
+
/*
* Create the publications and replication slots in preparation for logical
* replication. Returns the LSN from latest replication slot. It will be the
@@ -789,13 +815,26 @@ setup_publisher(struct LogicalRepInfo *dbinfo)
if (num_replslots == 0)
dbinfo[i].replslotname = pg_strdup(dbinfo[i].subname);
- /*
- * Create publication on publisher. This step should be executed
- * *before* promoting the subscriber to avoid any transactions between
- * consistent LSN and the new publication rows (such transactions
- * wouldn't see the new publication rows resulting in an error).
- */
- create_publication(conn, &dbinfo[i]);
+ if (check_publication_exists(conn, dbinfo[i].pubname, dbinfo[i].dbname))
+ {
+ /* Reuse existing publication on publisher. */
+ pg_log_info("use existing publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ dbinfo[i].made_publication = false;
+ }
+ else
+ {
+ /*
+ * Create publication on publisher. This step should be executed
+ * *before* promoting the subscriber to avoid any transactions
+ * between consistent LSN and the new publication rows (such
+ * transactions wouldn't see the new publication rows resulting in
+ * an error).
+ */
+ create_publication(conn, &dbinfo[i]);
+ pg_log_info("create publication \"%s\" in database \"%s\"",
+ dbinfo[i].pubname, dbinfo[i].dbname);
+ }
/* Create replication slot on publisher */
if (lsn)
@@ -1728,12 +1767,6 @@ drop_publication(PGconn *conn, const char *pubname, const char *dbname,
/*
* Retrieve and drop the publications.
- *
- * Since the publications were created before the consistent LSN, they
- * remain on the subscriber even after the physical replica is
- * promoted. Remove these publications from the subscriber because
- * they have no use. Additionally, if requested, drop all pre-existing
- * publications.
*/
static void
check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
@@ -1766,13 +1799,22 @@ check_and_drop_publications(PGconn *conn, struct LogicalRepInfo *dbinfo)
PQclear(res);
}
- /*
- * In dry-run mode, we don't create publications, but we still try to drop
- * those to provide necessary information to the user.
- */
if (!drop_all_pubs || dry_run)
- drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
- &dbinfo->made_publication);
+ {
+ /*
+ * Since the publications were created before the consistent LSN, they
+ * remain on the subscriber even after the physical replica is
+ * promoted. Only drop publications that were created by
+ * pg_createsubscriber during this operation. Pre-existing
+ * publications are preserved.
+ */
+ if (!drop_all_pubs && dbinfo->made_publication)
+ drop_publication(conn, dbinfo->pubname, dbinfo->dbname,
+ &dbinfo->made_publication);
+ else if (!drop_all_pubs)
+ pg_log_info("preserve existing publication \"%s\" in database \"%s\"",
+ dbinfo->pubname, dbinfo->dbname);
+ }
}
/*
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 229fef5b3b5..c4f10fdee7b 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -537,9 +537,171 @@ my $sysid_s = $node_s->safe_psql('postgres',
'SELECT system_identifier FROM pg_control_system()');
ok($sysid_p != $sysid_s, 'system identifier was changed');
+# Create user-defined publications.
+$node_p->safe_psql($db1,
+ "CREATE PUBLICATION test_pub_existing FOR TABLE tbl1");
+
+# Initialize node_s2 and node_s3 as a fresh standby of node_p for existing/new
+# publication test.
+$node_p->backup('backup_tablepub');
+my $node_s2 = PostgreSQL::Test::Cluster->new('node_s2');
+$node_s2->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s2->start;
+$node_s2->stop;
+
+my $node_s3 = PostgreSQL::Test::Cluster->new('node_s3');
+$node_s3->init_from_backup($node_p, 'backup_tablepub', has_streaming => 1);
+$node_s3->start;
+$node_s3->stop;
+
+# Test publication reuse and creation behavior with --dry-run.
+# This should reuse existing 'test_pub_existing' and create new 'test_pub_new',
+# demonstrating mixed publication handling without actual changes.
+($stdout, $stderr) = run_command(
+ [
+ 'pg_createsubscriber',
+ '--verbose',
+ '--dry-run',
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber --dry-run on node S2');
+
+like(
+ $stderr,
+ qr/use existing publication "test_pub_existing"/,
+ 'dry-run logs reuse of existing publication');
+like(
+ $stderr,
+ qr/create publication "test_pub_new"/,
+ 'dry-run logs creation of new publication');
+
+# Verify dry-run did not modify publisher state
+my $pub_names_db1 = $node_p->safe_psql($db1,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is( $pub_names_db1, qq(pub1
+test_pub1
+test_pub2
+test_pub_existing),
+ "existing publication remains unchanged after dry-run");
+
+my $pub_names_db2 = $node_p->safe_psql($db2,
+ "SELECT pubname FROM pg_publication ORDER BY pubname");
+is($pub_names_db2, 'pub2',
+ "dry-run did not actually create publications in db2");
+
+# Run pg_createsubscriber to test publication reuse and creation behavior.
+# This should reuse the existing 'test_pub_existing' publication in db1 and
+# create a new 'test_pub_new' publication in db2, demonstrating how the tool
+# handles mixed existing/new publication scenarios.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s2->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s2->host,
+ '--subscriber-port' => $node_s2->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ ],
+ 'run pg_createsubscriber on node S2');
+
+# Start subscriber
+$node_s2->start;
+
+# Verify that test_pub_new was created in db2
+$result = $node_p->safe_psql($db2,
+ "SELECT COUNT(*) FROM pg_publication WHERE pubname = 'test_pub_new'");
+is($result, '1', 'test_pub_new publication was created in db2');
+
+# Insert rows on P
+$node_p->safe_psql($db1, "INSERT INTO tbl1 VALUES('fourth row')");
+$node_p->safe_psql($db2, "INSERT INTO tbl2 VALUES('row 2')");
+
+# Get subscription names and publications
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT subname, subpublications FROM pg_subscription WHERE subname ~ '^pg_createsubscriber_'
+));
+@subnames = split("\n", $result);
+
+# Check result in database $db1
+$result = $node_s2->safe_psql($db1, 'SELECT * FROM tbl1');
+is( $result, qq(first row
+second row
+third row
+fourth row),
+ "logical replication works in database $db1");
+
+# Check result in database $db2
+$result = $node_s2->safe_psql($db2, 'SELECT * FROM tbl2');
+is( $result, qq(row 1
+row 2),
+ "logical replication works in database $db2");
+
+# Verify that the correct publications are being used
+$result = $node_s2->safe_psql(
+ 'postgres', qq(
+ SELECT s.subpublications
+ FROM pg_subscription s
+ WHERE s.subname ~ '^pg_createsubscriber_'
+ ORDER BY s.subdbid
+ )
+);
+
+is( $result, qq({test_pub_existing}
+{test_pub_new}),
+ "subscriptions use the correct publications");
+
+# Run pg_createsubscriber with --clean=publications to test that ALL
+# publications (both pre-existing and pg_createsubscriber-created) are dropped
+# when the --clean=publications option is used.
+command_ok(
+ [
+ 'pg_createsubscriber',
+ '--verbose', '--verbose',
+ '--recovery-timeout' => $PostgreSQL::Test::Utils::timeout_default,
+ '--pgdata' => $node_s3->data_dir,
+ '--publisher-server' => $node_p->connstr($db1),
+ '--socketdir' => $node_s3->host,
+ '--subscriber-port' => $node_s3->port,
+ '--database' => $db1,
+ '--database' => $db2,
+ '--publication' => 'test_pub_existing',
+ '--publication' => 'test_pub_new',
+ '--clean' => 'publications',
+ ],
+ 'run pg_createsubscriber on node S3');
+
+# Start subscriber
+$node_s3->start;
+
+# Confirm ALL publications were removed by --clean=publications
+my $pub_count_db1 =
+ $node_s3->safe_psql($db1, "SELECT COUNT(*) FROM pg_publication");
+is($pub_count_db1, '0',
+ 'all publications were removed from db1 by --clean=publications');
+
+my $pub_count_db2 =
+ $node_s3->safe_psql($db2, "SELECT COUNT(*) FROM pg_publication");
+is($pub_count_db2, '0',
+ 'all publications were removed from db2 by --clean=publications');
+
# clean up
$node_p->teardown_node;
$node_s->teardown_node;
+$node_s2->teardown_node;
+$node_s3->teardown_node;
$node_t->teardown_node;
$node_f->teardown_node;
--
2.34.1
image.pngimage/png; name=image.pngDownload
�PNG
IHDR ^�� IDATx��9��L�����M���G F N9u!$'s��$B�������(xw$�]��R/!�O���v��TU��J���� @ � @ xv�@ ������ � nG�z�v���+�0� �,xz�O_�8�K���UdB�39$UhU~�@ �@gd� @ � @�� �<$��C�
@ � @ ����~@ �<B��g�@ � �jl� @ �� ��������m �� @ � ��'P�!��zF�� @ � �7�� � �z��� @ �M �! @ � ��>C)� pI�� @ � @�� t���jhd� @ � �� �! @�Z��R�@ � ���
� @ OI���S+NA � �� @ � ��'����V���@ � @ ��� � �� ���,�� @ �K �! @ � ���D9���v�|��\��z=��i���|��G��/C �wM�����'g�����T�9B=�p`���2E�#�2QNy4"��P�V���fh���^�C�+�p�t�/�#�N��m�����fx �?���1Wu�9I���[���.����E�"�#�05u!Q������6����]NG��x��~��W��Z2��{&��G<\h�����l��<��
���/�����\�`{�c�l_h��Ib= p��[U;|�"��:q�?�V_������|�L��5��,AU����5�.Gr��}��2�@ �p8w�F.�����=�#qk/��{�SC��e]�����%�"��Eb�~,��
�edX�� Pu�^��'��.����Z����fT ��I�v:�6�f`�/�?��/#,� .B���6nY�n�v p����_��������/#,|me��kSyp��c>���p������N�Y�"ut7�6����_��2qU�K;u��|�
jM�� p/8���$����[�T�~X��W6���nY-�'��_0x�?�9\k��w���V���<���y���/T�w��MF��1�<�V��G�� �=��3��f�>�N6������l������p~�����!���#����-����CS5�~i
6�K Y����5���9�[��o��} L����Ch������a|�p`��x<_�k�,U�������E����������r�W����=��W?��.~&]@E���+��H�[����>,H�����S��J��vC���r$y,��^�(����p���!ofc;��:��ZL�������Ey����1$]����������b���UYc���p���4y��>�����l����n� �;oQ
r�jZk{�3�Q������m:9�
M����=��^p*5�n��: }\�z�������T&am�B�#K*+��x$��+8����y��%���H.{G����!�Q�'�l�����r*�~������O1���j�8�;���^��`��f����{Kv��Z���`���zW�Y�������C�t�q��)��j���j�����vG*:3��zDU���-������qz�*��n��J�B� �^�S�
���2wA�]NGH(y���wT�m�h�fK�m���i��0���
?M���;���WUq�d12Y�y�w;�$��^k����
�U���@��Wx����*.9�m��eR'D����z�v��
�j�zQ�(/J^����DZ��g����5Mj2S�E���It-Z���t�.nO�u�i�����{z< ��)7~W���k���-���8X����f1Mke/B2��A���wl���a,r���&i�������*)��9]��S�.��SfM7��
h�g�����������G�z:��Q��V��g��LG����8�/2w#�o�%�k+c����T�E�T��c9���W[���E�z)������/:�u
�q]�zo`vK���8]�fV�;R��9�(�T(<�Do�)���6Z}������v#� ���3w9��n�\� ��M��l%'��D���<�0�R1N�pqH
>�Rv?��:���n�Y���)�: ��@�� �v8�e�Q�x����>�� q
���e���-X�Yr���1�������.mJ�u�D#aR���7���}�l���'��������7�)���|��N��:��#`�gA���^�$�w5��>��=�U��WK�`��Wh�8q����-�hMtN�|�_�}��'mbk�$Wq)�2f�,"�Ce%�����g�Y��W�g���L��8P:�58�S��I�������(������ ���XV�;};������8��|]"�������[��=��k����tEW����z���>-�JI���{�lh@��jH��$���jw4�.�Z�\���E������K�6x�)�������={����9�[#�O��s&ozzn�F���J���A���U��Fd;7M�A8��i8�i5R���l~h���Q������i����
S����I��L���*�v=�����g���w��c��_�]�.eE���7��d�,e*=�s�=������������]��x���S�e��(���"�.��J�]��������v�E���G�R��E#�-:�Mr �oR7�E;����v~��b�YZg���b��tt-�D�i�����i]��2����CG�_O��f���Y�����i2�Q�J'�����G��o(�����Lq�Y�������ev~��E����~�S�p� >�$��^.�QnZQE'a�C���O=.�oq.�h.a���=2��~��\B��R�i�sDJswXL�f�l�;_�n����%�����>y��x���DO��$���z���"����|��&u*���t�|;����`�5��kl��������E����e��8���B������K��P 1�s�,g���)]����J�tR���L��\[�Ur���X����S�g��N�'�����[!��b���|
�_O���������NGL<i�+R��69�rLev��2�`��
{���iU��&{Z������#��$7s����{K�������W=ut�Cg�C1�CeU�gN�RC<����q�^��.
�FV=g�-{4:P���U}�},;jZ���+��_���8�=��W��*�������t9�1�IC\6��cBaEWs�q�~K���q�?�Zm���R�7w
�S���Z�9�����?��>y7��AQ������3���X�!����=�G�����~�:����!����enG����%UP�66�Ea�~�Q��v�C��p�/e{��_�g�]�7��=�8�Tmn eCp�M�A�r l���dr�����@�d� �D>�&�NU<�������3{����_����p2f�bo�9��M���8���t����,�a�r������#�9�K;3��������cIC�z|b��8$&<���S|'��d�c�N./��s,�A���x��LV�q/}2�1��?�{{��g�I�U��s�{U��j��J��+�����;���� ���^6IUs��8���%Hg�,G��x*�};���a���A G�9������v#�NOC��>z��������v�r��Z��g�U�Nbg.������Z �� ��h��I��
(gn|�Z��=0�����d�3��/�^����(��?���&]p�w�&����e����S��l�x�����Z>����>�b����S��w�����%����M5~�W�K�'��B�
���]�S�"����
��~ ��m���a�-*@:�L�X�9[�
D6�z5��>����n��\M>�9]�5i�p�_�����Jglu's{7�#u��I����9��O��512GK:��x��uWW]�]����weD����6T�f�VS�-:;L�^����(���r�#��I�:�����ia����k^��L�,�3k\Y9�V��v�A�� IDAT�6�\=�*��5f��U�s�)s��S���`o�+v�V��c<��<� �nK�� R;*@mEw���#�M������%��� �>f�����;U����K��*1��X%��g|5�X��n�����`4�N���6~��"���<��
���!��Ppu�A�ocf�������������[��������������{��opk�w�������v#�&gdg������3�!�'S�������nCg`����EY�����x�4Nd��N������"[M{�c9m�^n�+�L��8�c��1
���{���� ���Vt4j�n�Ch�&�_A�p�9�����E�J�u4v_vQ���:�rgv��\�p�L���~>-��m.���;�S�1���������''������\�:�]��� ��k���;���h�q��h��d��7�4�e�RZ��:3��t�����]g<V�
F�h^�0
�$��������"�+b�E�o��������P�����E����`T��2�Z�����~�>@!����7���c���N`=��:�����o}W3�t�=���4=�>F���N�z���t:�����f�2�����4V���@/�s�K�z89��i��������l��s�����������y��yXM�:7y��������pq���S��tY�\���i��ke5�W����@=Vd���Z��eU�s-)�O���4w��'~�c��q����(Y�?��r���8�c,����������y��h����Z��G��D9��&�����b��?Kf$�>_V���<�^j�i�9��n3����z7�C��82@��q8;GY��N������j]OuWS�.[����"Hw��LlA��O��2���������i���^�{�#�>L2���wv-���3�!K��*����������n��g���t���o�!�9V���J��>r�)�^��HC����bkM}gp`���r�p�$8���_�@fT����Y��c3�Q�?���~�_��1B��Q�`B���>�m����~�X������5��-M���e24W�y�I�v^�f�H���P�1���8�/����Z�����\=����:�����]�*�_��3��:WI�*=�e�
���18��qw��7c^�~��8��(N���U��M�*�������Z��i��ke����z�&�OO��V^�d�a�(�~J=�7��J"��:������nq�y,gP6>����}����l�q���N&#�g'�=-)��=6�
0J~���+�F~��5~X��.����-�{P0\��S�p�>��9�qk���F��U�o&`���x�O
�����x�V�$�� �d��=���NnvD�R�"�������y!f&P���<o�e��8�
5�Ii��I���S�yY����Z��d�^��3���9w��g��zYyH9;L�^D��8}�lbQG�B&��������Y�UV�������~]��S��!�����$�{v�vn�J]�t�#�j�l)-�ew�r���3�L�?��fK��%�W�*���Tx#�r�_f����!�(�,�x���@����>������O=z��t8���cmc�V�z��nm����}�'9��^K�L�(�����Wgaq�S"���L�l:���K�����������t���e4u�P%)����|T_��{��$C77�!M���g'p���e�R��L���{@��x��w��i�u��n�Q�5��f;u�_����� Ha����,���p���1�4jO��'�I)i��R��H%�^�p�zM{"��(*��R��1s�<�����#�Jnc����F��{������3�!KT@�W�b�:ZQ5Y���(����z0���s)+�$���q�>��+��z��|�����PN_��{�+�s5��Z�*���bV�\�+erZ��t��.���W��C���`SY�q!���@����0��T�QYEg�]��&����j����'4j �{_"�z�%tc���I��J�V>�Sh5<ibj�v���n�3��t��`�T/���'�iN%K��!�Dc��p�_���\����v�]{���\
���K�� �f���4��)����g��;�Le��:�$����/?)��qk��u�9�e4F��e���B<G
y/E u���H�'�����2��CJ���{z��;c�rc�^���3�J���@�F��Q��#������������3sv�����m�z�UT���\��JWq4�s�TW�jbC�x���:#~�^g�����\q�u,g4z�me25�l���6��{����p.�x��j������/�����Fu]�7����I��� ���c�YE'��5��8M@r�/u��9W�<�BB��T��E9�:��?�*�)���3�I���T�g�
���WYZ�U�����!P)�,A�cj:��������M �E�R�"���\���K ��A�2�G����*��}��]�x)ESR2R�~~M�<Gy/C�s�w"���w�������'i��Ei.sm��({�q�����X|�������(�����|������iK���i������u���t���hM6?�wm�����P�}W���?�,Y�����=u���N?�_W]����]�����Q�)��I%V�+z�d�([,��k�`V_��c\����N���^��NqE'�J�$�}���u�G���D��j'l/�i��N���m��|�v��a�
T*>��4�z��P%�|U����IT�w��*S��z�X�.����H�r1.���������X�3o�-rS�)<J_��i,@��q8k<�����ofr`KEPRm�$6W���Kb���uW�����{����Z��� �F�L�B�3�����j���e��9���\v9iggI�e���+`���Q�d[f�RV�*c��X��y*��Jr�`�����u*�f�����>�*���U�FK;�������sD���<�.�E���Ni]�;�DV+FC�c~�:S����$e�����EV{�nl��&�iU
����x�k�t6��� JH4���/���Z#r���{�������Y:����~��34�~�xul���t���Wf�6�Q��q�g���B�-`=Q��&�R�S�}&Yo� ��_�z:$�:�M�ai&��p������(����[T��J>��Gd�;����u�BV9�%����&��Hcc�` �����g#���,U�F��j���5�S�r�nU�/'Y9*�Z�]��ToU��r�����#�y��D�)�{��-
7GnVq�T����pu�#.ji�Ls�l/X��L�^���(VY<�'������
kGbN��N
`�kkJgNy4��a%n)}����s��;3��h���J�9wY�����i��P�]��#�
��j}���u=��)�!�/�(��=:�{��%.�Ww"��X�VF��+[��,_��]#6%>���J�/����h�f��Ikrp�'9���'��$� ���j�n���4i��N��l����D��WW������VHh���� @ � �x�������%�d,@�'����kPF p�V��-� @�o���K�������</����m����s� I� �{%�1~�%�]�@3�V�q" p&�C � ���7���h �x<��`0�.��Z=���a��JI\� ���P�7!�ZJ ��Vy�- @ � ���p�s7���#RU�C�{��Vfh@�$�!�1~�h� .@����"� �O �x)�E��q���2�����j�;�nXu��C�c�a�
C! �j�V����F�\� �&�Sj�L��'|x(��d�9���eQVl� �>Tab,
p�@a���[Lh��w �� @ � �*�� �/B���D @ �� R @ � �;'@h��� ��� @ � ��'��.Z�?&@ ���o����5p�_�2: �h��Qb������x�"���& ��
��/ ������S�f@���0���f������� IDAT�<���)+,} rH������ �� I! @ � @ x~xXC��j
6C � @ ��#�F@ ��� Z�6q�A � �� @ � ���>|� .O
� @ � <?<lK��j[b�� @ � ���@ ��� Z�y` @ �O ! @ � ��#@h���� p.�C � @ � ����l�V�F� @ � @ �4�C � �� ���+,� @�� `? @ � ���@!�"� �� @ � ��'��� ��?S$B � @ ��y�
@ x �V��0� �~����\z#���Z��� @ ��+ �������M �!p-���L�}+��zl������4�NG�AK��83�Cy���g���$����|<�������|���DI� @ OL@uHt{x������^w#5�����@"�� �\� ��+@F �C��/����w���0�b|��p?F��6�Dy&����h0~�sJ�9_@ I �! @ �H���#�6C xQ�/+_����/�j�&���!�=�a���5�R��6����L��U�`;��6g%�/ <0i>��na|�R�t@ �@���=�6@��
��"0YN26�;v�!��c~�L��/mX��������1�Av�� ��������� @ � �� ��� ��[PG' @ =�?�F�����doe.���,����k�1� �Kxu�j0}�,���������� ���%��� ��%l[:D�����z������q� .E`����������(#��Z�dT@��P�2���3"Q,�����Y7�������P�C�E�kG�l����>RVz
�Z"�
]S�7�Ix���0�+B1��@�s�`��Z#�������Y������2w�Vk,W!�w���Hp�W6����X�1���N��Gf!?}��������� �s�ZPSagP�!n#��uE����6]o�;)�1/h���d�����<���8w��ds4W���;*��ot
4%P���ZD�=�z���nX���Y�''�{wA���]F@ �.��|<�.�A�
��rz�{&�2���*2���P��#�/�3��$tYx�'��"�Ir��Hr���"1����9q�C��g���g�u�n/+��
^������[�f�������f���$H@ )-��TN�����1sV�?*�������G�rqc~����1��f�a]�~G��j���&��ir5����i|N��������u�.�O+ �*@ u��b; ��"�{�����g���V���>u����-�OF�U�h�N�
�I����)�"M:�srj���XH.��M;KDN�R�
?��"�)���[�]�"<���V_:�4% ��n�Te7����-=�j��o�X�:�R�5��Qr��G�[�1�C
�$� �:���ov��������e_�����)�:�*[V;�.�lk'�dQc��j�;c�h1W�U��E]m�S����4I�B`�������US��!"TIw�v� �X��?�*Y1� �� �z�(�@_�����������f11�bs8�>��m�%A����)�VeZc?�����uF�]�5��<k'�������}��S8��4I����w�
��#�>~6kd>��A)J��-1r"F��beE h���*<[z�5.���)�g-j���������A�E��� ^�@I��P�Ry�k�[u�U������-��#��O�-��m�UC��������>��.���\rn�K��Qu
�N��Uh��v�~UAx���R�ek�E���Y?���%��������$@h�>�� \��:_�>>=\�����{������n3��N��o��c�}��i����
?��e�5Bg���p��=�TL�����������}QS�1��W�]�Y�P�/;�Cz�������>��e@ hA U��6�".���*��U}�9S����W����|�����LG�2�y�cKc��������4{)�P\w,0Qz�����veuA��j�[����-�������u�@ � ��K$A x��SJ���,g{uq���_���/��z�'�+'=eF�N=e����~��~<���m�fE�y���W����_�'��{���Y%�D���-����-�/�3<31� �6�Z��f7���N%�j[�����'�Q������kk�������7~�c��<�������������H�1c�������K�ZB�K\��z�)Y� 3@ � ��+N�A���Hz�=�km�Mc<����Z��J�����/{�iS��Ur��n��+�\�b��%\�����s�Z�$� �"
Z��lu�b�e
�&����U�����{�CD���w~H��ci��W� ����AZ}���L@�� H���F��
K� �^r�F�c�
�I�^��zn��j���������HC���'�����4.�P��$�z���\�]�p���a5 ���-���b,6X�J�hX�
=�������W@���ff,"� �!�$��b�s:?���o#�=�X��K�{*���� pB���#/ �g"���������p?�%�&�qS�����3�@A�Y��!5��[:u��k�Qb�q\u���U[�Jr@ �����K0'u��{��]���XW���(��yknE�o�Z�� �Ze'���4eo#���l�j�t�dHn0�nM��g��nw����R�<��o)����c�M?�/��QmG @���N��^1�\4�dk���N.{�~>0���+�����F�����%��$�A�����>j�a7 �:�A��/"Z�w��&h���`�D5Lu}��a�y��O��f1�L����}�?���(L\�������]��>3���x�]�q ���=�j��X� ��.�_�Aq.���4��*�U3a�n�8W��<���%i������� � pS��-3��o��zf�F���K�����l��������g�p=�3S���N��H��\�� ���h���(�2�yGc�{�y�|M�@ ���5��Y�Hr�=�r�)�E�� p.B��$?�#�$��@���������:��������&f_&� ������_I�����]c��5��irr�a [@B@ 0�h&a�y'�q��sk���v���q��.���^���c�m{,����+��*z.�H>����4�>MQ� �f���E �������w!\���~ 5�Zj4�������W���B�������0/��Y��K��#�m�ZQk��_������u��U��m-�.F�+z/�e��p?Gq��� ?�C �@w[m�V����R�����[���IyU���U��3m[�������w�%��
7����.����?�aW�.Y��@ W#@h�j�Q@�Nl���>� ���������3�wa�#�L��*}k�19W(�i.�����4������V�i�Yq�������������E��������+;��{�:/5RF� ��9r @��@M�_u���1��b���/��w1��W��0<~��SS��-&o�����L�S��[Q��re���z,-�y{��lE��My��� �� ����H �� 7!��f~������z���|<�n���a���/X�\�m=��%�
i�f6�Y�Mk�IxN���mf��&���c;�k*��.���F�'_�*��i/9���A�c����6zY#���kUzz��Z�RJ����u�]woQqU{I���O��y��=��4�B x�-j
�E�����z
��X���Y5�QW�w���d�r�����pnk����4N���zj�=4��������u IDAT6��=N���Ft$�����(��z�F�3�Z����M�j�@�2����>o�� �B�~�f���TOK����_�z��O{[��sI��v��������� %������Jcr��#��.���l�J�Z-��H�9�-��]u�`lW������#��c_���o|����������\Mi�j�Z��=���v��}�-�K4�`[?5�Z �@�e 4q�}��Dj��4m�H��:�u[s��ic������Q����I����3,��nU�K��C���i���M�Y@�)B�MI����N6��n5�M\Q����jw<ec��E��LzO2�����wZ�4J��B������V���_�c�������JqP~i�6�a�l{�bM$�� M�*e�)o���t�����@��+��!�[��8�'.��h#�A ��� ��m!j��F6��������y{c��_"�����q�N�2���J��g�XTvw���@U�����%[����T��U� � ���� p>$@��'���p����YH0��B7�I2L�&�*�z#%�l�z��J����N$gd�
:� �T\3���{�����_����������������HQ���� L+I��S�i;-�P
�*AnzqOB��"�9W"�Y �{K�I���'����gnB \����
_�s���4�:����M��
�iv�&�V�*rbUS7��F9�PEZK��s��R�����_�/��S�l-r=�U���I��L{,�+{,�����N�ll�q(-"r��G���Wljf� �� �^� ��*l\� @ � �� @ � Z����� \� �^� �0�l��8 @ �k�V��x0�� `; @ � @ �O _� ��W.}|� @ hB`?�C��m��4�"@��g @ = ��#LDA � �'dA�nL6��LY,GDW��RpRP� @ xX�V��0��`���ds��a1�wS����[n^2 �����}9�i<���aX�.�t�W�uN�1(+�m����D�e�9[c�&�-2c�L>�l*3���f�G�N�D��'�c]��nu
��J�����x��+�
��3%Q!~?W�q����M�
>
K� ��V�Sp���x&���4��� [!�"@h5�E@ � ��B ��F��6�1J�gl���H���U�D}��\�T���
��m��`���K���q�/kL����;�X�$�A:q�Ct5o��,% �h�^�pvAJ�4������&��������.)�<���W��;�dE�Y�s�,����� \� ��K�E6 @ � ��r$6*AU-���vfZ�f�^%���'R�ug��M�z���f_?�E3Sbu$O��vG3��X_-��E��%��<2���LS7d?��Z���������f�����\�V�������J��
q�����������t���i�J���7���o��}>P�c��2��`Y�kd�*�l�K�?��.m�! @ xh�V��0�v�@ x����z����YL��Xl�w���(�=1�J������t8l$s<�H$�_e�b24�$%v;�W������N�b��|"�������*94�~��������06�������1��V����h,�N��j���&@���I),]�IY2�)*�s�W����^�f����Je�>���/D�)�mY���_���.n
@�� `!�Z���6@ � �����l��d�N�L���g��$x�����������!����z��8s�}��i����~������8C����4�]�(�]-�I����c���J7O�_uA��j�|=�m��iY��Sq��@���O�kx�B���_�d���f��?�W��"@ � �J���Uq�� �� ��!��hM~A0��< 6�w��r�5�(��P���"�����{G��������p���)�[>|2�����jVT����������j�~�Vj��H�&�����Jx���&]~^S��������{Qi$��F�����v���R�D7i @ xi�V_��q�1B � ������z�>�@;�����p�:�I��skc�_����9'b�-��@�0*��w��q�e�[�z�O}�=� @ =@Z ��
�! @ ��CHn������L'b��rT2��O+�HLv/�z=������K6�c�>z�o�j2[��n���'���h[��Z
"����G3���"�=��8��s��k����P@ ��m Z�-�C � ^� ~B &����`0���T��rk���a������6kR7y��[m�lw���^�������f%K��X��� @�
B�mh��e�( @ ��vf���yf�p=��[D���j���)z�R#��D����h�f?�{r���U�g���;Mt����e
�'��� �\�� �z?rC � �W&�r��<z����O��f1�L��t&���t������j&l�M��j4�����%kt�7��;�! @ �� ����"� ����h�Hw3�;�������a1��v���K�����U�@ ���"�z9In���v��/S�y=]�t,����@ ��� ��:e����@ xZz�Q'��M��/�>�w2���3�����*E�����{��h�~�0�8��Cu<V�s���>g\�@ wF s �3B�=E @ �'��_}mO�|����l?�i�3[��6;�h-yr?��4�7g�����$�������:������S�y���Q�|��5��{�h���b��������j��i���t����\�%��� �;B��^B�@ x�x����3c�v�s�������x�Tl:*��cAv!�����P���'����]��l�?ax��}���a������*�p�����f���%)k|��#��������m���>�
��l��_�����d+�Ul��Og\sU�� @�5 Z}�rq�q� �R&�]\�����L��xE3�������b����"�D��Kl�������0M��lf7g4��t���0�`�Hm�'v*�N��J"��r4�R��@���f&�Z'�`{�w#:�jr�v�!a�;�N]��=����j�.��6.�x<�U�j�&������k��T� p�0�&@h����@ � nM`�9�l�.�.�S3-����c��\��������*
hB�*�g��T������/Fn�<���"���[_�S�����@{��H)�ex������Et�|��������Si`UR||���;V�j��2�����o��M"��a�j�&����@ ��� ��pE��� �2��>�����N�y#s_wi8����j����&�����x<J���*f�b=#�X��C�x�Q��d��7*�.*>�d��ioLN�eW(\�y)�nc8�*�O���|�������2�7������X�*,���)��M�������>���
������QT@6A �>B�e�b0h�f�p=�/�(|��E�|�1��(qu���������V= v��R�:������t:�s��F^o7�M��d�9N�t8l�a�#�4����9�'#����%V�GZK��o�lmL���� ���@�����*^_�$mf�|u��T,���2�7*�mr(-"r��G�{�l��"��kI n����^�|?�3}��`|�^��
(�������.�� z����J5�k�T������M��k���zzt��U�;�����6�L������jcG�GI���m�:;fz�UG��j�����+z��@ xJ��Qm���%����0���a
���m�w�y�F�=�\�2j��`�#�J��5�H������jT IDATm=����
�l��� z ���6��e:�:���?q]*��!��H��� �
���^�9����F �g 0�bFe��B�RG��K���a���G. �'&�_�g��/�'�� \��z?e�=�}i�>���V�S�����j��~+��+q�|�_Q����q@ �A`��OL��xKw��[B7 ��$����
�Zdu��oL�����g�Q�����Pa�����==�� ���%pM�};�u^�>|WM�����5��hA��:���/��k��z��K_��j��& U���*�5:2@ � 4!pOinY j<^;��zs�����&��x�����7������(��`6A��������;���e�����
���e���z�����1����|��t\,WZ �z��j53���! @�� ` <15�e����Y�5��B���x��^�}����N��
�\����-J�lw<�/�[6cxfZ�������X�DG�9�������r0P/���^H����Yi�m��wN��sG�"��W�2���(m�7�eLoT���b�T
��5jo�5�����/��&������+����8���dG1�"�Q�dTLR��A W����������
2'��s_uT�kd��ig�e*�""��"y��zM�R
D��^�V��*^a���^^x��I�n
R���.���/I`�9��p���_�]9�����
"�QM,��t?���L��W�L*W�])���xn��~;���J�|�����
O�3M�T�l�
��&1&������b�k�|��g�6s2��8�p�MlS��8)8���q�/mG��(��D�+��������D������1K$���j��N����g�6=?�@��l}t���i��D�lWr���������OE�V���uN�I\��2L�8d/x��MG������.:��?��+���zYe�K3�Zf^�����u_��B������9��|l_��d6$����I����%�����ldk���������7'2\�,cE,4"�n�S6�s�Q�v}�w�"�`��+}��>U#�W�J�U'A��d����j3n�<�d��|F;���v@�g�+.}������QZ�\�#n� �t�3]��\�k�5�[7� U�K=lK��Y"jE��i����h*���o���Z��V;����.��6 ����U��]�D�nx~���$=���7VS�Om�5?�/^���XB��Q*��nz3g��H ��e*���ax$�
[�W�i����f��IR��B��"-z�������C�j��f�����v:J�\����K�U�+�Xq%����i=e���c����e��8�������4�.�.� ������*���'O&�g���A[J���y
J<6/\S�!�Uy��D�;��x����QE��������:��b�� ����v3��5wE��Su`���������3TT
�[EH�L�cIE���Ffo
��=O��U��+���U�9+}�I�R����kU)�:{��2U�>���m%;IV1�! <;'�U��>�'"90�q��|0��Tn��P����GZ���(� y�'�!K��������bs��b1�$����7m��x&2
{����M�U��E�#@��H�Ae]�'>��)�G������4��f�%�:Y�d�\Jz
6��)���XZ���{�~>M^�po�U�����q6��#)�iw�@h�;��9�����|�\)-:f���p�,&C3M���4����-^����m9J�{��_+����'f��Y�_��2�����R�{�z)�Q'����)����4
e�- o��f�H��mR��k<�l����%2��~b��(�nsk���Q�}L������*2���\�@�W���"�w����m�����ZGu�ZVCU
L&�?F�?~6kd>lpnXb�*����l%t�5���qU�)���}���mE��������S�5��[s]W%d���N���?S������>27RZ�zE��t:����Y��&��k���K!��Z�9fH#t:$�q8�v�|}6A-l��������^�Q]�\��o���i�T��g�:U�����
6�s93��l9�B�Q��{
:��lH�d��]�:S�z���'��������^��89�7��ooK��0�A���f�"@h�������M;Q���
!�������K��������<�Q����~np��EJ��+q�EJKk2;[c���a��n�IHU����g��R�\}�P��9���5����;��[�V�w� wmY�U36;n�p���]�Qe�{i�^�Z����f�n0/}�Q���K�w^G��Lw�s��:����Fz�Fc���k\)��r�����wz���is�� �&&M\a*����Z���*������f6��,��yQ��g�D��
�6?�g)v�����D�����]�H?��!�Hl�1^�O�����-�w��bxr5y3g\zS�3��f��6/i��@���0&���*I��Z��~�V����\_r�$:Ozv ��K���<?���)*���Get
Z�������\�W��e u@�b"�a�����j�V �Z�������R���\��[�T�e���;����I;LN��6�U�������?�z���?U�����T����X��c��%$i��*g>\�P��~jZ�F���V�h/�V��������T��]Bm������6?�!�r�����I��������+�E����Jv��������;C�s1�r����\��uFz^|]��d���J*�x���+������� ,�O�I���\��|��������&M���e�T�r9.����^�,���vi�3N��9���q���~cS�<� ��:E�M�He+n+��h4��E,�wg�9$��S�� %$U��/_��Q������k�C�������)l=����B����%u�����o��),���8j��q;+����g[���}���4���H�a__���R�o��X��xXT����\� /��#���
u���H,�Uw����i�������*+�����������R"n9���$�E�@ ���j���q�OV��?q-���`<�W���h��f� it��(��5��)����]�M
O�4�#���Ve}�:���^�BW ������������]����tz����~$Z�V-��������o���t1F�S4�Q��8�����>�a. ����no���=��d��!u�G;���R�:9�N�g�]w���r�d�����xF�e���u<�A�A�3�{S�^�b�Y�UX�����sQ�����UD##�Ss7F��PSM�}�1Po���\���Y�����i�kSjt��|��#��~ �s�DhlMm����+f�$��Zr���3+Q���|�O|��g�������#���j{6���P:��|iM%��j"�4�O -p�����o�i�+�/7JM�� �z��Pk����{�1�j����6E�� ����j��f����|���2~�'5���*8 Qxu�j��6H�s�
rC�Z�Y�UX�����/�^�����xWM�Uv��+.M6��X6�*z��\��Gc��3Z�� �G#p�p��&��.����8���#`�^,��%%��E'��~[U����l����h &: l��6��t|�4�ku��%O��� ��\N�F��T�/�T��
�Nj���z���������/���� ���@���2����������2���3~P#:��e��Y�d����s�9>s��%�y����tk���?[�v�c4��Q�����"�n�
�c'~�*b(����x��f~*������yJ�i�{��~��j�t�@ x��e�A�u����RJ�u=Eo��%���B|�������+Wk�����z������(�=�\�O9��;�����r�,�s'����������<�����y����_��Fb�z(Y�.~F'������MDv��� �^�s�9�~����(Z �pU:��n{1Q����}���0W��]_c!��V�,��<z~�_����y��0���IOE��w3�&����S3rG���_c\_���+U�����z�K�Y IDAT�Y�[����+���"yQJzia�U��!�@��������V��U2�����C-���ds[cqA���:�U�E`�����k�s!��H������#r��Q9������Po&�N��I7�o_��^����
��8�����cM�'��������/}�@ofv�V���P�D
�����0���P� ��@!+?QN;���@�&�=J����A#�5���ob�b�(����v�(����%�q�(������f����[���miPvE._��i/�����%��I��2�]A2VA xQW�e��+�B]%;�O^UY�!K{��r��?����F�� ���Cd������"%]��q����k�3I�J{�?�Y������!���O�7�v{U�e����Y���������>����Kk��L�l��}������=� ������{��n���N�J��
N��0:U�������������4��E��>�]_���{Y��NmKd��$+��O@��^Q�w'����]���M�C�]#�����6$C���FYu�
��c�k��0����gD������h��-Wo���L�q������^r�qu���X#$:�T����O�Z�Q�ha�=C��yqD��|/D�af <�O�����s����0,�lU����ldm�X���inhY+����-D���L�V�xn�dl��r_�s�F�{������|`�y���� �C����#o��������d-���0n����M{�Z)�Rc���uv �x@��$e�"��TR��68����*�U�}I���4�+��2�����:*��y;C�*"F�2�~��H/~�G?�>�?�r�y����= ��^���V����W
�Z\3=\�!��V:d��[���O���i�*[���M��K�dM�_���e]�OJx����88��aO ��,t����p=��J�����n������s�����3��#�c(���\"�$,������8�?�3�$�0��w��Bt���;�D��3B�7/�$�Zf�sok6z�%�y�A������~@#��E
����(�*Z��dC������C_��4����L��N-5���TC4��u:=I��������/H���tqU����.�\hc��?��{�Y�,[�W������/�vg���O�(�w�*5
���+bVk������zO���G/P�����im|������f � ��������UMH��D
��J�v^��,�\����r4�;^�������K�P%��\��X������x<���*9���N��J/J�;��6��ca� ����4�UJ�|���d_k~�,��E����;�`f�H@v������P�_���-�����h��N����OM���E��S%F*����b��u^�5_5��Yb~�V���RK�"D>q~�N�=������B��H/��)}�-�c�2�=���
>�l���a���VZ��I[��ZD`|'��#�k��H�>���lfb�����Zc��^WgH�����d�����(��~&�rV�l�E^Q��(��(�\H�
m=Y��� d�
v�&����~UU���ag�^E�sF{]��*����kE���9{oKo����*}^� �*S��j���[>�Y��
� �J nxDk��f��R��(X����a�~l�4}r�!K"�����.~ab�T9��n�;�8%}���Z�R�H�3����~zl��$��t��gzk~�s[5)�<���'�q�l��V`��k�nQ�+�n�g��SD����kU���������[M����b�� ��F���7�9m�K-Z���N�gUj�IQ�H{(WD�G5�:�<����io�"��9���r;�����0iY��H�'�\"F�igU.��4/��g����c���R��or���K��A�����\��n���[@��k�X�+�8�q��)]�SU���*���V�nI�>*C��~I g�A �{�Y�b��:��*(s������H�@�TW�mS��*�Q��G�@����2Yu��VyR5�
Z ���L�D����AU����iiE�����������d����weU�H���:d��_�����*�j�JK8�u`6\x^NI�~~�d^c��+��Eg�vS����,���������,�'}���L��E��/-����<e����p��#d�=�Bir�w�rj��E��B���<GRi���F�p��R����;YWU�S
^q�4�n���
%]}�a����1*����d����/gT���Q��W����*R�^��9dH<��,�@���c_�@�J$����R��z�Z�a��2G�@�^a�����454�P3��#�r�:��*������jU�t�"4�t�bM��QF��F�3]��W������v��+m(uff� ^�@�v���!#������<�������:d�����x0�n.��9�1�v����w�@�P�mql����P9���t\H�jZ �#�*cw��W�q�F\�>F�k:[:H%�����dc�5�Be>�&���#�������.6���N�:[W�� 9�+��T��W
c(r����i%&-���Z� d��{�"���z�IeV~�9���A�D�C�{'��V�m/�,���+2*�(��M���yJ36'K!�2+a�i���I�,�
H<��kE7�.K��@�J$�����c$;�QoRFd��A��"[����0J�_�@n�0��nV�9��#��W.[G9��� :<���Xh>v���]ayn���/j�����L�
���+�Vb��?��RI�:-��#P���p�+����\������8CjT7�^fc5�S4NaS1��=��ck���sS���&V��F(����|m]������y�O���x�,8�r^�lM^1!������4�H���X�vT�p�MMm����i��R3�;���SX��R��D�"]T";���id���.�d�8}q-:Wr���*QXED��.�J5�Me��$�;{B���H����������}N��d������S����:/�'���f������ @ x~��.��a��-P�;MG��-�s>����M�Vot'�����_{R��`��@ xV�V��d�� �� >@ �5�p=M�q ��gfJ�F�S��u1��Ol��F�����{w�_�����JD���TV�� p���o�<�e@ � �l��KVUo�8�����S����/����\f���S����<��0�4��3F���Q[�����|!�h��Z�H�@ � 8��{�����/�.�P���}�d��?XR@ dL�@
W
?w��;�g�%�������H�� ���>a�� <*�� p_��$g����Hn�|E�}��5�7\|�E��8"��W�E�t<�V3_&W����V����)9~���� �G$@h�K�l�lN��vt��40�����
"Y2�������}�V?e;_�Cf� ��9\;���.�����F�\������*� �f���5���J�,�H��[jU�����M��7��>�{��G���c�|YI�O�_���R�^�}��H�L��{1)�*q�>�B���� �Z��
� @ �#�����r�<Z��t4�7{�<��u�d�x�7����4������&oY�3QR����B�J���9���
5�TU~u�_����,$�g?�/��� @ .B�.�A � �G M�@)���/��g�����^������f4�����AB}6�����s=���j,y�����E~�-�����@�O��Y:�,G����R6�WZ�w�o 9��b����;D��1��� @��Z}��>�W2C � �kH�<IS8��bs8w3/�Jh����Wc���&�D�~nC��l�@��e��b����v���U���d=��C�U9����_i��S��|������ a{���<�\���� � C � Zm���� @ � ��w���������M���E���TD�� 8* IDAT&Mb���fbW�?"[��o�{}��Lq��W7�@��T�6��}wiVh�O����������6"� �� ���G5 ��� �H��T|d*�X&� &���~�W�o����0y�!����>���������6��~�ijf}j��� x �k�@ � � ���.��Y�d@ � p#����>dn=����>-���Yh8Ob���)O���r=�����xVsw'�Y� �{'�}������� @ �HnZ�8�Wx#������>"���G����17��-E4 @ �'�V��4�� �� �@ �E��������V�"����t\u�#���_@ � ^� ��W*�
_�@ � �z�yj��!�2�z�� ��@9 �'B�}�D @ ���h4�@c8�Jk����p?L��M`��U�hb9i @ ���Z�zC ��s�@�'�Lo������j��9��a �[��?��6���1�%3� @ �E���c�W_�"� �-���H����n���2�sz�>�t�WZ����j�*��4��+�p����}�:@���^/�! @ � �Hn.��',���Q���U�����z{�|����,��\ ��.��D&g_&Y���[JK��;
�?�m���U-5J�05����u| @ �� ��@���� pG0����*��~[g��*�gK�`�p�}f�m��\�TdG�g���U��N��}���7��Wo-"������nd������������vU;�*� Tae @ �I���K�;NB � �G"�D@��h<���zZ����������^;�4����� ���[�F�l�)��*��v�Z5���W�5��yki�}��%#h�s�2P`���:��I�k�Q�U#�<���{^3e��)�,B �����.B����6@ � �&���s���r���6<�;�}l �,�����l,y:��U�*
�J��o?Uv�*Y�>��X�S%�~[K.~F`4A��*.
�q���.���������^��`[?S�2�|Y
@ � � ��/@�� ��@ pm���x� `*�%�y��J���p:�V3?��"~�;���Q���e-D�^ki
�k���i0ecd��������w�?#��� @��Z}���'@ hF \�z���L � ������M�zu�P��fe&�?� �����?D�b��7p���*k������b�fb�x1�o�1�,���M��Q&���<���WVA�����/af�#e4��G�%��1B�7. �C � @�5�% p/&_���s/�����c ������ p]h� @ �K`����|�?��x@���^����� @ � ������x���ob����:%�! �{#@h��J{ @ � ���gNl����{�@�
xU�V_���� RX� @ �E [
~� _�{�� �B�}���4A �@[�~=�v���}��C������8�]e������TdI�{���8���Rg���I���y�5��
QF-�:'��Q$�X����lNz:,�=��WZ!�'�S��AmoB ��ais��7�Q����oo��efj�0��y�p�� ��z<��bi�Vcw��9��� �z�E��� p�M�� ���v9��Q�hs�W��tv�zL�������/�����}�Ug��4�c��1OI�(��U�� ���L�t'R� /�M�t
`H�A�d��(](W�G�m���NKmS-���t�>IH�����Y����e @�9B��Y�� ��~>ZF�D���i5�}9#�����x�����t���l���7�l_�Pzw���Q�!��W�U�0���G��X&n�Y���`�����@ xL*dw'���������?�����
T+]�Y����M�n������l������3�y�����@ M ZmJ�f�P@ � ��&.������YL�������g1Z�N��v���p�H�amlu����Y5���=���������� �+{BU'�A3" @ �M`��D�j��J)�(U3$�����{6������������m��B�.�������S���]q���ozH O���l��
�x<�V���� �O���������&,V����a1I?�?\|7O��#����yu������q������1B �Z�/B �N�V���X+�(�]���Z;3P�[b�qwEgv�;}�� ���E���B�Bls�z��7��o[
����@�V��"- <��?��^�
�������2�
b��4�� �z^d�X�h<�3��
.k��3*�� @ i�@?f]�=�lj;&�����{�V�F]Y�OF~rT�U?���Dd���M�mU�e@��V��w��� ����B�>��B����V�hnI0�7��C��3E" <��i7���L�B2w���������+��8�d��fuys] ��~Z����yI�U����B O@���".@ �@��\%�R�����<'s�j��W���No��xP8���B��@�n`^�@|��B����;�V�O7�[��!@�� Z}���9@ /I�N�V"���� \����:v_Z���?� �B��`�.��� ��
b����_g�����8VC � ��#��� � �g$@h�K� @��r�����p��N��X\�d4@ hL���@����Rv[��|w��n&�� jZ��f@ �we��#��|����V��Kj��(%�� <(�����j��+� ��_�-�F�G��yK�:�uBf� �� ��V��N� @ ���d��� ��[�����DV�� �'!_�����������z���_����[
,s���Y��+����YhK����K ������ ^�@|�����/���M�M�V�J���3��DVO����7\��a��j�B hL���@��J�����N�)��z<=����6��/cF�|;��5����Z �U� PC��j
6C ���haN����7�lCd�����B�7����V�/{�j|n�������"�B� @ %��g�����L��x<������_�7
���x�VY�<����U�[K^�wP�u�lmpc���@ B���� � nJ`��>3$'*���6�g35I�����V��r�{V��H�>m������
F�m��f�MI�� �^ ��Oi����L�m��?�3�
?�c0"M}�@+�����W?C�P���b�R����eAY��U������&B��Y�x
@�UL6��l���D�B�lwx�x.�$�)�.Y���.��:m��S:���y�x�i���xx
�!0\v�Wc�>�4����M+
��b
:�?[5���.���<������i����� <!B�OX��@�����:[9� Q{��<�x<l*�k�7[��)��c'�]~������A � PD`�9U3o��7!R���U$-�����NuZj��"@]�=��.��V���:7�2����@[�V��J�6@ �K���N�t���P�#%\iVf�����;D��v��6U7��iW�^,!��7�O�
��8cOm^~ @ �D@7��Y=�
zj[�=����myv�aqZz�e11��|��5N�'��(���~����X�,d ���&@hUc`@ �F`�n_01�^~SH7��� ��@ ( ���j��w� IDAT VA hB��jJ�� �kx0=���~��qF�`E��� �'��m;1��~�@�@�B����L�! @�5 ��^M��xdoY�Wo����1A � �k ���2�! �Z���� �R�����[;@����` ��� \�
! �HEV}�{L� : ��! @�|/��g�c���@ � ���������=��c�� p!�V��%% @@�lN�t8l&��h*� @ �&�TD�%�6C���};�xga���@C�V�" @ � �� 6A hI`8��i�&&�R�! � B�/P��@�n` @ � @�iZ--J6@ � @ � ����@W*��L� p;R��N9�! �k�0�������B���YJ? pUTW��2��CR�V�� � nG@���)��f�C � ����pP388�4#@���� p%rH������@ @ � @ ����C@ � ��JA � @�o�� @ �_�V��l�����^@ � @ ��x����1�B � @ � ^� �C �B��c�d@ � �#@j@ � �"@h��
S! ��� @ � �2�W ��r�; @ � @�U�' �+ �zE��� @ �@� �� @ �L���#��C �&tA � @ � �"������,B � @ � ��p��- Z�%}tC � �|� @ x*�V��8q�@�UR�������FF��@ � @ �<Gh����� �K��f��[� �c�o���P���t:
Z�le�����3}�?{��%�$������w�$���}v'Jr�@ ��#�f@ wE���]�@ �@����n����F��H��I~��|<M�� �v����.�������N�OM � @ � �� ���._�� ���C��|Y��P�}1T7���R�]���j�=�a���5�R��6����L��U�`;������|! @ � ��$�����D��� p6���p�����-�r����e����
�����t:6f:��r4�x1i��0.[�@ � p�0��} �z���u� ��?�f�����doe.���,����k�1�@;�� @ x1�V_��q��!��@ ��t���[��x|�g(
|� @ ��Kx���KNC ���~����s��~=����`<����/�
�6Q�1kW$�v4���
��#ee�x��%����Xp.����a&W�b=�,G����Fzy#K�+#�3��Oe����X�0B�fY���4�l����Lc��_���?��B~>�h]
��[Y@ ��}�@ F�����B ���~>M�� ���v9�,�d��2���%X<r��<��HB��@��E[��$���t$�z4�(T�Y}��r �\��|��g����+����A�r[�n��h]��-C3���th�Z���kWK��� @ � ���������kH�����m�����L�Yt[�r�^%9�/����F�2������G,�F�\�4I����p�MY&�T�c4�,����~���E2Sw����&Ex�#��tfhJ@�U����N�����$�,�eb��-�����%��4>���`A�@ � @�7wZ��7A �@��v������YL����+����� [����R����~^/��O���f��kJS�"���L�����g�g;�cM��H��O|������1���g�F�����4��#'b��[YQ��l�
���d��.mjJ�Y�Z�C�-��v���~nPFg�"3 @ m ���� ���%��� ���j�����g\���C������l��d�'o��l�1�>���z�f�����hs����Fz��&��B��G��f�F����EMYS?|��5��U�4f���%�{���*hJ�t� @ � ���_@ �@���Yq�)%z��c����h�����ye#r'�����n��$�|����*�:�����p���d���8�^���>V��(!%do ���m�~���1A � @ 8���C��XO^@ ��@-�)�XkYo��4?}�j�U�F������Mm�V�I�:�������{K��������qS��� @��� �t�>]�� �t$DT����
K� ���P��_��/}��l���a��i������Z����FF�������O
>a/w�6�q��rL$q�cn �B���������s@ � � �ZG��� ��@?�}�dk3�����p?�%�6��2-�[;f���,�*��Cj�y?�t��e�\��H���l�,���}�C � @ P�ZUJ�B �@eo#���J�M`���! 1��h�5AT���v��1���Je���� H�<������[�*��=�jTg @� �3 ��'@h���! �G'P�6����,Z���F��7�0��5:���Inw<��d2F����?�0q�o��fq��k�1��)��v
������8�M���}`�@ � ����FF@ �&�~�@�'��|��[�_=3n��f�%���l���V�EQ���Vb�3q����H�����x��'�� @ � �I����k��.@ 8�@���\��s7��e�Mc��?�7A��L�{f�F�&��>�3�l����k�NRUkTq������ �l>� �B !� ^� ��,t\� ^�����@�rC�5w[���F�"�v]K�6W������N,����z�C������`���qN'��#Qk����Qp�������oYv1��_�{�.�����U�P���J � @ � p>B��3D ���[�v:��3��p�-
�����8��-�?2��tH���vk�sE����3�~;����I@/}?n������!�y����}���}_�Kd�����w��E��j�We��N�@ � @��z�^�X�C ���������x<_��������
�z���^���$�z>K&���lP��������E�>��TM6;k�v:�T6�]�9��O��9T�94�^r�1;�
��9#E
l��F*�����4��~���w�Y������������������)sq�"�
@ x8
@ �Gh�� ������s7�� �.�zZn���Z�z����y����$�r�
$�?�7o��e ?�ZiLn�����E�g�M6G�E��\b����3s3��t�U�yd����c_�����-"D��(�����B�)�Z��=���v��}�-�k��`[?S�2�}�@ � � �z��� p���N6��n5�M\Q��l�;����l���"��������4|r=�������i���3����������Y)�/
B9�<[�XIg,HS�Jy���< ��\7\�v������eo�!TC � @ /N�����������W�����V���~U� ?����5��,O\�
���,`�nC`8Yl������:J��?I����U%�h�do��l�z��J����N$gd�=:� �T\3���{����_����������������HQ���� L+I��S�i;-�P
�*AnzqOB��"�9W"�Y �{K�I���'����g�C n��WrH���;r�����u�y����v��x�� IDAT���;�(���9z�(��G�s �ff.-�na�L������.2���V�)>����f@�'��� ��K������dFr@ � ^� .C C@��p0����M�g4�NQa�J���UK���������,)��o�]NG��t����t�����3�^yNh��J���c��B� �@ �@�@|oF�����@9��F���6������l�� d�<�����}������^m�����^������[pk���f��B������*�zkc�/���^~�@ � @
����NU�;�QX6fR���z�<��&I^��k�JE�Z�����O]��,�������P��fB��/��l�<����! @ xf�m?�#r�?��5�<�@���5����Mn`���������`80oN���9�;OIEq�t��e���g.���8�f����m�E�j`( ���f�V��<���W+��#�Z��m�s�@ �@`���`9"��%Vj��_`���/����d>�b;���������c�uPQ<MYSQ<MQ^���ng�x�;��x�����������L`�_y-��x_����}Y�5��y]S?'���Y7��a������a������p��^�5*�HGt��Z6��F/�e�F����!��j����r�����}��������aT8���a��l���C��� ����[����a���J���V_�����9uG!����wL���Kh��,g�3���Ef��'�%����_cZ"S-u2Fe��N6G������)\�[���m�uv�}Q+�7fJ�s��4����L����"���H�%�V]��4���5���#����>�[������_�;8
K*�f%�D�q8:�#���f���]�QQ�Y�� $w;��
�E��R�G��MG��Vr*g������H�9�$��i0(�D�W�&����#5�r���0�.��~�����/�(]��^
����`����I���~�z����j,:m+���� &�{�b��(g��9���5Cr�6�|���=(* �#..����z^'b@ �F \K��ibl#�~>^��_Z'�3��.7�K��G��GZb���j�\�.��i��w2&���wM��Q��j��v�
���*����@�~�^�J+�7�J�����TZ9�m~i�"����I�SQ���:*ESpTMI��A�g�����G�|�N��J�U��f�V���6N����m�!�����g���L���#����|��]��O��?.!i��W�,�5����5��2k�'k�)�r"%����\��Q4����h�NF5�A��%ODI;%��;j�
�qm�d�Q�c���\YR�0/*."���g(� p�B�K�:i�VQ3l+�������flJ�1��~�R�L���,�R��09d.g��T�����w��Ik�jy����Q5��U>*�^#�7���N���v41�/���@Jl�����2�����du��� ,S�D��L�H]/���OZ�d[O�X`�Y�W����G���N�HT����E!�?"�L�Q���V����O.��� �_q���p��Th�>
��,������7��Y ��9M�W��a���4Yl�i7/=�@��=�����|_����^�������N�����Q�M�*���?)��Pc�46�$B�y�]�x%+fZi�6c�;Jf)eH����u��IID�%�-.�����@�E��P���G�=��Y�`+�_�������y��J�-[j�v<�l�Zz����Dz ��M7yC�N&�?�����f����)����Hh����p4�����O�$O
/i����%h��f��q^d�: �*�y�>�������?���J&vcJ�u@�����2����|���~��������(�#QQD;F����(���?�?�MhR��O�@����-�+-�Z���HEQ�m� �� u����`����S������ sB�RPi3����=��{_x�c���tG�R�����~7���lo�9�S������^V�6��Y#��7�IZ.]���z*���������U
��u��q���i�2:Y �#a����l��F����lt+�{�z$�y_B��3=��r�����&O�qg�q���M�{l+$��^�K���"t?A"��$����@���u�z�&�Vw��r�}{��Z���:{r{��]�+KE��L�(���z�V)����1u��TE�d������0�e��SfM����w�����-�V��LX����WC;:jY|->�IE1�#���bE���g[-��&�Z������c�=h�Tx4��1X����������U��g
Y�$w���� psu-���W=Y
T>Eb�Px�7�����7�,6z�F����4��L�M������F�XQ/����a��M���y�z��-�����S�G��`VSx���a��WI����U}jL������N�AT�#QQ����"E�����a���b�O�/[QT�e�����E�U�`9MV�����^r�K�H#�j8<�<�>�F�l� '����8`gE}�K26����vO%8�k�n�<��������
G-9�
�����n�(�N����% @ x2
Z�.�b=��+�%�(j�������-��C�i �s�EG�5��_��
UdV����Q��$����D�H]���RQ�)�}U������l�rE��k��jed5���QgR3�B�7����Tp��:�����(���Q�dbu���Td��iFh� ��[�v�$p�=��i��*���
����lIxn����G ��-9! <�F-T{�b���E����^����o&��y�k�1F����N+��-� �!� ���b�� q�6�$_����{K��K���������i�#���
Z�tdQQd���DUDE��kWM�r�������>o���l������ ��m��W�����j���Q]�<�g���� r�{��v��Z�L�u������D��U"����MA~@ hB �����` x�2-�[;A��L3���Roh��z��[m�lw���^��K���#���&�7����v�Wm��i�(nA����Z��J�W#E�����Oec)Q���8�c�\� n�S���j ��X���4�:������N�2~�*�%l�.� 2J���E�����V�Vmi���)P� \�@�������`4�� ���V���M�M��nmLk
&�~>0���+���5k2�O�8����.�����M*���y���8������W���5�<5`��P;`P�Q��KpU �������L3��W 4�s5
�6�yQ�4z���I-`k���8��TOI�����B���X�z����*����@ �Z�%��$��;���b2�����Z� ZS!�b��^�qU�tE�G��oG�_iM�6��8�����^n�&{I����������48�Z����&�h��h�4�}�7"�<��I�FC'�u
BI���!�jA�����+�v%G[�,��
�z^�����(����� =��
�9���������J�6�U���G�lQ���Z��P=R��f@ �O�Q����bm�AO���u�gGc�{������ �U�(��,l$�������Fy��xd���?��6���7����v���;���H���=���
�����ZEE�(�mB �-J[[��9���Yz�1�<����~��!������.�����a�����rJ�+����������U/�R���v{�Z�������/sO�uB�����H-�Z�#�e��O�y����4@ x
Z�.nt7�n���~'O'c �����������*������{�{b����EG(�S����r=��j�qqJ�L��;bn�j���T��l['��KEQ��~}&E����SQ��QQ�i�\G �������58�_��z&�7��������|�e+��{)�s�H8�.u�?/,��Rvo�~>0�|�����
��tf�Ht��}T�f���I���f�\�
�?�:��K�; IDAT���"I��^Ro�nK�j��h4u1���Rc �W P�BE�~q��Qc��\_BD&�v�O��O��Uv)n����<c\Ye��<��e�wy�Ql�j��T�����U�j}o���#�2����\�O���������������
���������(:E#L���B �yB�)x�Z����0�+��H����z�xM'��MI�2��[��Ow\-��\N�N�g<I'�
bjiQl��}i.Nu��v�@��S1��;n�#i���(2��x�����kB���*���X ��A��G��_�*��^�.� �$����r����X�%@@ I����/A~���� ���tG���j�h��b���.����/�������:��(>�*_�u�M�����Q���P�������S�/3����*���,EV�2E���TPX�r�O/�NG��5=�(��_�lx����S�������K�s~B����Y���^jW~�N��z�i�N���� ��&��@���������>�R����l���g����ILDwx��d@�<uMHr����|<�jk6�#$��n��XR��]�b~<��7�����- �S=qZ�0cY�&�Y���/ �W"�m�T�!
����V�]���+���L(�Q���} �I0MW�>TTST������a�6��x-��pb�V����$j�+�T�e�]W��T�V�9�����U�Q�[u����ykiQo�:9W����������)s�����X����M��qO�w�h����j����ZQd����<i3��T{��.���0u�'a�Z��UZC�U���Iv�V/��RsW2�6L6�������L��~9�t6W%OG%`�Y9�O~d{f� YS����S(��j�L�FD&���\��x�U�_�������j�H��7]���J�������_4���O��%�(S$�Xsj.L2�|���\
����l���g5���� ^���B�6C5Rq+�]�w��b��%���i�U,��=F���V���R^"�e� ��m��S��"[���L��r������3��h�wR��������������z)E�������������k����p������hTW-k)*���/*����(;��ds�c-���q��$�Gp��]l!�z������D�P���j������g���T~��
�GN��)���<m�cW�Qre(��d�UM������(�9���\Y~����b���g\U+���k�^�������0�^y�! <�B�-�j���:�b��Sr����Mw=�Vr�
�P�n��s���q��u�����y�_9W��������b����{J���Z���hD�����w+�K[��5�2��e��
�=��(2�.�WTx*�0��K�6=�vYz.I��jT�;I��f���+�0u�������Y��2Z����$M6'=�G>c�����2+��
����p�4��p�,���Dp�u�c�\���!&CSS��d)dX�����Pj}F�� �T���(��$�����+��ca�0�#�?xv�z�A*|���;��@ ����4� ����6'VZ����:�H��#2�Z��I
7ca���{�����w�������*^_�$m��+��]M$�U���bYE�E��|�Js��]���b�~N���d����(��i��j���6�N���������+ePYQ��(��+�&��KT�'�Z&9��>�I������J�w�*�����E7�H�r1TKb�7B�}EnK ~�p~����v@ x�@ � �W"@h��J_��@���hP�������� @ � @�� \�CB���h\�@�~�������b�6��r�@ � @ �@ <B��TZ�
�b����R�s���+��X@ �@ @ � ^� ���.~�>���%�^?���<"5 @ � @ �O��Z�&mt���t����<�N����%��N ������N��|��|�6�|����*�x����&�9��s���Z�|q#3�����I���y�5b�^�!��e[�$y=�DK����:�t�����l�g������J�/RX@� �jA�@ OJ@�%G��60C��A�]NG�lR�}��\g7�F���f��6�i�&���/����y��n���k��Y2=���T&��aX������ @ H ����2 �<����E���<���vzZ�|�����?��t�����t�7��l���7�lc���?����?��Y����{���%B���
�A(X����U��@ � @����\B�7��r@ �(�p���E=u<6��������y�V��,�)��������������Q�!�jr�UY�f�v����8r��
�P8��-��q�6*H@ �@-@ ��s ��\��7� �"��w����������&,���%&{XL�����3��0yNdU�����^>J��q��bW����-y.x@ � T �Z�������N��k���@:\����4������$�|)���Y-V'����g��Z����oU�c@ � @�� ��y�V��<�� �#
z�}��������(�[��A��D����R
�@ x�@ ON����0�A x]Q���?���P[
��
��Td5\����7G)T|�� �@ � @ �Zm�����}�
��@al� �zk;�@ � @�� ����V���0� ��@Al�8�����p���>�*@ � : # �B��V��@ ���V�#���bS8-&= ��[Hy!�
@ � �I���� �\� : �����'�-�w4����U����8���C @ � �g'�h�Z}��^@ hJ`��7I����~�nlu�{��YU�B � ^� NB ��� ���� <-����_
c���ep������"�������p���=��>?<� @ � �&@h�o��� �'��D
}z���:\
���������W[}��=��
��G���"�=�F @ � ��>9���/B� �R�����,G��|���z>������td�$�2�c�[���h �O����d=7�o6�nv��<� 4#@*@ �@� ��,�C ��3�l���} J��TO�mx�lwx�x��I�S$],��y����/G���V���8n��ae�
>��A � @ '@h���Q �`;.J`�8�������KT�x<l&}�M�V/YK��U�C��� @ ���x>��>_�� d����p���DU�:)�J�2f��:]F^���������E�^,!��7�O�
��8cOm^~ @ �M � @ ���"" @������Cj��E�\[ �� � @ � �� �z}�h����<�p�C����f_z_����
@ � ��^�}B�/P��@ ���������������zL� �G&��� ����gF@ xq���������W?��`~-�� @ � �� ��;(L��s�;<5��������� @ � �F^1��W,u|� ��L6�h:6��Y��@ ��-�� �B�=@D @ �$dC � @ �H���=�
6A�� `; @ � @ �O � �U��� @ � �� �A � .A@�V�� � nG@*��)G3 p
m�k��@���n]���C��x�b���% ��
��/ ���w>����@�� p��s�Y��o IDAT`nE���V����&@��������@���*�*?| @ � @ OB 7 @ W!@h�*�Q@ � ��� @ xL�V����"�^@ � @ � �� 6"@h�&A � @ ����.@ ��mZ�
w�B � ���� @ x�V�� q�!�T@ � @ � �� v#@h�7rA � @ ��m�� p'��IA` @ �I � @ � �g%@h�YK� �� y @ � @ x~x�B�=�D @ � \� 2! @ �J�����vA � ���� @ x�V_��qy���#��0�w���{��}� @ �@_�s)�V/E�� ^��~>0�|�
E��|�u6��6����h�Rf+�L�F��c����-E��0���&E��� ,~ @ C�����B ���_`!��/v�n��F���,���2�M.����@ � @�u Z}�����#��xx�/+_9���/�j�&�����m����0RY�r_ ��G������� @ ����Z�jA ���&��I��fr��� x#/��M$�*������z�(� prC � � ���-;,� @@�D<M�� ����nvA,�� @ � ��`�F ���7+��p�- @ ������ �z;�h� pM��@O����~=��*��������P��@I1�2f�����&�S�n.�_���/t���B�b��L�k<���\��-I�r���o��Z#�������Y����E���o��r����-�|S=����vc��FA� �� �A � �� ��'*L\� ���~>M����+����4`))����05�"��i����G��3��$t� �J���e3I�`;I��p������
��c�@��u�T�Lv��r�w�{�p�9���e��u�v@ � @�� Z}�����"�7hF ��>L���n���i5�M�`9�3�i����\���4���]�q;�+$���$X,�<�����"M:.�-\S,$��q���%"���y�����HfJ�n�6!�1��Kg��D_�����4I�<3(%�U,����q|����zVR�J�S�-?� @ xD�|G��Qa`
�k�n���2OO����+����� ���4.��IZc�|�����|�5&�v��p�Hi�^j����=z��e�s�v�����& ����8`94B?�yd���Y#�a�{&K�T8����J@���T�mJ�_TviS��^>Z���-��A 4$@2@ ��3 �����o� �H\5�����g����!������p�#�n&�ro8y��ec���q���[6�7���D������+�+7�!�X�"��':\|��YS?|��5�},��Vi������d�j@ � @ -Zm���'���f���2-)%z��V/�1����|��H=�.'=eB���b��""��$�|����*�:�����p����������B����zVD@I)���{�r�/ @ � � ��3B��\:�@���hYO��Z�{�x�g"����Z��J���8��������*9�VG7�z������&� @�rA � ^� ���*n�� �@����7,�&xH�����$�Kk7�+�a�eZ��v�a�V�j���������+�C�S�O�����i�M��!��Qm��4�! @ � �W @h�J�� >@����g�����X`b������`4MeZ.�v
���ycU���8����v@�����5>F1c% @ ������"@h���c! ��$P�6��,��6?�6��tk���?[�v�c4Eo�������=�j<-� PN�-� ��&@h����! �G P�6����,Z�����Yo&�a��kt��GKsg���O��f1�L����}�?���(L\�������e���J'k������@ � <!B�OX����hN������[�c�;DQj���FF6x�U+��(q�o+�$� @ � z#���&@h����! �F ������}�`���Mt���4vz�S|��K�;6K��^��������UmFO[{+���\_cc�H@���+ @ B��@ x~���"'���j���2\���~ 5�Zj4�������:}�|/tn?�i�u:g�������9���O�D�i��5���G�|�|��v��-�.F�+z/����z�H@ � @��V��G^�E 9�&��t<�g����[�[��7q�w[nd��~>�F+�t�����\��������N���j�K��[���g��CcH~^c�~>�A�����DV=�]�yW�[��Y@ � �{&�mOF����(�@ �!��f~������z���|<�#���^|��,u.������i�
���5K������$,��a&\�j��Y;������)����n�x�%��b�����c<�*K��5����T_����l���R:�Ef����{��;�=r�L?������G�R�������� ,~ @ �&@h��[! <��o?w3���r���H����X�.~������%���mx�?�7o��FH~����Jcr{�g�|/z�<�g�9*-j��#]���C�fP�4��X��LS�h��(Q���[D$�5�Q��i#��2S*�"�{*(��6���[4����v���j���e/�Wn���1� @ � �B���D� %nK`8�����7qE1��g�����=��.2�=��;�I�'�s%,[�Q�I��i+jg�/�1 �j�J|�f�8(��Y�������H:cA��U��S�Hm�I����Ze�k�����TG6j6N�I�K�-�b~_�@�or7{D���[�� p���j�^���/M ��� ��g/a�� PH`8Yl������:J��?I����U%�f�do��l�z��J����N$gd�]:� �T\3���{����_����������������HQ���� L+I��S�i;-�P
�*AnzqOB��"�9W"�Y �{���/�X�U���!�k5�t4�GV����)���
S@�/��.t{�b����%�}�������/%E��?�=��R�X� ^� ��W)i�� <�yU��u��?�j��;�@ u�vT^MQ��l� � @h�
� @ � ���3����^K
5w����]$� J�wTr�� f5 �D�B�/W�8@ � ����q�����(S�Eo��(���x4��vT^MQ�8�3@�@�Z}�R�F@ ����<=�p=��G��]��{�{0`����Up��p�?_�6��z� %�@�Z}�B��G#��� @��p=-��0�������a7S��rDtU���!��� ������3��D�B��*� �� 0���tX_�a�<� {��z�*g������s�}0��
���x���4����z_y?��bR��d���=��TT�E[�u��C�����hi#��T��o�e���z�R�V�Z����a�M��V�����k==����\�^�`�k���f�Y[�����A�|�7��:M��j��y�=
�VH��W��Q.�2[����&��M�����,��fv��{T�d�������������e5�Z��2=���~��wY
\� ��+�F
@ ����GS����X���)��
�
�4��MgJh�K�n(q�6U��
�
r����Y �9�%���������9?I��2��� IDAT�F�u��r�a�r�al���H�)�L�${({���)G+���)�C��JK��%���j������KRn����V�8W��S���b6�7���3�9C
�?��k�s�+� s�Y:U��l����{��&����8,�i�b&wfj��Qf�P�����L���b��B$
���7$��A���we��$���re�L*�������?!b��������� �I ����Q�=�� p��+= ��AFj9l}WZDLR1�,�%��tOi��2�73��jq�GF$;�~ha��0NS.���
:�E��}?��������s[L�m����|I�(��������8c* +�29(/$��(+�(7�luHm��tET��jc�u�PPL[
��+�����+��i1�����Tz2-Q�&nad��6md�d�%��z�����U�"���yGJ��
�m$�I�(��UQ��&�<��Mi��"~�,��l�������AuV����%���_q�.���� \A ����@ @ @ &G %IF<�Q����"u/���I��R��mt�gQW���Z#�#M[�2�J��L�68~*}��6V�D�xry{����
�j3d4R �������=�X���d��&E��g��:�:�$�P/�����ZWu�\4lBA�G1�) zPL'I�d���0��T���nB�����mR��Vm���A+�z��c�8�n��g����T3wH�����F�O�Q���E��T�i^����=t�l32
�?f�O�������@ �@Z���} � �mD�BU��#m�
�DL+x8|K����V�I���������KGS����S�-jC���l���Q|��A�W�H*-�XD�/�[���w<�=
_�����r���am��CRW3����i������M�c��e��v�R��6!Le���/��@�c�>��8c�EZN5Z�E�Y;4h��,^}�X��p�����|�q9w�l�t�����gn�{t�A�Oxtx�����d� ��!����� � ��J#��������i=���W��\������T/_u7�������w��N<��(��O�g����o)�}�+������_
�7m��������a��O> �������%� �c|�7X�9RV�F�m���F�/��KP��)���������;/_T�n������$[��=�aI�4�)�t/t������6��+��> @Z��&l� � � ��� �(���cQ^�9h���v�-���t�)}~,W���m��TX%�L;�0��A>���A
�7�J��`Mn����X�4���r�f�#}kKH�� ����e�V��������xB-�����`}���J�'