Separate GUC for replication origins
Hi,
We usually use max_replication_slots as the maximum number of replication
origins. It predates the logical replication infrastructure (commit
5aa2350426c). However, it is confusing (if it is the first time you are dealing
with logical replication. Why do I need to set replication slots on subscriber
if the replication slots are stored on publisher?) and it has a limitation (you
cannot have a setup where you want to restrict the number of replication slots
and have a high number of subscriptions or vice-versa). The initial commit also
mentions that it is not adequate (origin.c).
/*
* XXX: max_replication_slots is arguably the wrong thing to use, as here
* we keep the replay state of *remote* transactions. But for now it seems
* sufficient to reuse it, rather than introduce a separate GUC.
*/
and another comment suggests that we should have a separate GUC for it.
/*
* Base address into a shared memory array of replication states of size
* max_replication_slots.
*
* XXX: Should we use a separate variable to size this rather than
* max_replication_slots?
*/
The commit a8500750ca0 is an attempt to clarify that the max_replication_slots
can have different meanings depending on the node role (publisher or
subscriber).
I'm attaching a patch that adds max_replication_origins. It basically replaces
all of the points that refers to max_replication_slots on the subscriber. It
uses the same default value as max_replication_slots (10). I did nothing to
keep the backward compatibility. I mean has a special value (like -1) that
means same value as max_replication_slots. Is it worth it? (If not, it should
be mentioned in the commit message.)
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v1-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v1-0001-Separate-GUC-for-replication-origins.patch?="Download
From 5f9e43ce48b523e3bca8e1ef8ee9a427c676381c Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v1] Separate GUC for replication origins
This feature already exists but it is provided by an existing GUC:
max_replication_slots. The new GUC (max_replication_origins) defines the
maximum number of replication origins that can be tracked
simultaneously. The max_replication_slots was used for this purpose but
it is confusing (when you are learning about logical replication) and
introduces a limitation (you cannot have a small number of replication
slots and a high number of subscriptions). It uses the same default
value as max_replication_slots.
---
doc/src/sgml/config.sgml | 26 ++------
doc/src/sgml/logical-replication.sgml | 6 +-
src/backend/replication/logical/launcher.c | 4 +-
src/backend/replication/logical/origin.c | 65 +++++++++----------
src/backend/utils/misc/guc_tables.c | 12 ++++
src/backend/utils/misc/postgresql.conf.sample | 2 +
src/bin/pg_basebackup/pg_createsubscriber.c | 18 ++---
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 14 ++--
src/bin/pg_upgrade/t/004_subscription.pl | 14 ++--
src/include/replication/origin.h | 3 +
11 files changed, 83 insertions(+), 85 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index e0c8325a39..779fcddd34 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4493,13 +4493,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5202,10 +5195,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-replication-origins" xreflabel="max_replication_origins">
+ <term><varname>max_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5217,18 +5210,13 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. The default is 10. This parameter
+ can only be set at server start.
+
+ <literal>max_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 8290cd1a08..1c4b6586db 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2144,9 +2144,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2181,7 +2179,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-replication-origins"><varname>max_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index e5fdca8bbf..c4a4df1af2 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -334,7 +334,7 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index baf696d8e6..2e223942e3 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -99,6 +99,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_replication_origins = 10;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +154,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +165,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +186,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +352,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +511,13 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return size;
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +526,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +543,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +565,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +578,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +620,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +713,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +723,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +791,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +847,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +912,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +953,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1019,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1105,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1119,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1154,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1190,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1531,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8cf1afbad2..2e4f89c5ae 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3278,6 +3278,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of simultaneously defined replication origins."),
+ NULL
+ },
+ &max_replication_origins,
+ 10, 0, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index a2ac7575ca..5a689bb02e 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -383,6 +383,8 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_replication_origins = 10 # maximum number of replication origins
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index e96370a9ec..89d7133dd8 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -964,7 +964,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -983,7 +983,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -991,7 +991,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1004,14 +1004,14 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
}
max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1020,12 +1020,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 0a900edb65..01758de701 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -262,7 +262,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -280,7 +280,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 94164f0472..0d33051dbb 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1815,7 +1815,7 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
+ * Verify that the max_replication_origins configuration specified is enough for
* creating the subscriptions. This is required to create the replication
* origin for each subscription.
*/
@@ -1824,7 +1824,7 @@ check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1839,16 +1839,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_replication_origins)
+ pg_fatal("\"max_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index c59b83af9c..244084e860 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,7 +41,7 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
+# Check that pg_upgrade fails when max_replication_origins configured in the new
# cluster is less than the number of subscriptions in the old cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
@@ -52,10 +52,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_replication_origins.
command_checks_all(
[
'pg_upgrade', '--no-sync', '-d', $old_sub->data_dir,
@@ -66,14 +66,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 7189ba9e76..e104c6fc0d 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
On 10.12.24 19:41, Euler Taveira wrote:
I'm attaching a patch that adds max_replication_origins. It basically
replaces
all of the points that refers to max_replication_slots on the subscriber. It
uses the same default value as max_replication_slots (10). I did nothing to
keep the backward compatibility. I mean has a special value (like -1) that
means same value as max_replication_slots. Is it worth it? (If not, it
should
be mentioned in the commit message.)
I think some backward compatibility tweak like that would be useful.
Otherwise, the net effect of this is that most people will have to do
one more thing than before to keep their systems working. Also, all
deployment and automation scripts would have to be updated for this.
On Thu, Dec 19, 2024, at 10:31 AM, Peter Eisentraut wrote:
On 10.12.24 19:41, Euler Taveira wrote:
I'm attaching a patch that adds max_replication_origins. It basically
replaces
all of the points that refers to max_replication_slots on the subscriber. It
uses the same default value as max_replication_slots (10). I did nothing to
keep the backward compatibility. I mean has a special value (like -1) that
means same value as max_replication_slots. Is it worth it? (If not, it
should
be mentioned in the commit message.)I think some backward compatibility tweak like that would be useful.
Otherwise, the net effect of this is that most people will have to do
one more thing than before to keep their systems working. Also, all
deployment and automation scripts would have to be updated for this.
This new patch includes the special value (-1) that uses max_replication_slots
instead.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v2-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v2-0001-Separate-GUC-for-replication-origins.patch?="Download
From 8a80eb6a8aa3cbfb2c0eb09ee3799e857729e205 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v2] Separate GUC for replication origins
This feature already exists but it is provided by an existing GUC:
max_replication_slots. The new GUC (max_replication_origins) defines the
maximum number of replication origins that can be tracked
simultaneously. The max_replication_slots was used for this purpose but
it is confusing (when you are learning about logical replication) and
introduces a limitation (you cannot have a small number of replication
slots and a high number of subscriptions).
For backward compatibility, the default is -1, indicating that the value
of max_replication_slots is used instead.
---
doc/src/sgml/config.sgml | 27 ++----
doc/src/sgml/logical-replication.sgml | 6 +-
src/backend/replication/logical/launcher.c | 4 +-
src/backend/replication/logical/origin.c | 91 ++++++++++++-------
src/backend/utils/misc/guc_tables.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/bin/pg_basebackup/pg_createsubscriber.c | 18 ++--
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 14 +--
src/bin/pg_upgrade/t/004_subscription.pl | 14 +--
src/include/replication/origin.h | 3 +
11 files changed, 111 insertions(+), 85 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8683f0bdf5..1f15d97055 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4493,13 +4493,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5202,10 +5195,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-replication-origins" xreflabel="max_replication_origins">
+ <term><varname>max_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5217,18 +5210,14 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to -1, indicating
+ that the value of <xref linkend="guc-max-replication-slots"/> should be
+ used instead. This parameter can only be set at server start.
+
+ <literal>max_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 8290cd1a08..1c4b6586db 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2144,9 +2144,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2181,7 +2179,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-replication-origins"><varname>max_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a..581bdd091f 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,7 +325,7 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 1b586cb1cf..63bbcde2a6 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc_hooks.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_replication_origins = -1;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,38 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return size;
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_replication_origins == -1)
+ {
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", max_replication_slots);
+ SetConfigOption("max_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
+
+ /*
+ * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
+ * However, if the DBA explicitly set max_replication_origins = -1 in
+ * the config file, then PGC_S_DYNAMIC_DEFAULT will fail to override
+ * that and we must force the matter with PGC_S_OVERRIDE.
+ */
+ if (max_replication_origins == -1) /* failed to apply it? */
+ SetConfigOption("max_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_OVERRIDE);
+ }
+ Assert(max_replication_origins != -1);
+
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +552,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +569,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +591,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +604,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +646,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +739,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +749,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +817,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +873,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +938,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +979,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1045,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1131,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1145,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1180,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1216,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1557,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index c9d8cd796a..72f5218b1a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3278,6 +3278,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of simultaneously defined replication origins."),
+ NULL
+ },
+ &max_replication_origins,
+ -1, -1, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index b2bc43383d..785238ae76 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -383,6 +383,9 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_replication_origins = -1 # maximum number of replication origins
+ # -1 to use max_replication_slots
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index faf18ccf13..4cbeeb5021 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -964,7 +964,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -983,7 +983,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -991,7 +991,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1004,14 +1004,14 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
}
max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1020,12 +1020,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index 5426159fa5..40f3f420dd 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -262,7 +262,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -280,7 +280,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 7ca1d8fffc..51d9f7408d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1815,7 +1815,7 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
+ * Verify that the max_replication_origins configuration specified is enough for
* creating the subscriptions. This is required to create the replication
* origin for each subscription.
*/
@@ -1824,7 +1824,7 @@ check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1839,16 +1839,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_replication_origins)
+ pg_fatal("\"max_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index 661a2715c6..043a538ee4 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,7 +41,7 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
+# Check that pg_upgrade fails when max_replication_origins configured in the new
# cluster is less than the number of subscriptions in the old cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
@@ -52,10 +52,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_replication_origins.
command_checks_all(
[
'pg_upgrade', '--no-sync', '-d', $old_sub->data_dir,
@@ -66,14 +66,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb..8f65d4ef30 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
Hi Euler,
As you may have already read, separating the max_replication_slots parameter
is also discussed in the following thread.
/messages/by-id/CAA4eK1KF4MbyWmPj9ctNo8Fiei=K91RGYtzV5ELeCvR=_rqNgg@mail.gmail.com
21ac553b42d53249e42
I agree with separating the parameters.
I think we need to discuss the names of the parameter.
Parameter names were also discussed in the above thread.
It was suggested to name them 'max_replication_origin_states' or
'max_tracked_replication_origins'.
Personally, I find 'max_replication_origin_states' easier to understand,
although it is a long name.
Best Regards,
Keisuke Kuroda
NTT Comware
On Wed, Jan 8, 2025 at 7:39 AM Euler Taveira <euler@eulerto.com> wrote:
On Thu, Dec 19, 2024, at 10:31 AM, Peter Eisentraut wrote:
On 10.12.24 19:41, Euler Taveira wrote:
I'm attaching a patch that adds max_replication_origins. It basically
replaces
all of the points that refers to max_replication_slots on the subscriber. It
uses the same default value as max_replication_slots (10). I did nothing to
keep the backward compatibility. I mean has a special value (like -1) that
means same value as max_replication_slots. Is it worth it? (If not, it
should
be mentioned in the commit message.)I think some backward compatibility tweak like that would be useful.
Otherwise, the net effect of this is that most people will have to do
one more thing than before to keep their systems working. Also, all
deployment and automation scripts would have to be updated for this.This new patch includes the special value (-1) that uses max_replication_slots
instead.
Thank you for working on this. The proposed idea makes sense to me.
Here are some comments on v2 patch:
---
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("cannot start logical
replication workers when \"max_replication_slots\"=0")));
Need to update the error message too.
---
+ {"max_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of
simultaneously defined replication origins."),
+ NULL
+ },
I think the description is not accurate; this GUC controls the maximum
number of simultaneous replication origins that can be setup.
---
Given that max_replication_origins doesn't control the maximum number
of replication origins that can be defined, probably we need to find a
better name. As Kuroda-san already mentioned some proposed names,
max_tracked_replication_origins or max_replication_origin_states seem
reasonable to me.
---
+#include "utils/guc_hooks.h"
I think #include'ing guc.h would be more appropriate.
------
ereport(PANIC,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg("replication slot checkpoint has wrong
checksum %u, expected %u",
crc, file_crc)));
I don't understand why we use the term "replication slot checkpoint"
in an error message for the replication origin code. It could be fixed
in a separate patch for back-patching.
---
It's not related to the proposed patch but I found a suspicious
behavior; when the server startup we try to restore the
replorigin_checkpoint file only when max_replication_slots (or
max_replication_origins with the patch) > 0. Therefore, we don't get
the error message "could not find free replication state, increase
\"max_replication_slots\"" when it's set to 0, even if we have some
replication states in the replorigin_checkpoint file. Which seems
quite confusing.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Jan 24, 2025, at 8:12 PM, Masahiko Sawada wrote:
Here are some comments on v2 patch:
--- /* Report this after the initial starting message for consistency. */ - if (max_replication_slots == 0) + if (max_replication_origins == 0) ereport(ERROR, (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED), errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));Need to update the error message too.
Good catch!
--- + {"max_replication_origins", + PGC_POSTMASTER, + REPLICATION_SUBSCRIBERS, + gettext_noop("Sets the maximum number of simultaneously defined replication origins."), + NULL + },I think the description is not accurate; this GUC controls the maximum
number of simultaneous replication origins that can be setup.
Instead of "defined" I use "configured". It seems closer to "setup".
---
Given that max_replication_origins doesn't control the maximum number
of replication origins that can be defined, probably we need to find a
better name. As Kuroda-san already mentioned some proposed names,
max_tracked_replication_origins or max_replication_origin_states seem
reasonable to me.
The max_replication_origins name is not accurate. I chose it because (a) it is
a runtime limit, (b) it is short and (c) a description can provide the exact
meaning. I think the proposed names don't still reflect the exact meaning. The
"tracked" word provides the meaning that the replication origin is tracked but
in this case it should mean "setup". An existing replication origin that is not
in use is tracked although its information is not available in the
pg_replication_origin_status. The "states" word doesn't make sense in this
context. Do you mean "status" (same as the view name)?
Under reflection, an accurate name is max_replication_origin_session_setup. A
counter argument is that it is a long name (top-5 length).
postgres=# select n, length(n) from (values('max_replication_origins'),
('max_tracked_replication_origins'),('max_replication_origin_states'),
('max_replication_origin_session_setup')) as gucs(n);
n | length
--------------------------------------+--------
max_replication_origins | 23
max_tracked_replication_origins | 31
max_replication_origin_states | 29
max_replication_origin_session_setup | 36
(4 rows)
postgres=# select name, length(name) from pg_settings order by 2 desc
limit 15;
name | length
---------------------------------------------+--------
max_parallel_apply_workers_per_subscription | 43
ssl_passphrase_command_supports_reload | 38
autovacuum_vacuum_insert_scale_factor | 37
autovacuum_multixact_freeze_max_age | 35
debug_logical_replication_streaming | 35
idle_in_transaction_session_timeout | 35
autovacuum_vacuum_insert_threshold | 34
log_parameter_max_length_on_error | 33
vacuum_multixact_freeze_table_age | 33
max_sync_workers_per_subscription | 33
client_connection_check_interval | 32
max_parallel_maintenance_workers | 32
shared_memory_size_in_huge_pages | 32
restrict_nonsystem_relation_kind | 32
autovacuum_analyze_scale_factor | 31
(15 rows)
--- +#include "utils/guc_hooks.h"I think #include'ing guc.h would be more appropriate.
Fixed.
I also updated the pg_createsubscriber documentation that refers to
max_replication_slots.
Since we don't have an agreement about the name, I still kept
max_replication_origins.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v3-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v3-0001-Separate-GUC-for-replication-origins.patch?="Download
From ece93307b4085595a3610f0d53d28df1d3b9a76f Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v3] Separate GUC for replication origins
This feature already exists but it is provided by an existing GUC:
max_replication_slots. The new GUC (max_replication_origins) defines the
maximum number of replication origins that can be configured
simultaneously. The max_replication_slots was used for this purpose but
it is confusing (when you are learning about logical replication) and
introduces a limitation (you cannot have a small number of replication
slots and a high number of subscriptions).
For backward compatibility, the default is -1, indicating that the value
of max_replication_slots is used instead.
---
doc/src/sgml/config.sgml | 27 ++----
doc/src/sgml/logical-replication.sgml | 6 +-
doc/src/sgml/ref/pg_createsubscriber.sgml | 2 +-
src/backend/replication/logical/launcher.c | 6 +-
src/backend/replication/logical/origin.c | 91 ++++++++++++-------
src/backend/utils/misc/guc_tables.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/bin/pg_basebackup/pg_createsubscriber.c | 18 ++--
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 14 +--
src/bin/pg_upgrade/t/004_subscription.pl | 14 +--
src/include/replication/origin.h | 3 +
12 files changed, 113 insertions(+), 87 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a782f109982..0db28d81538 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4350,13 +4350,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5062,10 +5055,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-replication-origins" xreflabel="max_replication_origins">
+ <term><varname>max_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5077,18 +5070,14 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to -1, indicating
+ that the value of <xref linkend="guc-max-replication-slots"/> should be
+ used instead. This parameter can only be set at server start.
+
+ <literal>max_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 613abcd28b7..508bafbbdf6 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2371,9 +2371,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2408,7 +2406,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-replication-origins"><varname>max_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index 26b8e64a4e0..70931a5495f 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -295,7 +295,7 @@ PostgreSQL documentation
<para>
The target server must be used as a physical standby. The target server
- must have <xref linkend="guc-max-replication-slots"/> and <xref
+ must have <xref linkend="guc-max-replication-origins"/> and <xref
linkend="guc-max-logical-replication-workers"/> configured to a value
greater than or equal to the number of specified databases. The target
server must have <xref linkend="guc-max-worker-processes"/> configured to a
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a8..5025bc1c8c4 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,10 +325,10 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
+ errmsg("cannot start logical replication workers when \"max_replication_origins\"=0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 1b586cb1cf2..dfdece081cd 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_replication_origins = -1;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,38 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return size;
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_replication_origins == -1)
+ {
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", max_replication_slots);
+ SetConfigOption("max_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
+
+ /*
+ * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
+ * However, if the DBA explicitly set max_replication_origins = -1 in
+ * the config file, then PGC_S_DYNAMIC_DEFAULT will fail to override
+ * that and we must force the matter with PGC_S_OVERRIDE.
+ */
+ if (max_replication_origins == -1) /* failed to apply it? */
+ SetConfigOption("max_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_OVERRIDE);
+ }
+ Assert(max_replication_origins != -1);
+
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +552,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +569,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +591,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +604,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +646,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +739,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +749,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +817,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +873,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +938,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +979,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1045,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1131,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1145,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1180,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1216,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1557,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 71448bb4fdd..db0640e8bf4 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3279,6 +3279,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of simultaneously configured replication origins."),
+ NULL
+ },
+ &max_replication_origins,
+ -1, -1, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa7..249d3b50620 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -375,6 +375,9 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_replication_origins = -1 # maximum number of configured replication origins
+ # -1 to use max_replication_slots
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index faf18ccf131..4cbeeb5021d 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -964,7 +964,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -983,7 +983,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -991,7 +991,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1004,14 +1004,14 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
}
max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1020,12 +1020,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index c8dbdb7e9b7..98942d226c0 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -274,7 +274,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -293,7 +293,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 7ca1d8fffc9..51d9f7408d8 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1815,7 +1815,7 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
+ * Verify that the max_replication_origins configuration specified is enough for
* creating the subscriptions. This is required to create the replication
* origin for each subscription.
*/
@@ -1824,7 +1824,7 @@ check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1839,16 +1839,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_replication_origins)
+ pg_fatal("\"max_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index 13773316e1d..c69ad5da8fa 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,7 +41,7 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
+# Check that pg_upgrade fails when max_replication_origins configured in the new
# cluster is less than the number of subscriptions in the old cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
@@ -52,10 +52,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_replication_origins.
command_checks_all(
[
'pg_upgrade',
@@ -72,14 +72,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb0..8f65d4ef301 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira <euler@eulerto.com> wrote:
Under reflection, an accurate name is max_replication_origin_session_setup. A
counter argument is that it is a long name (top-5 length).postgres=# select n, length(n) from (values('max_replication_origins'),
('max_tracked_replication_origins'),('max_replication_origin_states'),
('max_replication_origin_session_setup')) as gucs(n);
n | length
--------------------------------------+--------
max_replication_origins | 23
max_tracked_replication_origins | 31
max_replication_origin_states | 29
max_replication_origin_session_setup | 36
(4 rows)
The other possibility is max_replication_origin_sessions.
--
With Regards,
Amit Kapila.
On Wed, Feb 5, 2025, at 1:56 AM, Amit Kapila wrote:
On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira <euler@eulerto.com> wrote:
Under reflection, an accurate name is max_replication_origin_session_setup. A
counter argument is that it is a long name (top-5 length).postgres=# select n, length(n) from (values('max_replication_origins'),
('max_tracked_replication_origins'),('max_replication_origin_states'),
('max_replication_origin_session_setup')) as gucs(n);
n | length
--------------------------------------+--------
max_replication_origins | 23
max_tracked_replication_origins | 31
max_replication_origin_states | 29
max_replication_origin_session_setup | 36
(4 rows)The other possibility is max_replication_origin_sessions.
Looks reasonable to me.
Opinions?
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Wed, Feb 5, 2025 at 4:39 PM Euler Taveira <euler@eulerto.com> wrote:
On Wed, Feb 5, 2025, at 1:56 AM, Amit Kapila wrote:
On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira <euler@eulerto.com> wrote:
Under reflection, an accurate name is max_replication_origin_session_setup. A
counter argument is that it is a long name (top-5 length).postgres=# select n, length(n) from (values('max_replication_origins'),
('max_tracked_replication_origins'),('max_replication_origin_states'),
('max_replication_origin_session_setup')) as gucs(n);
n | length
--------------------------------------+--------
max_replication_origins | 23
max_tracked_replication_origins | 31
max_replication_origin_states | 29
max_replication_origin_session_setup | 36
(4 rows)The other possibility is max_replication_origin_sessions.
Looks reasonable to me.
Opinions?
+1
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Feb 5, 2025, at 9:49 PM, Masahiko Sawada wrote:
On Wed, Feb 5, 2025 at 4:39 PM Euler Taveira <euler@eulerto.com> wrote:
On Wed, Feb 5, 2025, at 1:56 AM, Amit Kapila wrote:
On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira <euler@eulerto.com> wrote:
Under reflection, an accurate name is max_replication_origin_session_setup. A
counter argument is that it is a long name (top-5 length).postgres=# select n, length(n) from (values('max_replication_origins'),
('max_tracked_replication_origins'),('max_replication_origin_states'),
('max_replication_origin_session_setup')) as gucs(n);
n | length
--------------------------------------+--------
max_replication_origins | 23
max_tracked_replication_origins | 31
max_replication_origin_states | 29
max_replication_origin_session_setup | 36
(4 rows)The other possibility is max_replication_origin_sessions.
Looks reasonable to me.
Opinions?
+1
Here is another patch that only changes the GUC name to
max_replication_origin_sessions.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v4-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v4-0001-Separate-GUC-for-replication-origins.patch?="Download
From 5500a6bd126534aa11477f923a2fafdb5fdc6b00 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v4] Separate GUC for replication origins
The new GUC (max_replication_origin_sessions) defines the maximum number
of replication origins that can be configured simultaneously. The
max_replication_slots was used for this purpose but it is confusing
(when you are learning about logical replication) and introduces a
limitation (you cannot have a small number of replication slots and a
high number of subscriptions).
For backward compatibility, the default is -1, indicating that the value
of max_replication_slots is used instead.
---
doc/src/sgml/config.sgml | 27 ++----
doc/src/sgml/logical-replication.sgml | 6 +-
doc/src/sgml/ref/pg_createsubscriber.sgml | 2 +-
src/backend/replication/logical/launcher.c | 6 +-
src/backend/replication/logical/origin.c | 92 ++++++++++++-------
src/backend/utils/misc/guc_tables.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/bin/pg_basebackup/pg_createsubscriber.c | 18 ++--
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 18 ++--
src/bin/pg_upgrade/t/004_subscription.pl | 17 ++--
src/include/replication/origin.h | 3 +
12 files changed, 118 insertions(+), 90 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index fc186657a5..3f419e41be 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4356,13 +4356,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5068,10 +5061,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-replication-origin-sessions" xreflabel="max_replication_origin_sessions">
+ <term><varname>max_replication_origin_sessions</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_replication_origin_sessions</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5083,18 +5076,14 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to -1, indicating
+ that the value of <xref linkend="guc-max-replication-slots"/> should be
+ used instead. This parameter can only be set at server start.
+
+ <literal>max_replication_origin_sessions</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 613abcd28b..1e1e381c5c 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2371,9 +2371,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2408,7 +2406,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-replication-origin-sessions"><varname>max_replication_origin_sessions</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index 26b8e64a4e..a938390d50 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -295,7 +295,7 @@ PostgreSQL documentation
<para>
The target server must be used as a physical standby. The target server
- must have <xref linkend="guc-max-replication-slots"/> and <xref
+ must have <xref linkend="guc-max-replication-origin-sessions"/> and <xref
linkend="guc-max-logical-replication-workers"/> configured to a value
greater than or equal to the number of specified databases. The target
server must have <xref linkend="guc-max-worker-processes"/> configured to a
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a..c4b3ab5b19 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,10 +325,10 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_replication_origin_sessions == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
+ errmsg("cannot start logical replication workers when \"max_replication_origin_sessions\"=0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 1b586cb1cf..def9031bf2 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_replication_origin_sessions = -1;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_replication_origin_sessions */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_replication_origin_sessions.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_replication_origin_sessions == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_replication_origin_sessions\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,39 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_replication_origin_sessions == 0)
return size;
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_replication_origin_sessions == -1)
+ {
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", max_replication_slots);
+ SetConfigOption("max_replication_origin_sessions", buf,
+ PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
+
+ /*
+ * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
+ * However, if the DBA explicitly set max_replication_origin_sessions
+ * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
+ * fail to override that and we must force the matter with
+ * PGC_S_OVERRIDE.
+ */
+ if (max_replication_origin_sessions == -1) /* failed to apply it? */
+ SetConfigOption("max_replication_origin_sessions", buf,
+ PGC_POSTMASTER, PGC_S_OVERRIDE);
+ }
+ Assert(max_replication_origin_sessions != -1);
+
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_replication_origin_sessions, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +553,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_replication_origin_sessions == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +570,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +592,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_replication_origin_sessions.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +605,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_replication_origin_sessions == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +647,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +740,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_replication_origin_sessions == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +750,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_replication_origin_sessions == 0 last run, or we
+ * just brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +818,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_replication_origin_sessions)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_replication_origin_sessions\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +874,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +939,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +980,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origin_sessions\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1046,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationState *state;
@@ -1110,7 +1132,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_replication_origin_sessions > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1146,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1181,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_replication_origin_sessions\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1217,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_replication_origin_sessions != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1558,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origin_sessions; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 382c774b24..94c83be74b 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3280,6 +3280,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_replication_origin_sessions",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of simultaneously configured replication origins."),
+ NULL
+ },
+ &max_replication_origin_sessions,
+ -1, -1, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index f039eaa0c6..9363792c7c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -378,6 +378,9 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_replication_origin_sessions = -1 # maximum number of configured replication origins
+ # -1 to use max_replication_slots
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index faf18ccf13..b5e6bfa6e1 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -964,7 +964,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -983,7 +983,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_replication_origin_sessions >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -991,7 +991,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_replication_origin_sessions', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1004,14 +1004,14 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
}
max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_replication_origin_sessions: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1020,12 +1020,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d replication origin sessions, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_replication_origin_sessions", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index c8dbdb7e9b..c64d5fbcc7 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -274,7 +274,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_replication_origin_sessions = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -293,7 +293,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_replication_origin_sessions = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 7ca1d8fffc..4c9153e4a4 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1815,16 +1815,16 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
- * creating the subscriptions. This is required to create the replication
- * origin for each subscription.
+ * Verify that the max_replication_origin_sessions configuration specified is
+ * enough for creating the subscriptions. This is required to create the
+ * replication origin for each subscription.
*/
static void
check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_replication_origin_sessions;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1839,16 +1839,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_replication_origin_sessions';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_replication_origin_sessions = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_replication_origin_sessions)
+ pg_fatal("\"max_replication_origin_sessions\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_replication_origin_sessions, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index 13773316e1..27c9adeded 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,8 +41,9 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
-# cluster is less than the number of subscriptions in the old cluster.
+# Check that pg_upgrade fails when max_replication_origin_sessions configured
+# in the new cluster is less than the number of subscriptions in the old
+# cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
$publisher->safe_psql('postgres', "CREATE PUBLICATION regress_pub1");
@@ -52,10 +53,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_replication_origin_sessions = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_replication_origin_sessions.
command_checks_all(
[
'pg_upgrade',
@@ -72,14 +73,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_replication_origin_sessions" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_replication_origin_sessions'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_replication_origin_sessions
+$new_sub->append_conf('postgresql.conf', "max_replication_origin_sessions = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb..bd3fd9ce5e 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_replication_origin_sessions;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
On Tue, Feb 11, 2025 at 12:27 PM Euler Taveira <euler@eulerto.com> wrote:
On Wed, Feb 5, 2025, at 9:49 PM, Masahiko Sawada wrote:
On Wed, Feb 5, 2025 at 4:39 PM Euler Taveira <euler@eulerto.com> wrote:
On Wed, Feb 5, 2025, at 1:56 AM, Amit Kapila wrote:
On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira <euler@eulerto.com> wrote:
Under reflection, an accurate name is max_replication_origin_session_setup. A
counter argument is that it is a long name (top-5 length).postgres=# select n, length(n) from (values('max_replication_origins'),
('max_tracked_replication_origins'),('max_replication_origin_states'),
('max_replication_origin_session_setup')) as gucs(n);
n | length
--------------------------------------+--------
max_replication_origins | 23
max_tracked_replication_origins | 31
max_replication_origin_states | 29
max_replication_origin_session_setup | 36
(4 rows)The other possibility is max_replication_origin_sessions.
Looks reasonable to me.
Opinions?
+1
Here is another patch that only changes the GUC name to
max_replication_origin_sessions.
Thank you for updating the patch! The patch looks mostly good to me.
Here is one minor point:
In logical-replication.sgml there is another place we need to update
(in 29.13.2. Prepare for subscriber upgrades):
<listitem>
<para>
The new cluster must have
<link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
configured to a value greater than or equal to the number of
subscriptions present in the old cluster.
</para>
</listitem>
</itemizedlist>
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Feb 13, 2025 at 6:48 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
Thank you for updating the patch! The patch looks mostly good to me.
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_replication_origin_sessions == -1)
Shouldn't we let users use max_replication_origin_sessions for
subscribers? Maintaining this mapping for a long time can create
confusion such that some users will keep using max_replication_slots
for origins, and others will start using
max_replication_origin_sessions. Such a mapping would be useful if we
were doing something like this in back-branches, but doing it in a
major version appears to be more of a maintenance burden.
--
With Regards,
Amit Kapila.
On Sat, Mar 1, 2025, at 10:08 AM, Amit Kapila wrote:
On Thu, Feb 13, 2025 at 6:48 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
Thank you for updating the patch! The patch looks mostly good to me.
+ /* + * Prior to PostgreSQL 18, max_replication_slots was used to set the + * number of replication origins. For backward compatibility, -1 indicates + * to use the fallback value (max_replication_slots). + */ + if (max_replication_origin_sessions == -1)Shouldn't we let users use max_replication_origin_sessions for
subscribers? Maintaining this mapping for a long time can create
confusion such that some users will keep using max_replication_slots
for origins, and others will start using
max_replication_origin_sessions. Such a mapping would be useful if we
were doing something like this in back-branches, but doing it in a
major version appears to be more of a maintenance burden.
It was my initial proposal. Of course, it breaks compatibility but it
forces the users to adopt this new GUC. It will be a smooth adoption
because if we use the same value as max_replication_slots it means
most of the scenarios will be covered (10 is a good start point for the
majority of replication scenarios in my experience).
However, Peter E suggested that it would be a good idea to have a
backward compatibility so I added it.
We need to decide which option we want:
1. no backward compatibility and max_replication_origin_sessions = 10 as
default value. It might break scenarios that have the number of
subscriptions greater than the default value.
2. backward compatibility for a certain number of releases until the
tools can be adjusted. However, the applications that use it were
adjusted as part of this patch. The drawback here is that once we
accept -1, we cannot remove it without breaking compatibility.
My preference was 1 but I'm fine with 2 too. Since it is a major
release, it is natural to introduce features that break things.
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Wed, Mar 5, 2025 at 6:24 AM Euler Taveira <euler@eulerto.com> wrote:
On Sat, Mar 1, 2025, at 10:08 AM, Amit Kapila wrote:
On Thu, Feb 13, 2025 at 6:48 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
Thank you for updating the patch! The patch looks mostly good to me.
+ /* + * Prior to PostgreSQL 18, max_replication_slots was used to set the + * number of replication origins. For backward compatibility, -1 indicates + * to use the fallback value (max_replication_slots). + */ + if (max_replication_origin_sessions == -1)Shouldn't we let users use max_replication_origin_sessions for
subscribers? Maintaining this mapping for a long time can create
confusion such that some users will keep using max_replication_slots
for origins, and others will start using
max_replication_origin_sessions. Such a mapping would be useful if we
were doing something like this in back-branches, but doing it in a
major version appears to be more of a maintenance burden.It was my initial proposal. Of course, it breaks compatibility but it
forces the users to adopt this new GUC. It will be a smooth adoption
because if we use the same value as max_replication_slots it means
most of the scenarios will be covered (10 is a good start point for the
majority of replication scenarios in my experience).However, Peter E suggested that it would be a good idea to have a
backward compatibility so I added it.We need to decide which option we want:
1. no backward compatibility and max_replication_origin_sessions = 10 as
default value. It might break scenarios that have the number of
subscriptions greater than the default value.
To avoid breakage, can we add a check during the upgrade to ensure
that users have set max_replication_origin_sessions appropriately? See
the current check in function
check_new_cluster_subscription_configuration(). It uses
max_replication_slots, we can change it to use
max_replication_origin_sessions for versions greater than or equal to
18. Will that address this concern?
2. backward compatibility for a certain number of releases until the
tools can be adjusted. However, the applications that use it were
adjusted as part of this patch. The drawback here is that once we
accept -1, we cannot remove it without breaking compatibility.
Right, I find that path will add more maintenance burden in terms of
remembering this and finding a way to deprecate such a check.
My preference was 1 but I'm fine with 2 too. Since it is a major
release, it is natural to introduce features that break things.
+1.
Does anyone else have an opinion on this?
--
With Regards,
Amit Kapila.
On Tue, Mar 4, 2025 at 10:42 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Mar 5, 2025 at 6:24 AM Euler Taveira <euler@eulerto.com> wrote:
On Sat, Mar 1, 2025, at 10:08 AM, Amit Kapila wrote:
On Thu, Feb 13, 2025 at 6:48 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
Thank you for updating the patch! The patch looks mostly good to me.
+ /* + * Prior to PostgreSQL 18, max_replication_slots was used to set the + * number of replication origins. For backward compatibility, -1 indicates + * to use the fallback value (max_replication_slots). + */ + if (max_replication_origin_sessions == -1)Shouldn't we let users use max_replication_origin_sessions for
subscribers? Maintaining this mapping for a long time can create
confusion such that some users will keep using max_replication_slots
for origins, and others will start using
max_replication_origin_sessions. Such a mapping would be useful if we
were doing something like this in back-branches, but doing it in a
major version appears to be more of a maintenance burden.It was my initial proposal. Of course, it breaks compatibility but it
forces the users to adopt this new GUC. It will be a smooth adoption
because if we use the same value as max_replication_slots it means
most of the scenarios will be covered (10 is a good start point for the
majority of replication scenarios in my experience).However, Peter E suggested that it would be a good idea to have a
backward compatibility so I added it.We need to decide which option we want:
1. no backward compatibility and max_replication_origin_sessions = 10 as
default value. It might break scenarios that have the number of
subscriptions greater than the default value.To avoid breakage, can we add a check during the upgrade to ensure
that users have set max_replication_origin_sessions appropriately? See
the current check in function
check_new_cluster_subscription_configuration(). It uses
max_replication_slots, we can change it to use
max_replication_origin_sessions for versions greater than or equal to
18. Will that address this concern?
I think that the patch already replaced max_replication_slots with
max_replication_origin_sessions in that function.
2. backward compatibility for a certain number of releases until the
tools can be adjusted. However, the applications that use it were
adjusted as part of this patch. The drawback here is that once we
accept -1, we cannot remove it without breaking compatibility.Right, I find that path will add more maintenance burden in terms of
remembering this and finding a way to deprecate such a check.My preference was 1 but I'm fine with 2 too. Since it is a major
release, it is natural to introduce features that break things.+1.
+1
A major release would be a good timing to break off the relationship
between the number of origins and the number of replication slots.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Mar 5, 2025 at 12:42 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Tue, Mar 4, 2025 at 10:42 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Mar 5, 2025 at 6:24 AM Euler Taveira <euler@eulerto.com> wrote:
On Sat, Mar 1, 2025, at 10:08 AM, Amit Kapila wrote:
On Thu, Feb 13, 2025 at 6:48 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
Thank you for updating the patch! The patch looks mostly good to me.
+ /* + * Prior to PostgreSQL 18, max_replication_slots was used to set the + * number of replication origins. For backward compatibility, -1 indicates + * to use the fallback value (max_replication_slots). + */ + if (max_replication_origin_sessions == -1)Shouldn't we let users use max_replication_origin_sessions for
subscribers? Maintaining this mapping for a long time can create
confusion such that some users will keep using max_replication_slots
for origins, and others will start using
max_replication_origin_sessions. Such a mapping would be useful if we
were doing something like this in back-branches, but doing it in a
major version appears to be more of a maintenance burden.It was my initial proposal. Of course, it breaks compatibility but it
forces the users to adopt this new GUC. It will be a smooth adoption
because if we use the same value as max_replication_slots it means
most of the scenarios will be covered (10 is a good start point for the
majority of replication scenarios in my experience).However, Peter E suggested that it would be a good idea to have a
backward compatibility so I added it.We need to decide which option we want:
1. no backward compatibility and max_replication_origin_sessions = 10 as
default value. It might break scenarios that have the number of
subscriptions greater than the default value.To avoid breakage, can we add a check during the upgrade to ensure
that users have set max_replication_origin_sessions appropriately? See
the current check in function
check_new_cluster_subscription_configuration(). It uses
max_replication_slots, we can change it to use
max_replication_origin_sessions for versions greater than or equal to
18. Will that address this concern?I think that the patch already replaced max_replication_slots with
max_replication_origin_sessions in that function.
Then, that should be sufficient to avoid upgrade related risks.
--
With Regards,
Amit Kapila.
On 11.02.25 21:25, Euler Taveira wrote:
Here is another patch that only changes the GUC name to
max_replication_origin_sessions.
I think the naming and description of this is still confusing.
What does this name mean? There is (I think) no such thing as a
"replication origin session". So why would there be a maximum of those?
The description in the documentation, after the patch, says "Specifies
how many replication origins (see Chapter 48) can be tracked
simultaneously". But Chapter 48 does not say anything about what it
means for a replication slot to be tracked. The only use of the word
tracked in that chapter is to say that replication slots do the tracking
of replication progress.
Both of these are confusing independently, but moreover we are now using
two different words ("sessions" and "tracked") for apparently the same
thing, but neither of which is adequately documented in those terms
anywhere else.
I agree that the originally proposed name max_replication_origins is not
good, because you can "create" (using pg_replication_origin_create())
more than the configured maximum. What is the term for what the setting
actually controls? How many are "active"? "In use"? Per session? etc.
On Wed, Mar 5, 2025 at 4:38 PM Peter Eisentraut <peter@eisentraut.org> wrote:
On 11.02.25 21:25, Euler Taveira wrote:
Here is another patch that only changes the GUC name to
max_replication_origin_sessions.I think the naming and description of this is still confusing.
...
...
I agree that the originally proposed name max_replication_origins is not
good, because you can "create" (using pg_replication_origin_create())
more than the configured maximum. What is the term for what the setting
actually controls? How many are "active"? "In use"? Per session? etc.
It controls the number of active sessions using origin. The idea is
that to track replication progress via replication_origin we need to
do replorigin_session_setup(). If you look in the code, we have used
the term replorigin_session* in many places, so we thought of naming
this as max_replication_origin_sessions. But the other options could
be max_active_replication_origins or max_replication_origins_in_use.
--
With Regards,
Amit Kapila.
On Thu, Mar 6, 2025, at 6:55 AM, Amit Kapila wrote:
On Wed, Mar 5, 2025 at 4:38 PM Peter Eisentraut <peter@eisentraut.org> wrote:
On 11.02.25 21:25, Euler Taveira wrote:
Here is another patch that only changes the GUC name to
max_replication_origin_sessions.I think the naming and description of this is still confusing.
...
...I agree that the originally proposed name max_replication_origins is not
good, because you can "create" (using pg_replication_origin_create())
more than the configured maximum. What is the term for what the setting
actually controls? How many are "active"? "In use"? Per session? etc.It controls the number of active sessions using origin. The idea is
that to track replication progress via replication_origin we need to
do replorigin_session_setup(). If you look in the code, we have used
the term replorigin_session* in many places, so we thought of naming
this as max_replication_origin_sessions. But the other options could
be max_active_replication_origins or max_replication_origins_in_use.
The word "session" is correlated to "replication origin" but requires some
knowledge to know the replication progress tracking design. The word "active"
can express the fact that it was setup and is currently in use. I vote for
max_active_replication_origins.
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Thu, Mar 6, 2025 at 6:36 PM Euler Taveira <euler@eulerto.com> wrote:
On Thu, Mar 6, 2025, at 6:55 AM, Amit Kapila wrote:
On Wed, Mar 5, 2025 at 4:38 PM Peter Eisentraut <peter@eisentraut.org> wrote:
On 11.02.25 21:25, Euler Taveira wrote:
Here is another patch that only changes the GUC name to
max_replication_origin_sessions.I think the naming and description of this is still confusing.
...
...I agree that the originally proposed name max_replication_origins is not
good, because you can "create" (using pg_replication_origin_create())
more than the configured maximum. What is the term for what the setting
actually controls? How many are "active"? "In use"? Per session? etc.It controls the number of active sessions using origin. The idea is
that to track replication progress via replication_origin we need to
do replorigin_session_setup(). If you look in the code, we have used
the term replorigin_session* in many places, so we thought of naming
this as max_replication_origin_sessions. But the other options could
be max_active_replication_origins or max_replication_origins_in_use.The word "session" is correlated to "replication origin" but requires some
knowledge to know the replication progress tracking design. The word "active"
can express the fact that it was setup and is currently in use. I vote for
max_active_replication_origins.
Sounds reasonable. Let's go with max_active_replication_origins then,
unless people think otherwise.
--
With Regards,
Amit Kapila.
On 07.03.25 04:51, Amit Kapila wrote:
I agree that the originally proposed name max_replication_origins is not
good, because you can "create" (using pg_replication_origin_create())
more than the configured maximum. What is the term for what the setting
actually controls? How many are "active"? "In use"? Per session? etc.It controls the number of active sessions using origin. The idea is
that to track replication progress via replication_origin we need to
do replorigin_session_setup(). If you look in the code, we have used
the term replorigin_session* in many places, so we thought of naming
this as max_replication_origin_sessions. But the other options could
be max_active_replication_origins or max_replication_origins_in_use.The word "session" is correlated to "replication origin" but requires some
knowledge to know the replication progress tracking design. The word "active"
can express the fact that it was setup and is currently in use. I vote for
max_active_replication_origins.Sounds reasonable. Let's go with max_active_replication_origins then,
unless people think otherwise.
Is that maximum active for the whole system, or maximum active per
session, or maximum active per created origin, or some combination of these?
On Fri, Mar 7, 2025, at 11:47 AM, Peter Eisentraut wrote:
Is that maximum active for the whole system, or maximum active per
session, or maximum active per created origin, or some combination of these?
It is a cluster-wide setting. Similar to max_replication_slots. I will make
sure the GUC description is clear about it. It seems the Replication Progress
Tracking chapter needs an update to specify this information too.
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Fri, 7 Mar 2025 at 21:21, Euler Taveira <euler@eulerto.com> wrote:
On Fri, Mar 7, 2025, at 11:47 AM, Peter Eisentraut wrote:
Is that maximum active for the whole system, or maximum active per
session, or maximum active per created origin, or some combination of these?It is a cluster-wide setting. Similar to max_replication_slots. I will make
sure the GUC description is clear about it. It seems the Replication Progress
Tracking chapter needs an update to specify this information too.
I reviewed the discussion on this thread and believe we now have an
agreement on the design and GUC names. However, the patch still needs
updates and extensive testing, especially considering its impact on
backward compatibility. I'm unsure if this feature can be committed in
the current CommitFest. If you're okay with it, we can move it to the
next CommitFest. However, if you prefer to keep it, we can do our best
to complete it and make a final decision at the end of the CommitFest.
Regards,
Vignesh
On Wed, Mar 12, 2025, at 8:47 AM, vignesh C wrote:
I reviewed the discussion on this thread and believe we now have an
agreement on the design and GUC names. However, the patch still needs
updates and extensive testing, especially considering its impact on
backward compatibility. I'm unsure if this feature can be committed in
the current CommitFest. If you're okay with it, we can move it to the
next CommitFest. However, if you prefer to keep it, we can do our best
to complete it and make a final decision at the end of the CommitFest.
This is a mechanical patch. I was waiting if someone would object or suggest a
better GUC name. It seems to me it isn't. The pre existing GUC
(max_replication_slots) already has coverage. I don't know what additional
tests you have in mind. Could you elaborate?
I'm biased to say that it is one of the easiest patches to commit because it is
just assigning a new GUC name for a pre existing functionality. If there is no
interested in it, it will be moved to the next CF.
I'm adding 2 patches. The 0001 contains the same version as the previous one
but I renamed the GUC name to max_active_replication_origins. I'm also
attaching 0002 if we decide that backward compatibility is not a concern so it
removes it and assign 10 as the default value. I'm adding an additional suffix
so CF bot doesn't grab 0002.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v5-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v5-0001-Separate-GUC-for-replication-origins.patch?="Download
From 09986ee856eac6f6399fa354f576050c4a5d5c16 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v5 1/2] Separate GUC for replication origins
The new GUC (max_active_replication_origins) defines the maximum number
of active replication origins. The max_replication_slots was used for
this purpose but it is confusing (when you are learning about logical
replication) and introduces a limitation (you cannot have a small number
of replication slots and a high number of subscriptions).
For backward compatibility, the default is -1, indicating that the value
of max_replication_slots is used instead.
---
doc/src/sgml/config.sgml | 27 ++----
doc/src/sgml/logical-replication.sgml | 6 +-
doc/src/sgml/ref/pg_createsubscriber.sgml | 2 +-
src/backend/replication/logical/launcher.c | 6 +-
src/backend/replication/logical/origin.c | 92 ++++++++++++-------
src/backend/utils/misc/guc_tables.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/bin/pg_basebackup/pg_createsubscriber.c | 18 ++--
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 18 ++--
src/bin/pg_upgrade/t/004_subscription.pl | 17 ++--
src/include/replication/origin.h | 3 +
12 files changed, 118 insertions(+), 90 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8c82b39a89d..1ec4aceef4f 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4376,13 +4376,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5128,10 +5121,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-active-replication-origins" xreflabel="max_active_replication_origins">
+ <term><varname>max_active_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_active_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5143,18 +5136,14 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to -1, indicating
+ that the value of <xref linkend="guc-max-replication-slots"/> should be
+ used instead. This parameter can only be set at server start.
+
+ <literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3d18e507bbc..ee8f87c0905 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2371,9 +2371,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2413,7 +2411,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index b4b996236e4..4f88e6dddbd 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -311,7 +311,7 @@ PostgreSQL documentation
<para>
The target server must be used as a physical standby. The target server
- must have <xref linkend="guc-max-replication-slots"/> and <xref
+ must have <xref linkend="guc-max-active-replication-origins"/> and <xref
linkend="guc-max-logical-replication-workers"/> configured to a value
greater than or equal to the number of specified databases. The target
server must have <xref linkend="guc-max-worker-processes"/> configured to a
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a8..10677da56b2 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,10 +325,10 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
+ errmsg("cannot start logical replication workers when \"max_active_replication_origins\"=0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index c3c1d7a2a51..8252e03f978 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_active_replication_origins = -1;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_active_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_active_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_active_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,39 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return size;
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_active_replication_origins == -1)
+ {
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", max_replication_slots);
+ SetConfigOption("max_active_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
+
+ /*
+ * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
+ * However, if the DBA explicitly set max_active_replication_origins
+ * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
+ * fail to override that and we must force the matter with
+ * PGC_S_OVERRIDE.
+ */
+ if (max_active_replication_origins == -1) /* failed to apply it? */
+ SetConfigOption("max_active_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_OVERRIDE);
+ }
+ Assert(max_active_replication_origins != -1);
+
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_active_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +553,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +570,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +592,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_active_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +605,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +647,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +740,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +750,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_active_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +818,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_active_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_active_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +874,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +939,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +980,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1046,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1132,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_active_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1146,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1181,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1217,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_active_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1558,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 508970680d1..fbaa85c3afd 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3304,6 +3304,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_active_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of active replication origins."),
+ NULL
+ },
+ &max_active_replication_origins,
+ -1, -1, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 36cb64d7ebc..a27840f25c2 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -382,6 +382,9 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_active_replication_origins = -1 # maximum number of active replication origins
+ # -1 to use max_replication_slots
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index a5a2d61165d..032ce657083 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -999,7 +999,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -1018,7 +1018,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_active_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -1026,7 +1026,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_active_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1039,14 +1039,14 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
}
max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_active_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1055,12 +1055,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d active replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_active_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index c35fa108ce3..9ace3274e36 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -274,7 +274,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_active_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -293,7 +293,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_active_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 88db8869b6e..47a1abd393b 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1827,16 +1827,16 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
- * creating the subscriptions. This is required to create the replication
- * origin for each subscription.
+ * Verify that the max_active_replication_origins configuration specified is
+ * enough for creating the subscriptions. This is required to create the
+ * replication origin for each subscription.
*/
static void
check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_active_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1851,16 +1851,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_active_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_active_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_active_replication_origins)
+ pg_fatal("\"max_active_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_active_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index e3ccff9f270..c545abf6581 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,8 +41,9 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
-# cluster is less than the number of subscriptions in the old cluster.
+# Check that pg_upgrade fails when max_active_replication_origins configured
+# in the new cluster is less than the number of subscriptions in the old
+# cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
$publisher->safe_psql('postgres', "CREATE PUBLICATION regress_pub1");
@@ -52,10 +53,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_active_replication_origins.
command_checks_all(
[
'pg_upgrade',
@@ -72,14 +73,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_active_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_active_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_active_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb0..9cb2248fa9f 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_active_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
v5-0002-max_active_replication_origins-defaults-to-10.patch.nocfbotapplication/octet-stream; name="=?UTF-8?Q?v5-0002-max=5Factive=5Freplication=5Forigins-defaults-to-10.pa?= =?UTF-8?Q?tch.nocfbot?="Download
From fcbefaf80e74fbcecfbe09b2c7697bf2a6bc9b83 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Wed, 12 Mar 2025 19:50:04 -0300
Subject: [PATCH v5 2/2] max_active_replication_origins defaults to 10
---
doc/src/sgml/config.sgml | 5 ++--
src/backend/replication/logical/origin.c | 28 +------------------
src/backend/utils/misc/guc_tables.c | 2 +-
src/backend/utils/misc/postgresql.conf.sample | 3 +-
4 files changed, 5 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 1ec4aceef4f..4eba6f18036 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -5136,9 +5136,8 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting. It defaults to -1, indicating
- that the value of <xref linkend="guc-max-replication-slots"/> should be
- used instead. This parameter can only be set at server start.
+ will prevent the server from starting. It defaults to 10. This parameter
+ can only be set at server start.
<literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 8252e03f978..6583dd497da 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -101,7 +101,7 @@
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
/* GUC variables */
-int max_active_replication_origins = -1;
+int max_active_replication_origins = 10;
/*
* Replay progress of a single remote node.
@@ -515,32 +515,6 @@ ReplicationOriginShmemSize(void)
if (max_active_replication_origins == 0)
return size;
- /*
- * Prior to PostgreSQL 18, max_replication_slots was used to set the
- * number of replication origins. For backward compatibility, -1 indicates
- * to use the fallback value (max_replication_slots).
- */
- if (max_active_replication_origins == -1)
- {
- char buf[32];
-
- snprintf(buf, sizeof(buf), "%d", max_replication_slots);
- SetConfigOption("max_active_replication_origins", buf,
- PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
-
- /*
- * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
- * However, if the DBA explicitly set max_active_replication_origins
- * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
- * fail to override that and we must force the matter with
- * PGC_S_OVERRIDE.
- */
- if (max_active_replication_origins == -1) /* failed to apply it? */
- SetConfigOption("max_active_replication_origins", buf,
- PGC_POSTMASTER, PGC_S_OVERRIDE);
- }
- Assert(max_active_replication_origins != -1);
-
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index fbaa85c3afd..d891eb34c70 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3312,7 +3312,7 @@ struct config_int ConfigureNamesInt[] =
NULL
},
&max_active_replication_origins,
- -1, -1, MAX_BACKENDS,
+ 10, 0, MAX_BACKENDS,
NULL, NULL, NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index a27840f25c2..fbdd4a8ef26 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -382,8 +382,7 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
-#max_active_replication_origins = -1 # maximum number of active replication origins
- # -1 to use max_replication_slots
+#max_active_replication_origins = 10 # maximum number of active replication origins
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
--
2.39.5
On Thu, 13 Mar 2025 at 05:59, Euler Taveira <euler@eulerto.com> wrote:
On Wed, Mar 12, 2025, at 8:47 AM, vignesh C wrote:
I reviewed the discussion on this thread and believe we now have an
agreement on the design and GUC names. However, the patch still needs
updates and extensive testing, especially considering its impact on
backward compatibility. I'm unsure if this feature can be committed in
the current CommitFest. If you're okay with it, we can move it to the
next CommitFest. However, if you prefer to keep it, we can do our best
to complete it and make a final decision at the end of the CommitFest.This is a mechanical patch. I was waiting if someone would object or suggest a
better GUC name. It seems to me it isn't. The pre existing GUC
(max_replication_slots) already has coverage. I don't know what additional
tests you have in mind. Could you elaborate?
I was considering any potential impact on pg_upgrade and
pg_createsubscriber. I will run tests with the latest posted patch to
verify.
I'm biased to say that it is one of the easiest patches to commit because it is
just assigning a new GUC name for a pre existing functionality. If there is no
interested in it, it will be moved to the next CF.
Sounds like a plan! Let's verify it and work towards getting it committed.
Regards,
Vignesh
On Thu, 13 Mar 2025 at 05:59, Euler Taveira <euler@eulerto.com> wrote:
On Wed, Mar 12, 2025, at 8:47 AM, vignesh C wrote:
I reviewed the discussion on this thread and believe we now have an
agreement on the design and GUC names. However, the patch still needs
updates and extensive testing, especially considering its impact on
backward compatibility. I'm unsure if this feature can be committed in
the current CommitFest. If you're okay with it, we can move it to the
next CommitFest. However, if you prefer to keep it, we can do our best
to complete it and make a final decision at the end of the CommitFest.This is a mechanical patch. I was waiting if someone would object or suggest a
better GUC name. It seems to me it isn't. The pre existing GUC
(max_replication_slots) already has coverage. I don't know what additional
tests you have in mind. Could you elaborate?I'm biased to say that it is one of the easiest patches to commit because it is
just assigning a new GUC name for a pre existing functionality. If there is no
interested in it, it will be moved to the next CF.I'm adding 2 patches. The 0001 contains the same version as the previous one
but I renamed the GUC name to max_active_replication_origins. I'm also
attaching 0002 if we decide that backward compatibility is not a concern so it
removes it and assign 10 as the default value. I'm adding an additional suffix
so CF bot doesn't grab 0002.
Few comments:
1) After selecting max_active_replication_origins setting in the
SELECT query having order by, the first record returned will be the
one with max_active_replication_origins, rather than the second
record, because max_active_replication_origins appears before
max_logical_replication_workers in the order.
res = PQexec(conn,
"SELECT setting FROM
pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_active_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
The code below in the function should be modified accordingly:
max_lrworkers = atoi(PQgetvalue(res, 0, 0));
max_reporigins = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
2) I felt max_replication_slots must be max_active_replication_origins
here in logical-replication.sgml:
The new cluster must have
<link linkend= guc-max-replication-slots
<varname>max_replication_slots</varname></link>
configured to a value greater than or equal to the number of
subscriptions present in the old cluster.
Regards,
Vignesh
On Thu, Mar 13, 2025, at 11:10 AM, vignesh C wrote:
Few comments:
Thanks for taking a look.
1) After selecting max_active_replication_origins setting in the
SELECT query having order by, the first record returned will be the
one with max_active_replication_origins, rather than the second
record, because max_active_replication_origins appears before
max_logical_replication_workers in the order.
Fixed.
2) I felt max_replication_slots must be max_active_replication_origins
here in logical-replication.sgml:
Fixed.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v6-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v6-0001-Separate-GUC-for-replication-origins.patch?="Download
From dfe9ff1e74bb171516b7008096a3eb1834b4f61d Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v6 1/2] Separate GUC for replication origins
The new GUC (max_active_replication_origins) defines the maximum number
of active replication origins. The max_replication_slots was used for
this purpose but it is confusing (when you are learning about logical
replication) and introduces a limitation (you cannot have a small number
of replication slots and a high number of subscriptions).
For backward compatibility, the default is -1, indicating that the value
of max_replication_slots is used instead.
---
doc/src/sgml/config.sgml | 27 ++----
doc/src/sgml/logical-replication.sgml | 8 +-
doc/src/sgml/ref/pg_createsubscriber.sgml | 2 +-
src/backend/replication/logical/launcher.c | 6 +-
src/backend/replication/logical/origin.c | 92 ++++++++++++-------
src/backend/utils/misc/guc_tables.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/bin/pg_basebackup/pg_createsubscriber.c | 20 ++--
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 18 ++--
src/bin/pg_upgrade/t/004_subscription.pl | 17 ++--
src/include/replication/origin.h | 3 +
12 files changed, 120 insertions(+), 92 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 8c82b39a89d..1ec4aceef4f 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4376,13 +4376,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5128,10 +5121,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-active-replication-origins" xreflabel="max_active_replication_origins">
+ <term><varname>max_active_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_active_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5143,18 +5136,14 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to -1, indicating
+ that the value of <xref linkend="guc-max-replication-slots"/> should be
+ used instead. This parameter can only be set at server start.
+
+ <literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3d18e507bbc..0b86ccc4907 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2371,9 +2371,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2413,7 +2411,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
@@ -2580,7 +2578,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
The new cluster must have
- <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
configured to a value greater than or equal to the number of
subscriptions present in the old cluster.
</para>
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index b4b996236e4..4f88e6dddbd 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -311,7 +311,7 @@ PostgreSQL documentation
<para>
The target server must be used as a physical standby. The target server
- must have <xref linkend="guc-max-replication-slots"/> and <xref
+ must have <xref linkend="guc-max-active-replication-origins"/> and <xref
linkend="guc-max-logical-replication-workers"/> configured to a value
greater than or equal to the number of specified databases. The target
server must have <xref linkend="guc-max-worker-processes"/> configured to a
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a8..10677da56b2 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,10 +325,10 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
+ errmsg("cannot start logical replication workers when \"max_active_replication_origins\"=0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index c3c1d7a2a51..8252e03f978 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_active_replication_origins = -1;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_active_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_active_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_active_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,39 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return size;
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_active_replication_origins == -1)
+ {
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", max_replication_slots);
+ SetConfigOption("max_active_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
+
+ /*
+ * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
+ * However, if the DBA explicitly set max_active_replication_origins
+ * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
+ * fail to override that and we must force the matter with
+ * PGC_S_OVERRIDE.
+ */
+ if (max_active_replication_origins == -1) /* failed to apply it? */
+ SetConfigOption("max_active_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_OVERRIDE);
+ }
+ Assert(max_active_replication_origins != -1);
+
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_active_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +553,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +570,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +592,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_active_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +605,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +647,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +740,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +750,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_active_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +818,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_active_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_active_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +874,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +939,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +980,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1046,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1132,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_active_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1146,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1181,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1217,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_active_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1558,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 508970680d1..fbaa85c3afd 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3304,6 +3304,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_active_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of active replication origins."),
+ NULL
+ },
+ &max_active_replication_origins,
+ -1, -1, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 36cb64d7ebc..a27840f25c2 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -382,6 +382,9 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_active_replication_origins = -1 # maximum number of active replication origins
+ # -1 to use max_replication_slots
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index a5a2d61165d..60050a2c3e2 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -999,7 +999,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -1018,7 +1018,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_active_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -1026,7 +1026,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_active_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1038,15 +1038,15 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, true);
}
- max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 0, 0));
+ max_lrworkers = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_active_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1055,12 +1055,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d active replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_active_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index c35fa108ce3..9ace3274e36 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -274,7 +274,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_active_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -293,7 +293,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_active_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 88db8869b6e..47a1abd393b 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1827,16 +1827,16 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
- * creating the subscriptions. This is required to create the replication
- * origin for each subscription.
+ * Verify that the max_active_replication_origins configuration specified is
+ * enough for creating the subscriptions. This is required to create the
+ * replication origin for each subscription.
*/
static void
check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_active_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1851,16 +1851,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_active_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_active_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_active_replication_origins)
+ pg_fatal("\"max_active_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_active_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index e3ccff9f270..c545abf6581 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,8 +41,9 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
-# cluster is less than the number of subscriptions in the old cluster.
+# Check that pg_upgrade fails when max_active_replication_origins configured
+# in the new cluster is less than the number of subscriptions in the old
+# cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
$publisher->safe_psql('postgres', "CREATE PUBLICATION regress_pub1");
@@ -52,10 +53,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_active_replication_origins.
command_checks_all(
[
'pg_upgrade',
@@ -72,14 +73,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_active_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_active_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_active_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb0..9cb2248fa9f 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_active_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
v6-0002-max_active_replication_origins-defaults-to-10.patch.nocfbotapplication/octet-stream; name="=?UTF-8?Q?v6-0002-max=5Factive=5Freplication=5Forigins-defaults-to-10.pa?= =?UTF-8?Q?tch.nocfbot?="Download
From 8eb0692ad9f065404c1cfd8bd6d072ec0c8f8597 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Wed, 12 Mar 2025 19:50:04 -0300
Subject: [PATCH v6 2/2] max_active_replication_origins defaults to 10
---
doc/src/sgml/config.sgml | 5 ++--
src/backend/replication/logical/origin.c | 28 +------------------
src/backend/utils/misc/guc_tables.c | 2 +-
src/backend/utils/misc/postgresql.conf.sample | 3 +-
4 files changed, 5 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 1ec4aceef4f..4eba6f18036 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -5136,9 +5136,8 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting. It defaults to -1, indicating
- that the value of <xref linkend="guc-max-replication-slots"/> should be
- used instead. This parameter can only be set at server start.
+ will prevent the server from starting. It defaults to 10. This parameter
+ can only be set at server start.
<literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 8252e03f978..6583dd497da 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -101,7 +101,7 @@
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
/* GUC variables */
-int max_active_replication_origins = -1;
+int max_active_replication_origins = 10;
/*
* Replay progress of a single remote node.
@@ -515,32 +515,6 @@ ReplicationOriginShmemSize(void)
if (max_active_replication_origins == 0)
return size;
- /*
- * Prior to PostgreSQL 18, max_replication_slots was used to set the
- * number of replication origins. For backward compatibility, -1 indicates
- * to use the fallback value (max_replication_slots).
- */
- if (max_active_replication_origins == -1)
- {
- char buf[32];
-
- snprintf(buf, sizeof(buf), "%d", max_replication_slots);
- SetConfigOption("max_active_replication_origins", buf,
- PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
-
- /*
- * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
- * However, if the DBA explicitly set max_active_replication_origins
- * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
- * fail to override that and we must force the matter with
- * PGC_S_OVERRIDE.
- */
- if (max_active_replication_origins == -1) /* failed to apply it? */
- SetConfigOption("max_active_replication_origins", buf,
- PGC_POSTMASTER, PGC_S_OVERRIDE);
- }
- Assert(max_active_replication_origins != -1);
-
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index fbaa85c3afd..d891eb34c70 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3312,7 +3312,7 @@ struct config_int ConfigureNamesInt[] =
NULL
},
&max_active_replication_origins,
- -1, -1, MAX_BACKENDS,
+ 10, 0, MAX_BACKENDS,
NULL, NULL, NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index a27840f25c2..fbdd4a8ef26 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -382,8 +382,7 @@
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
-#max_active_replication_origins = -1 # maximum number of active replication origins
- # -1 to use max_replication_slots
+#max_active_replication_origins = 10 # maximum number of active replication origins
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from max_logical_replication_workers
--
2.39.5
On Fri, 14 Mar 2025 at 06:25, Euler Taveira <euler@eulerto.com> wrote:
On Thu, Mar 13, 2025, at 11:10 AM, vignesh C wrote:
Few comments:
Thanks for taking a look.
1) After selecting max_active_replication_origins setting in the
SELECT query having order by, the first record returned will be the
one with max_active_replication_origins, rather than the second
record, because max_active_replication_origins appears before
max_logical_replication_workers in the order.Fixed.
2) I felt max_replication_slots must be max_active_replication_origins
here in logical-replication.sgml:Fixed.
Few comments:
1) Should we add a test case to verify that if
max_active_replication_origins is set to -1, it will use
max_replication_slots value:
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility,
-1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_active_replication_origins == -1)
Maybe since the default is -1, we can just add the verification in one
of the tests.
2) Should we consider about the origin's created by user using the
replication managment function at [1] or is it intentionally left out:
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
[1]: https://www.postgresql.org/docs/current/replication-origins.html
Regards,
Vignesh
On Fri, Mar 14, 2025, at 5:48 AM, vignesh C wrote:
1) Should we add a test case to verify that if
max_active_replication_origins is set to -1, it will use
max_replication_slots value:
I don't think so. I added the following assert to test exactly this condition.
if (max_active_replication_origins == -1) /* failed to apply it? */
SetConfigOption("max_active_replication_origins", buf,
PGC_POSTMASTER, PGC_S_OVERRIDE);
}
Assert(max_active_replication_origins != -1);
2) Should we consider about the origin's created by user using the replication managment function at [1] or is it intentionally left out: - <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link> + <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link> must be set to at least the number of subscriptions that will be added to the subscriber, plus some reserve for table synchronization.
I kept a minimal patch. The current sentence is vague because (a) it doesn't
refer to replication solutions that don't create subscription (as you said) and
(b) it doesn't specify the maximum number of replication origins that are
simultaneously used for table synchronization.
We can certainly expand the replication origin documentation but I don't think
it is material for this patch. I also don't think it is appropriate to expose
implementation details about table synchronization in a end user page. If we
can explain it without saying too much about the implementation details, fine.
--
Euler Taveira
EDB https://www.enterprisedb.com/
On Thu, Mar 13, 2025 at 5:55 PM Euler Taveira <euler@eulerto.com> wrote:
On Thu, Mar 13, 2025, at 11:10 AM, vignesh C wrote:
Few comments:
Thanks for taking a look.
1) After selecting max_active_replication_origins setting in the
SELECT query having order by, the first record returned will be the
one with max_active_replication_origins, rather than the second
record, because max_active_replication_origins appears before
max_logical_replication_workers in the order.Fixed.
2) I felt max_replication_slots must be max_active_replication_origins
here in logical-replication.sgml:Fixed.
Thank you for updating the patch. I have one comment:
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
+#max_active_replication_origins = 10 # maximum number of active
replication origins
+ # (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from
max_logical_replication_workers
#max_parallel_apply_workers_per_subscription = 2 # taken from
max_logical_replication_workers
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.
The rest looks good to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Mon, Mar 17, 2025, at 8:44 PM, Masahiko Sawada wrote:
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.
Good point. Looking at the documentation, the old max_replication_slots
parameter was the first one in that section so I decided to use the same order
for the postgresql.conf.sample.
--
Euler Taveira
EDB https://www.enterprisedb.com/
Attachments:
v7-0001-Separate-GUC-for-replication-origins.patchtext/x-patch; name="=?UTF-8?Q?v7-0001-Separate-GUC-for-replication-origins.patch?="Download
From bdd76cf3698d61eab455cd327de63c87769ef368 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Tue, 3 Sep 2024 12:10:20 -0300
Subject: [PATCH v7 1/2] Separate GUC for replication origins
The new GUC (max_active_replication_origins) defines the maximum number
of active replication origins. The max_replication_slots was used for
this purpose but it is confusing (when you are learning about logical
replication) and introduces a limitation (you cannot have a small number
of replication slots and a high number of subscriptions).
For backward compatibility, the default is -1, indicating that the value
of max_replication_slots is used instead.
---
doc/src/sgml/config.sgml | 27 ++----
doc/src/sgml/logical-replication.sgml | 8 +-
doc/src/sgml/ref/pg_createsubscriber.sgml | 2 +-
src/backend/replication/logical/launcher.c | 6 +-
src/backend/replication/logical/origin.c | 92 ++++++++++++-------
src/backend/utils/misc/guc_tables.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 3 +
src/bin/pg_basebackup/pg_createsubscriber.c | 20 ++--
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 18 ++--
src/bin/pg_upgrade/t/004_subscription.pl | 17 ++--
src/include/replication/origin.h | 3 +
12 files changed, 120 insertions(+), 92 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 3d62c8bd274..5371324c369 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4376,13 +4376,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5128,10 +5121,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-active-replication-origins" xreflabel="max_active_replication_origins">
+ <term><varname>max_active_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_active_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5143,18 +5136,14 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to -1, indicating
+ that the value of <xref linkend="guc-max-replication-slots"/> should be
+ used instead. This parameter can only be set at server start.
+
+ <literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3d18e507bbc..0b86ccc4907 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2371,9 +2371,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2413,7 +2411,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
@@ -2580,7 +2578,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
The new cluster must have
- <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
configured to a value greater than or equal to the number of
subscriptions present in the old cluster.
</para>
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index b4b996236e4..4f88e6dddbd 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -311,7 +311,7 @@ PostgreSQL documentation
<para>
The target server must be used as a physical standby. The target server
- must have <xref linkend="guc-max-replication-slots"/> and <xref
+ must have <xref linkend="guc-max-active-replication-origins"/> and <xref
linkend="guc-max-logical-replication-workers"/> configured to a value
greater than or equal to the number of specified databases. The target
server must have <xref linkend="guc-max-worker-processes"/> configured to a
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a8..10677da56b2 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,10 +325,10 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
+ errmsg("cannot start logical replication workers when \"max_active_replication_origins\"=0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index c3c1d7a2a51..8252e03f978 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_active_replication_origins = -1;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_active_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_active_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_active_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,39 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return size;
+ /*
+ * Prior to PostgreSQL 18, max_replication_slots was used to set the
+ * number of replication origins. For backward compatibility, -1 indicates
+ * to use the fallback value (max_replication_slots).
+ */
+ if (max_active_replication_origins == -1)
+ {
+ char buf[32];
+
+ snprintf(buf, sizeof(buf), "%d", max_replication_slots);
+ SetConfigOption("max_active_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
+
+ /*
+ * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
+ * However, if the DBA explicitly set max_active_replication_origins
+ * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
+ * fail to override that and we must force the matter with
+ * PGC_S_OVERRIDE.
+ */
+ if (max_active_replication_origins == -1) /* failed to apply it? */
+ SetConfigOption("max_active_replication_origins", buf,
+ PGC_POSTMASTER, PGC_S_OVERRIDE);
+ }
+ Assert(max_active_replication_origins != -1);
+
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_active_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +553,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +570,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +592,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_active_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +605,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +647,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +740,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +750,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_active_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +818,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_active_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_active_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +874,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +939,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +980,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1046,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1132,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_active_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1146,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1181,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1217,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_active_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1558,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 9c0b10ad4dc..464e12ddde0 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3313,6 +3313,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_active_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of active replication origins."),
+ NULL
+ },
+ &max_active_replication_origins,
+ -1, -1, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 8de86e0c945..51a076d7b49 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -380,6 +380,9 @@
# These settings are ignored on a publisher.
+#max_active_replication_origins = -1 # maximum number of active replication origins
+ # -1 to use max_replication_slots
+ # (change requires restart)
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 6baf92e8024..5018b0e9aaa 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -1001,7 +1001,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -1020,7 +1020,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_active_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -1028,7 +1028,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_active_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1040,15 +1040,15 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, true);
}
- max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 0, 0));
+ max_lrworkers = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_active_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1057,12 +1057,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d active replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_active_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index c35fa108ce3..9ace3274e36 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -274,7 +274,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_active_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -293,7 +293,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_active_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 88db8869b6e..47a1abd393b 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1827,16 +1827,16 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
- * creating the subscriptions. This is required to create the replication
- * origin for each subscription.
+ * Verify that the max_active_replication_origins configuration specified is
+ * enough for creating the subscriptions. This is required to create the
+ * replication origin for each subscription.
*/
static void
check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_active_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1851,16 +1851,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_active_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_active_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_active_replication_origins)
+ pg_fatal("\"max_active_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_active_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index e3ccff9f270..c545abf6581 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,8 +41,9 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
-# cluster is less than the number of subscriptions in the old cluster.
+# Check that pg_upgrade fails when max_active_replication_origins configured
+# in the new cluster is less than the number of subscriptions in the old
+# cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
$publisher->safe_psql('postgres', "CREATE PUBLICATION regress_pub1");
@@ -52,10 +53,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_active_replication_origins.
command_checks_all(
[
'pg_upgrade',
@@ -72,14 +73,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_active_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_active_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_active_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb0..9cb2248fa9f 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_active_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.39.5
v7-0002-max_active_replication_origins-defaults-to-10.patchtext/x-patch; name="=?UTF-8?Q?v7-0002-max=5Factive=5Freplication=5Forigins-defaults-to-10.pa?= =?UTF-8?Q?tch?="Download
From 1b99c32c27bd00e95063477a54c88345070ed278 Mon Sep 17 00:00:00 2001
From: Euler Taveira <euler@eulerto.com>
Date: Wed, 12 Mar 2025 19:50:04 -0300
Subject: [PATCH v7 2/2] max_active_replication_origins defaults to 10
---
doc/src/sgml/config.sgml | 5 ++--
src/backend/replication/logical/origin.c | 28 +------------------
src/backend/utils/misc/guc_tables.c | 2 +-
src/backend/utils/misc/postgresql.conf.sample | 3 +-
4 files changed, 5 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 5371324c369..f65db2e1cc1 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -5136,9 +5136,8 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting. It defaults to -1, indicating
- that the value of <xref linkend="guc-max-replication-slots"/> should be
- used instead. This parameter can only be set at server start.
+ will prevent the server from starting. It defaults to 10. This parameter
+ can only be set at server start.
<literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 8252e03f978..6583dd497da 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -101,7 +101,7 @@
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
/* GUC variables */
-int max_active_replication_origins = -1;
+int max_active_replication_origins = 10;
/*
* Replay progress of a single remote node.
@@ -515,32 +515,6 @@ ReplicationOriginShmemSize(void)
if (max_active_replication_origins == 0)
return size;
- /*
- * Prior to PostgreSQL 18, max_replication_slots was used to set the
- * number of replication origins. For backward compatibility, -1 indicates
- * to use the fallback value (max_replication_slots).
- */
- if (max_active_replication_origins == -1)
- {
- char buf[32];
-
- snprintf(buf, sizeof(buf), "%d", max_replication_slots);
- SetConfigOption("max_active_replication_origins", buf,
- PGC_POSTMASTER, PGC_S_DYNAMIC_DEFAULT);
-
- /*
- * We prefer to report this value's source as PGC_S_DYNAMIC_DEFAULT.
- * However, if the DBA explicitly set max_active_replication_origins
- * equals to -1 in the config file, then PGC_S_DYNAMIC_DEFAULT will
- * fail to override that and we must force the matter with
- * PGC_S_OVERRIDE.
- */
- if (max_active_replication_origins == -1) /* failed to apply it? */
- SetConfigOption("max_active_replication_origins", buf,
- PGC_POSTMASTER, PGC_S_OVERRIDE);
- }
- Assert(max_active_replication_origins != -1);
-
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 464e12ddde0..6004fe1629f 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3321,7 +3321,7 @@ struct config_int ConfigureNamesInt[] =
NULL
},
&max_active_replication_origins,
- -1, -1, MAX_BACKENDS,
+ 10, 0, MAX_BACKENDS,
NULL, NULL, NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 51a076d7b49..e604b73595c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -380,8 +380,7 @@
# These settings are ignored on a publisher.
-#max_active_replication_origins = -1 # maximum number of active replication origins
- # -1 to use max_replication_slots
+#max_active_replication_origins = 10 # maximum number of active replication origins
# (change requires restart)
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
--
2.39.5
On Mon, Mar 17, 2025 at 6:05 PM Euler Taveira <euler@eulerto.com> wrote:
On Mon, Mar 17, 2025, at 8:44 PM, Masahiko Sawada wrote:
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.Good point. Looking at the documentation, the old max_replication_slots
parameter was the first one in that section so I decided to use the same order
for the postgresql.conf.sample.
Thank you for updating the patch!
The patch looks good to me. I've updated the commit message and
squashed the second patch as we agreed that we don't need to have the
codes for supporting backward compatibility IIUC. I'm going to push
the patch barring any objections and further comments.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
v8-0001-Add-GUC-option-to-control-maximum-active-replicat.patchapplication/octet-stream; name=v8-0001-Add-GUC-option-to-control-maximum-active-replicat.patchDownload
From 7addaf530bdf48e2eff832d80063790c64f13d66 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Tue, 18 Mar 2025 21:46:28 -0700
Subject: [PATCH v8] Add GUC option to control maximum active replication
origins
This commit introduces a new GUC option max_active_replication_origins
to control the maximum number of active replication
origins. Previously, this was controlled by
'max_replication_slots'. Having a separate GUC option provides better
flexibility for setting up subscribers, as they may not require
replication slots (for cascading replication) but always require
replication origins.
Author: Euler Taveira <euler@eulerto.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: vignesh C <vignesh21@gmail.com>
Discussion: https://postgr.es/m/b81db436-8262-4575-b7c4-bc0c1551000b@app.fastmail.com
---
doc/src/sgml/config.sgml | 26 ++------
doc/src/sgml/logical-replication.sgml | 8 +--
doc/src/sgml/ref/pg_createsubscriber.sgml | 2 +-
src/backend/replication/logical/launcher.c | 6 +-
src/backend/replication/logical/origin.c | 66 +++++++++----------
src/backend/utils/misc/guc_tables.c | 12 ++++
src/backend/utils/misc/postgresql.conf.sample | 2 +
src/bin/pg_basebackup/pg_createsubscriber.c | 20 +++---
.../t/040_pg_createsubscriber.pl | 4 +-
src/bin/pg_upgrade/check.c | 18 ++---
src/bin/pg_upgrade/t/004_subscription.pl | 17 ++---
src/include/replication/origin.h | 3 +
12 files changed, 92 insertions(+), 92 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 4fd377d1c5f..07c1ac1afd0 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4476,13 +4476,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
-
- <para>
- Note that this parameter also applies on the subscriber side, but with
- a different meaning. See <xref linkend="guc-max-replication-slots-subscriber"/>
- in <xref linkend="runtime-config-replication-subscriber"/> for more
- details.
- </para>
</listitem>
</varlistentry>
@@ -5228,10 +5221,10 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
- <varlistentry id="guc-max-replication-slots-subscriber" xreflabel="max_replication_slots">
- <term><varname>max_replication_slots</varname> (<type>integer</type>)
+ <varlistentry id="guc-max-active-replication-origins" xreflabel="max_active_replication_origins">
+ <term><varname>max_active_replication_origins</varname> (<type>integer</type>)
<indexterm>
- <primary><varname>max_replication_slots</varname> configuration parameter</primary>
+ <primary><varname>max_active_replication_origins</varname> configuration parameter</primary>
<secondary>in a subscriber</secondary>
</indexterm>
</term>
@@ -5243,18 +5236,13 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
be created on the server. Setting it to a lower value than the current
number of tracked replication origins (reflected in
<link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>)
- will prevent the server from starting.
- <literal>max_replication_slots</literal> must be set to at least the
+ will prevent the server from starting. It defaults to 10. This parameter
+ can only be set at server start.
+
+ <literal>max_active_replication_origins</literal> must be set to at least the
number of subscriptions that will be added to the subscriber, plus some
reserve for table synchronization.
</para>
-
- <para>
- Note that this parameter also applies on a sending server, but with
- a different meaning. See <xref linkend="guc-max-replication-slots"/>
- in <xref linkend="runtime-config-replication-sender"/> for more
- details.
- </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 3d18e507bbc..0b86ccc4907 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -2371,9 +2371,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
<sect2 id="logical-replication-config-publisher">
@@ -2413,7 +2411,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<title>Subscribers</title>
<para>
- <link linkend="guc-max-replication-slots-subscriber"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
must be set to at least the number of subscriptions that will be added to
the subscriber, plus some reserve for table synchronization.
</para>
@@ -2580,7 +2578,7 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER
<listitem>
<para>
The new cluster must have
- <link linkend="guc-max-replication-slots"><varname>max_replication_slots</varname></link>
+ <link linkend="guc-max-active-replication-origins"><varname>max_active_replication_origins</varname></link>
configured to a value greater than or equal to the number of
subscriptions present in the old cluster.
</para>
diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml
index b4b996236e4..4f88e6dddbd 100644
--- a/doc/src/sgml/ref/pg_createsubscriber.sgml
+++ b/doc/src/sgml/ref/pg_createsubscriber.sgml
@@ -311,7 +311,7 @@ PostgreSQL documentation
<para>
The target server must be used as a physical standby. The target server
- must have <xref linkend="guc-max-replication-slots"/> and <xref
+ must have <xref linkend="guc-max-active-replication-origins"/> and <xref
linkend="guc-max-logical-replication-workers"/> configured to a value
greater than or equal to the number of specified databases. The target
server must have <xref linkend="guc-max-worker-processes"/> configured to a
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index a3c7adbf1a8..10677da56b2 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -31,7 +31,7 @@
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "replication/logicallauncher.h"
-#include "replication/slot.h"
+#include "replication/origin.h"
#include "replication/walreceiver.h"
#include "replication/worker_internal.h"
#include "storage/ipc.h"
@@ -325,10 +325,10 @@ logicalrep_worker_launch(LogicalRepWorkerType wtype,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when \"max_replication_slots\"=0")));
+ errmsg("cannot start logical replication workers when \"max_active_replication_origins\"=0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index c3c1d7a2a51..6583dd497da 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -90,6 +90,7 @@
#include "storage/lmgr.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
+#include "utils/guc.h"
#include "utils/pg_lsn.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
@@ -99,6 +100,9 @@
#define PG_REPLORIGIN_CHECKPOINT_FILENAME PG_LOGICAL_DIR "/replorigin_checkpoint"
#define PG_REPLORIGIN_CHECKPOINT_TMPFILE PG_REPLORIGIN_CHECKPOINT_FILENAME ".tmp"
+/* GUC variables */
+int max_active_replication_origins = 10;
+
/*
* Replay progress of a single remote node.
*/
@@ -151,7 +155,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_active_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -162,10 +166,7 @@ TimestampTz replorigin_session_origin_timestamp = 0;
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_active_replication_origins.
*/
static ReplicationState *replication_states;
@@ -186,12 +187,12 @@ static ReplicationState *session_replication_state = NULL;
#define REPLICATION_STATE_MAGIC ((uint32) 0x1257DADE)
static void
-replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
+replorigin_check_prerequisites(bool check_origins, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_origins && max_active_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when \"max_replication_slots\" is 0")));
+ errmsg("cannot query or manipulate replication origin when \"max_active_replication_origins\" is 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -352,7 +353,7 @@ replorigin_state_clear(RepOriginId roident, bool nowait)
restart:
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -511,18 +512,13 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return size;
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_active_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -531,7 +527,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -548,7 +544,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -570,7 +566,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_active_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -583,7 +579,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -625,7 +621,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -718,7 +714,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_active_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -728,8 +724,8 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
- * up a standby.
+ * might have had max_active_replication_origins == 0 last run, or we just
+ * brought up a standby.
*/
if (fd < 0 && errno == ENOENT)
return;
@@ -796,10 +792,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_active_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase \"max_replication_slots\"")));
+ errmsg("could not find free replication state, increase \"max_active_replication_origins\"")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -852,7 +848,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -917,7 +913,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -958,7 +954,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
if (replication_state == NULL)
{
@@ -1024,7 +1020,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
@@ -1110,7 +1106,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_active_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1124,7 +1120,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1159,7 +1155,7 @@ replorigin_session_setup(RepOriginId node, int acquired_by)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with ID %d",
node),
- errhint("Increase \"max_replication_slots\" and try again.")));
+ errhint("Increase \"max_active_replication_origins\" and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1195,7 +1191,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_active_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1536,7 +1532,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_active_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ead80257192..cbb381bece5 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3363,6 +3363,18 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_active_replication_origins",
+ PGC_POSTMASTER,
+ REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of active replication origins."),
+ NULL
+ },
+ &max_active_replication_origins,
+ 10, 0, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Sets the amount of time to wait before forcing "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 6abd1baeac8..3a246199368 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -389,6 +389,8 @@
# These settings are ignored on a publisher.
+#max_active_replication_origins = 10 # maximum number of active replication origins
+ # (change requires restart)
#max_logical_replication_workers = 4 # taken from max_worker_processes
# (change requires restart)
#max_sync_workers_per_subscription = 2 # taken from max_logical_replication_workers
diff --git a/src/bin/pg_basebackup/pg_createsubscriber.c b/src/bin/pg_basebackup/pg_createsubscriber.c
index 6baf92e8024..5018b0e9aaa 100644
--- a/src/bin/pg_basebackup/pg_createsubscriber.c
+++ b/src/bin/pg_basebackup/pg_createsubscriber.c
@@ -1001,7 +1001,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
bool failed = false;
int max_lrworkers;
- int max_repslots;
+ int max_reporigins;
int max_wprocs;
pg_log_info("checking settings on subscriber");
@@ -1020,7 +1020,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
* Since these parameters are not a requirement for physical replication,
* we should check it to make sure it won't fail.
*
- * - max_replication_slots >= number of dbs to be converted
+ * - max_active_replication_origins >= number of dbs to be converted
* - max_logical_replication_workers >= number of dbs to be converted
* - max_worker_processes >= 1 + number of dbs to be converted
*------------------------------------------------------------------------
@@ -1028,7 +1028,7 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
res = PQexec(conn,
"SELECT setting FROM pg_catalog.pg_settings WHERE name IN ("
"'max_logical_replication_workers', "
- "'max_replication_slots', "
+ "'max_active_replication_origins', "
"'max_worker_processes', "
"'primary_slot_name') "
"ORDER BY name");
@@ -1040,15 +1040,15 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, true);
}
- max_lrworkers = atoi(PQgetvalue(res, 0, 0));
- max_repslots = atoi(PQgetvalue(res, 1, 0));
+ max_reporigins = atoi(PQgetvalue(res, 0, 0));
+ max_lrworkers = atoi(PQgetvalue(res, 1, 0));
max_wprocs = atoi(PQgetvalue(res, 2, 0));
if (strcmp(PQgetvalue(res, 3, 0), "") != 0)
primary_slot_name = pg_strdup(PQgetvalue(res, 3, 0));
pg_log_debug("subscriber: max_logical_replication_workers: %d",
max_lrworkers);
- pg_log_debug("subscriber: max_replication_slots: %d", max_repslots);
+ pg_log_debug("subscriber: max_active_replication_origins: %d", max_reporigins);
pg_log_debug("subscriber: max_worker_processes: %d", max_wprocs);
if (primary_slot_name)
pg_log_debug("subscriber: primary_slot_name: %s", primary_slot_name);
@@ -1057,12 +1057,12 @@ check_subscriber(const struct LogicalRepInfo *dbinfo)
disconnect_database(conn, false);
- if (max_repslots < num_dbs)
+ if (max_reporigins < num_dbs)
{
- pg_log_error("subscriber requires %d replication slots, but only %d remain",
- num_dbs, max_repslots);
+ pg_log_error("subscriber requires %d active replication origins, but only %d remain",
+ num_dbs, max_reporigins);
pg_log_error_hint("Increase the configuration parameter \"%s\" to at least %d.",
- "max_replication_slots", num_dbs);
+ "max_active_replication_origins", num_dbs);
failed = true;
}
diff --git a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
index c35fa108ce3..9ace3274e36 100644
--- a/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
+++ b/src/bin/pg_basebackup/t/040_pg_createsubscriber.pl
@@ -274,7 +274,7 @@ max_worker_processes = 8
# Check some unmet conditions on node S
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 1
+max_active_replication_origins = 1
max_logical_replication_workers = 1
max_worker_processes = 2
});
@@ -293,7 +293,7 @@ command_fails(
'standby contains unmet conditions on node S');
$node_s->append_conf(
'postgresql.conf', q{
-max_replication_slots = 10
+max_active_replication_origins = 10
max_logical_replication_workers = 4
max_worker_processes = 8
});
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 88daa808035..117f461d46a 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -1802,16 +1802,16 @@ check_new_cluster_logical_replication_slots(void)
/*
* check_new_cluster_subscription_configuration()
*
- * Verify that the max_replication_slots configuration specified is enough for
- * creating the subscriptions. This is required to create the replication
- * origin for each subscription.
+ * Verify that the max_active_replication_origins configuration specified is
+ * enough for creating the subscriptions. This is required to create the
+ * replication origin for each subscription.
*/
static void
check_new_cluster_subscription_configuration(void)
{
PGresult *res;
PGconn *conn;
- int max_replication_slots;
+ int max_active_replication_origins;
/* Subscriptions and their dependencies can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
@@ -1826,16 +1826,16 @@ check_new_cluster_subscription_configuration(void)
conn = connectToServer(&new_cluster, "template1");
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
- "WHERE name = 'max_replication_slots';");
+ "WHERE name = 'max_active_replication_origins';");
if (PQntuples(res) != 1)
pg_fatal("could not determine parameter settings on new cluster");
- max_replication_slots = atoi(PQgetvalue(res, 0, 0));
- if (old_cluster.nsubs > max_replication_slots)
- pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
+ max_active_replication_origins = atoi(PQgetvalue(res, 0, 0));
+ if (old_cluster.nsubs > max_active_replication_origins)
+ pg_fatal("\"max_active_replication_origins\" (%d) must be greater than or equal to the number of "
"subscriptions (%d) on the old cluster",
- max_replication_slots, old_cluster.nsubs);
+ max_active_replication_origins, old_cluster.nsubs);
PQclear(res);
PQfinish(conn);
diff --git a/src/bin/pg_upgrade/t/004_subscription.pl b/src/bin/pg_upgrade/t/004_subscription.pl
index e3ccff9f270..c545abf6581 100644
--- a/src/bin/pg_upgrade/t/004_subscription.pl
+++ b/src/bin/pg_upgrade/t/004_subscription.pl
@@ -41,8 +41,9 @@ chdir ${PostgreSQL::Test::Utils::tmp_check};
my $connstr = $publisher->connstr . ' dbname=postgres';
# ------------------------------------------------------
-# Check that pg_upgrade fails when max_replication_slots configured in the new
-# cluster is less than the number of subscriptions in the old cluster.
+# Check that pg_upgrade fails when max_active_replication_origins configured
+# in the new cluster is less than the number of subscriptions in the old
+# cluster.
# ------------------------------------------------------
# It is sufficient to use disabled subscription to test upgrade failure.
$publisher->safe_psql('postgres', "CREATE PUBLICATION regress_pub1");
@@ -52,10 +53,10 @@ $old_sub->safe_psql('postgres',
$old_sub->stop;
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 0");
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 0");
# pg_upgrade will fail because the new cluster has insufficient
-# max_replication_slots.
+# max_active_replication_origins.
command_checks_all(
[
'pg_upgrade',
@@ -72,14 +73,14 @@ command_checks_all(
],
1,
[
- qr/"max_replication_slots" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
+ qr/"max_active_replication_origins" \(0\) must be greater than or equal to the number of subscriptions \(1\) on the old cluster/
],
[qr//],
- 'run of pg_upgrade where the new cluster has insufficient max_replication_slots'
+ 'run of pg_upgrade where the new cluster has insufficient max_active_replication_origins'
);
-# Reset max_replication_slots
-$new_sub->append_conf('postgresql.conf', "max_replication_slots = 10");
+# Reset max_active_replication_origins
+$new_sub->append_conf('postgresql.conf', "max_active_replication_origins = 10");
# Cleanup
$publisher->safe_psql('postgres', "DROP PUBLICATION regress_pub1");
diff --git a/src/include/replication/origin.h b/src/include/replication/origin.h
index 33a7e59ddb0..9cb2248fa9f 100644
--- a/src/include/replication/origin.h
+++ b/src/include/replication/origin.h
@@ -37,6 +37,9 @@ extern PGDLLIMPORT RepOriginId replorigin_session_origin;
extern PGDLLIMPORT XLogRecPtr replorigin_session_origin_lsn;
extern PGDLLIMPORT TimestampTz replorigin_session_origin_timestamp;
+/* GUCs */
+extern PGDLLIMPORT int max_active_replication_origins;
+
/* API for querying & manipulating replication origins */
extern RepOriginId replorigin_by_name(const char *roname, bool missing_ok);
extern RepOriginId replorigin_create(const char *roname);
--
2.43.5
On Wed, Mar 19, 2025 at 10:43 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Mon, Mar 17, 2025 at 6:05 PM Euler Taveira <euler@eulerto.com> wrote:
On Mon, Mar 17, 2025, at 8:44 PM, Masahiko Sawada wrote:
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.Good point. Looking at the documentation, the old max_replication_slots
parameter was the first one in that section so I decided to use the same order
for the postgresql.conf.sample.Thank you for updating the patch!
*
<para>
Logical replication requires several configuration options to be set. Most
- options are relevant only on one side of the replication. However,
- <varname>max_replication_slots</varname> is used on both the publisher and
- the subscriber, but it has a different meaning for each.
+ options are relevant only on one side of the replication.
</para>
In this para, after removing the content about max_replication_slots,
the other line: "Most options are relevant only on one side of the
replication." doesn't make sense because there is no other option that
applies to both sides and if there is one then we should mention about
that.
The patch looks good to me.
Other than the above, the patch looks good to me as well.
--
With Regards,
Amit Kapila.
On Wed, Mar 19, 2025 at 8:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Mar 19, 2025 at 10:43 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Mon, Mar 17, 2025 at 6:05 PM Euler Taveira <euler@eulerto.com> wrote:
On Mon, Mar 17, 2025, at 8:44 PM, Masahiko Sawada wrote:
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.Good point. Looking at the documentation, the old max_replication_slots
parameter was the first one in that section so I decided to use the same order
for the postgresql.conf.sample.Thank you for updating the patch!
* <para> Logical replication requires several configuration options to be set. Most - options are relevant only on one side of the replication. However, - <varname>max_replication_slots</varname> is used on both the publisher and - the subscriber, but it has a different meaning for each. + options are relevant only on one side of the replication. </para>In this para, after removing the content about max_replication_slots,
the other line: "Most options are relevant only on one side of the
replication." doesn't make sense because there is no other option that
applies to both sides and if there is one then we should mention about
that.
Good point. How about the following change?
<para>
- Logical replication requires several configuration options to be set. Most
+ Logical replication requires several configuration options to be set. These
options are relevant only on one side of the replication.
</para>
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Mar 20, 2025 at 10:37 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Wed, Mar 19, 2025 at 8:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Mar 19, 2025 at 10:43 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Mon, Mar 17, 2025 at 6:05 PM Euler Taveira <euler@eulerto.com> wrote:
On Mon, Mar 17, 2025, at 8:44 PM, Masahiko Sawada wrote:
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.Good point. Looking at the documentation, the old max_replication_slots
parameter was the first one in that section so I decided to use the same order
for the postgresql.conf.sample.Thank you for updating the patch!
* <para> Logical replication requires several configuration options to be set. Most - options are relevant only on one side of the replication. However, - <varname>max_replication_slots</varname> is used on both the publisher and - the subscriber, but it has a different meaning for each. + options are relevant only on one side of the replication. </para>In this para, after removing the content about max_replication_slots,
the other line: "Most options are relevant only on one side of the
replication." doesn't make sense because there is no other option that
applies to both sides and if there is one then we should mention about
that.Good point. How about the following change?
<para> - Logical replication requires several configuration options to be set. Most + Logical replication requires several configuration options to be set. These options are relevant only on one side of the replication. </para>
LGTM.
--
With Regards,
Amit Kapila.
On Thu, Mar 20, 2025 at 8:38 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Thu, Mar 20, 2025 at 10:37 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Wed, Mar 19, 2025 at 8:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Mar 19, 2025 at 10:43 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Mon, Mar 17, 2025 at 6:05 PM Euler Taveira <euler@eulerto.com> wrote:
On Mon, Mar 17, 2025, at 8:44 PM, Masahiko Sawada wrote:
I would suggest putting the new max_active_replication_origins after
max_parallel_apply_workers_per_subscription as both
max_sync_workers_per_subscription and
max_parallel_apply_workers_per_subscription are related to
max_logical_replication_workers.Good point. Looking at the documentation, the old max_replication_slots
parameter was the first one in that section so I decided to use the same order
for the postgresql.conf.sample.Thank you for updating the patch!
* <para> Logical replication requires several configuration options to be set. Most - options are relevant only on one side of the replication. However, - <varname>max_replication_slots</varname> is used on both the publisher and - the subscriber, but it has a different meaning for each. + options are relevant only on one side of the replication. </para>In this para, after removing the content about max_replication_slots,
the other line: "Most options are relevant only on one side of the
replication." doesn't make sense because there is no other option that
applies to both sides and if there is one then we should mention about
that.Good point. How about the following change?
<para> - Logical replication requires several configuration options to be set. Most + Logical replication requires several configuration options to be set. These options are relevant only on one side of the replication. </para>LGTM.
Thank you for reviewing it. I've committed the patch.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com