[PATCH] Note effect of max_replication_slots on subscriber side in documentation.
Hey, all,
The configuration parameter max_replication_slots is most notably used
to control how many replication slots can be created on a server, but it
also controls how many replication origins can be tracked on the
subscriber side.
This is noted in the Configuration Settings section in the Logical
Replication Chapter [1]https://www.postgresql.org/docs/13/logical-replication-config.html, but it is not mentioned in the documentation
the parameter itself [2]https://www.postgresql.org/docs/13/runtime-config-replication.html#GUC-MAX-REPLICATION-SLOTS.
The attached patch adds an extra paragraph explaining its effect on
subscribers.
Using max_replication_slots for sizing the number available of
replication origin states is a little odd, and is actually noted twice
in the source code [3]https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/origin.c;h=685eaa6134e7cad193b583ff28284d877a6d8055;hb=HEAD#l162 [4]https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/origin.c;h=685eaa6134e7cad193b583ff28284d877a6d8055;hb=HEAD#l495:
XXX: Should we use a separate variable to size this rather than
max_replication_slots?
XXX: max_replication_slots is arguably the wrong thing to use, as here
we keep the replay state of *remote* transactions. But for now it
seems sufficient to reuse it, rather than introduce a separate GUC.
This is a different usage of max_replication_slots than originally
intended, managing resource usage on the subscriber side, rather than
the provider side. This manifests itself in the awkwardness of the
documentation, where max_replication_slots is only listed in the Sending
Server section, and not mentioned in the Subscribers section.
Given this, I think introducing a new parameter would make sense
(max_replication_origins? slightly confusing because there's no limit on
the number of records in pg_replication_origins; tracking of replication
origins is displayed in pg_replication_origin_status).
I'd be happy to make a patch for a new GUC parameter, if people think
it's worth it to separate the functionality. Until then, however, the
addition to the documentation should help prevent confusion.
- Paul
[1]: https://www.postgresql.org/docs/13/logical-replication-config.html
[2]: https://www.postgresql.org/docs/13/runtime-config-replication.html#GUC-MAX-REPLICATION-SLOTS
[3]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/origin.c;h=685eaa6134e7cad193b583ff28284d877a6d8055;hb=HEAD#l162
[4]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/origin.c;h=685eaa6134e7cad193b583ff28284d877a6d8055;hb=HEAD#l495
Attachments:
max_replication_slots_subscriber_doc_v00.diffapplication/octet-stream; name=max_replication_slots_subscriber_doc_v00.diffDownload
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index f1037df5a9..c2e382e222 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3965,6 +3965,17 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
+
+ <para>
+ On the subscriber side, specifies how many replication origins (see
+ <xref linkend="replication-origins"/>) can be tracked simultaneously,
+ effectively limiting how many logical replication subscriptions can
+ be created on the server. Setting it a lower value than the current
+ number of tracked replication origins (reflected in
+ <link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>,
+ not <link linkend="catalog-pg-replication-origin">pg_replication_origin</link>)
+ will prevent the server from starting.
+ </para>
</listitem>
</varlistentry>
Hey, all,
I went ahead and made a patch for introducing a new GUC variable,
max_replication_origins, to replace the awkward re-use of
max_replication_slots.
I'm mostly indifferent whether a new GUC variable is necessary, or
simply just updating the existing documentation (the first patch I
sent) is sufficient, but one of them should definitely be done to
clear up the confusion.
- Paul
On Tue, Feb 16, 2021 at 1:03 PM Paul Martinez <paulmtz@google.com> wrote:
Show quoted text
Hey, all,
The configuration parameter max_replication_slots is most notably used
to control how many replication slots can be created on a server, but it
also controls how many replication origins can be tracked on the
subscriber side.This is noted in the Configuration Settings section in the Logical
Replication Chapter [1], but it is not mentioned in the documentation
the parameter itself [2].The attached patch adds an extra paragraph explaining its effect on
subscribers.Using max_replication_slots for sizing the number available of
replication origin states is a little odd, and is actually noted twice
in the source code [3] [4]:XXX: Should we use a separate variable to size this rather than
max_replication_slots?XXX: max_replication_slots is arguably the wrong thing to use, as here
we keep the replay state of *remote* transactions. But for now it
seems sufficient to reuse it, rather than introduce a separate GUC.This is a different usage of max_replication_slots than originally
intended, managing resource usage on the subscriber side, rather than
the provider side. This manifests itself in the awkwardness of the
documentation, where max_replication_slots is only listed in the Sending
Server section, and not mentioned in the Subscribers section.Given this, I think introducing a new parameter would make sense
(max_replication_origins? slightly confusing because there's no limit on
the number of records in pg_replication_origins; tracking of replication
origins is displayed in pg_replication_origin_status).I'd be happy to make a patch for a new GUC parameter, if people think
it's worth it to separate the functionality. Until then, however, the
addition to the documentation should help prevent confusion.- Paul
[1]: https://www.postgresql.org/docs/13/logical-replication-config.html
[2]:
https://www.postgresql.org/docs/13/runtime-config-replication.html#GUC-MAX-REPLICATION-SLOTS
[3]:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/origin.c;h=685eaa6134e7cad193b583ff28284d877a6d8055;hb=HEAD#l162
[4]:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/origin.c;h=685eaa6134e7cad193b583ff28284d877a6d8055;hb=HEAD#l495
Attachments:
max_replication_origins_v00.diffapplication/octet-stream; name=max_replication_origins_v00.diffDownload
commit 65258e95db0c461a9b48df17e7be01312a8489ab
Author: Paul Julius Martinez <paulmtz@google.com>
Date: Thu Feb 18 21:55:02 2021 +0000
Add max_replication_origins GUC parameter to replace some usages of max_replication_slots.
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index e81141e45c..07a6fd632c 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4693,6 +4693,25 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
<variablelist>
+ <varlistentry id="guc-max-replication-origins" xreflabel="max_replication_origins">
+ <term><varname>max_replication_origins</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>max_replication_origins</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies how many replication origins (see <xref linkend="replication-origins"/>) can
+ be tracked simultaneously, effectively limiting how many logical replication subscriptions
+ can be created on the server. Setting it a lower value than the current
+ number of tracked replication origins (reflected in
+ <link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>,
+ not <link linkend="catalog-pg-replication-origin">pg_replication_origin</link>)
+ will prevent the server from starting.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-max-logical-replication-workers" xreflabel="max_logical_replication_workers">
<term><varname>max_logical_replication_workers</varname> (<type>int</type>)
<indexterm>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index d0742f2c52..9b754a4d17 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -602,7 +602,7 @@
</para>
<para>
- The subscriber also requires the <varname>max_replication_slots</varname>
+ The subscriber also requires the <varname>max_replication_origins</varname>
to be set. In this case it should be set to at least the number of
subscriptions that will be added to the subscriber.
<varname>max_logical_replication_workers</varname> must be set to at
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index cb462a052a..efec1e0115 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -32,6 +32,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
+#include "replication/logical.h"
#include "replication/logicallauncher.h"
#include "replication/logicalworker.h"
#include "replication/slot.h"
@@ -286,10 +287,10 @@ logicalrep_worker_launch(Oid dbid, Oid subid, const char *subname, Oid userid,
subname)));
/* Report this after the initial starting message for consistency. */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("cannot start logical replication workers when max_replication_slots = 0")));
+ errmsg("cannot start logical replication workers when max_replication_origins = 0")));
/*
* We need to do the modification of the shared memory under lock so that
diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication/logical/origin.c
index 685eaa6134..3c6f0d9826 100644
--- a/src/backend/replication/logical/origin.c
+++ b/src/backend/replication/logical/origin.c
@@ -146,7 +146,7 @@ typedef struct ReplicationStateCtl
{
/* Tranche to use for per-origin LWLocks */
int tranche_id;
- /* Array of length max_replication_slots */
+ /* Array of length max_replication_origins */
ReplicationState states[FLEXIBLE_ARRAY_MEMBER];
} ReplicationStateCtl;
@@ -155,12 +155,13 @@ RepOriginId replorigin_session_origin = InvalidRepOriginId; /* assumed identity
XLogRecPtr replorigin_session_origin_lsn = InvalidXLogRecPtr;
TimestampTz replorigin_session_origin_timestamp = 0;
+/* GUCs */
+/* The maximum number of replication origins that can be tracked simultaneously. */
+int max_replication_origins = 0;
+
/*
* Base address into a shared memory array of replication states of size
- * max_replication_slots.
- *
- * XXX: Should we use a separate variable to size this rather than
- * max_replication_slots?
+ * max_replication_origins.
*/
static ReplicationState *replication_states;
@@ -182,10 +183,10 @@ static ReplicationState *session_replication_state = NULL;
static void
replorigin_check_prerequisites(bool check_slots, bool recoveryOK)
{
- if (check_slots && max_replication_slots == 0)
+ if (check_slots && max_replication_origins == 0)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot query or manipulate replication origin when max_replication_slots = 0")));
+ errmsg("cannot query or manipulate replication origin when max_replication_origins = 0")));
if (!recoveryOK && RecoveryInProgress())
ereport(ERROR,
@@ -338,7 +339,7 @@ restart:
tuple = NULL;
LWLockAcquire(ReplicationOriginLock, LW_EXCLUSIVE);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -491,18 +492,13 @@ ReplicationOriginShmemSize(void)
{
Size size = 0;
- /*
- * XXX: max_replication_slots is arguably the wrong thing to use, as here
- * we keep the replay state of *remote* transactions. But for now it seems
- * sufficient to reuse it, rather than introduce a separate GUC.
- */
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return size;
size = add_size(size, offsetof(ReplicationStateCtl, states));
size = add_size(size,
- mul_size(max_replication_slots, sizeof(ReplicationState)));
+ mul_size(max_replication_origins, sizeof(ReplicationState)));
return size;
}
@@ -511,7 +507,7 @@ ReplicationOriginShmemInit(void)
{
bool found;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
replication_states_ctl = (ReplicationStateCtl *)
@@ -528,7 +524,7 @@ ReplicationOriginShmemInit(void)
replication_states_ctl->tranche_id = LWTRANCHE_REPLICATION_ORIGIN_STATE;
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
LWLockInitialize(&replication_states[i].lock,
replication_states_ctl->tranche_id);
@@ -550,7 +546,7 @@ ReplicationOriginShmemInit(void)
*
* So its just the magic, followed by the statically sized
* ReplicationStateOnDisk structs. Note that the maximum number of
- * ReplicationState is determined by max_replication_slots.
+ * ReplicationState is determined by max_replication_origins.
* ---------------------------------------------------------------------------
*/
void
@@ -563,7 +559,7 @@ CheckPointReplicationOrigin(void)
uint32 magic = REPLICATION_STATE_MAGIC;
pg_crc32c crc;
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -605,7 +601,7 @@ CheckPointReplicationOrigin(void)
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
/* write actual data */
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationStateOnDisk disk_state;
ReplicationState *curstate = &replication_states[i];
@@ -698,7 +694,7 @@ StartupReplicationOrigin(void)
already_started = true;
#endif
- if (max_replication_slots == 0)
+ if (max_replication_origins == 0)
return;
INIT_CRC32C(crc);
@@ -708,7 +704,7 @@ StartupReplicationOrigin(void)
fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
/*
- * might have had max_replication_slots == 0 last run, or we just brought
+ * might have had max_replication_origins == 0 last run, or we just brought
* up a standby.
*/
if (fd < 0 && errno == ENOENT)
@@ -776,10 +772,10 @@ StartupReplicationOrigin(void)
COMP_CRC32C(crc, &disk_state, sizeof(disk_state));
- if (last_state == max_replication_slots)
+ if (last_state == max_replication_origins)
ereport(PANIC,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("could not find free replication state, increase max_replication_slots")));
+ errmsg("could not find free replication state, increase max_replication_origins")));
/* copy data to shared memory */
replication_states[last_state].roident = disk_state.roident;
@@ -833,7 +829,7 @@ replorigin_redo(XLogReaderState *record)
xlrec = (xl_replorigin_drop *) XLogRecGetData(record);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state = &replication_states[i];
@@ -898,7 +894,7 @@ replorigin_advance(RepOriginId node,
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -939,7 +935,7 @@ replorigin_advance(RepOriginId node,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with OID %u",
node),
- errhint("Increase max_replication_slots and try again.")));
+ errhint("Increase max_replication_origins and try again.")));
if (replication_state == NULL)
{
@@ -1002,7 +998,7 @@ replorigin_get_progress(RepOriginId node, bool flush)
/* prevent slots from being concurrently dropped */
LWLockAcquire(ReplicationOriginLock, LW_SHARED);
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
@@ -1078,7 +1074,7 @@ replorigin_session_setup(RepOriginId node)
registered_cleanup = true;
}
- Assert(max_replication_slots > 0);
+ Assert(max_replication_origins > 0);
if (session_replication_state != NULL)
ereport(ERROR,
@@ -1092,7 +1088,7 @@ replorigin_session_setup(RepOriginId node)
* Search for either an existing slot for the origin, or a free one we can
* use.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *curstate = &replication_states[i];
@@ -1126,7 +1122,7 @@ replorigin_session_setup(RepOriginId node)
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("could not find free replication state slot for replication origin with OID %u",
node),
- errhint("Increase max_replication_slots and try again.")));
+ errhint("Increase max_replication_origins and try again.")));
else if (session_replication_state == NULL)
{
/* initialize new slot */
@@ -1158,7 +1154,7 @@ replorigin_session_reset(void)
{
ConditionVariable *cv;
- Assert(max_replication_slots != 0);
+ Assert(max_replication_origins != 0);
if (session_replication_state == NULL)
ereport(ERROR,
@@ -1522,7 +1518,7 @@ pg_show_replication_origin_status(PG_FUNCTION_ARGS)
* filled. Note that we do not take any locks, so slightly corrupted/out
* of date values are a possibility.
*/
- for (i = 0; i < max_replication_slots; i++)
+ for (i = 0; i < max_replication_origins; i++)
{
ReplicationState *state;
Datum values[REPLICATION_ORIGIN_PROGRESS_COLS];
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 00018abb7d..9436057332 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -69,6 +69,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
#include "postmaster/walwriter.h"
+#include "replication/logical.h"
#include "replication/logicallauncher.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -3005,6 +3006,16 @@ static struct config_int ConfigureNamesInt[] =
check_max_worker_processes, NULL, NULL
},
+ {
+ {"max_replication_origins", PGC_POSTMASTER, REPLICATION_SUBSCRIBERS,
+ gettext_noop("Sets the maximum number of replication origins that can be tracked simultaneously."),
+ NULL
+ },
+ &max_replication_origins,
+ 4, 0, MAX_BACKENDS,
+ NULL, NULL, NULL
+ },
+
{
{"max_logical_replication_workers",
PGC_POSTMASTER,
diff --git a/src/include/replication/logical.h b/src/include/replication/logical.h
index c253403372..3512651c04 100644
--- a/src/include/replication/logical.h
+++ b/src/include/replication/logical.h
@@ -14,6 +14,9 @@
#include "replication/output_plugin.h"
#include "replication/slot.h"
+/* GUCs */
+extern PGDLLIMPORT int max_replication_origins;
+
struct LogicalDecodingContext;
typedef void (*LogicalOutputPluginWriterWrite) (struct LogicalDecodingContext *lr,
On Thu, Feb 25, 2021 at 2:19 AM Paul Martinez <paulmtz@google.com> wrote:
Hey, all,
I went ahead and made a patch for introducing a new GUC variable,
max_replication_origins, to replace the awkward re-use of
max_replication_slots.I'm mostly indifferent whether a new GUC variable is necessary, or
simply just updating the existing documentation (the first patch I
sent) is sufficient, but one of them should definitely be done to
clear up the confusion.
+1. I also think one of them is required. I think users who are using
cascaded replication (means subscribers are also publishers), setting
this parameter might be a bit confusing and difficult. Anybody else
has an opinion on this matter?
For docs only patch, I have few suggestions:
1. On page [1], it is not very clear that we are suggesting to set
max_replication_slots for origins whereas your new doc patch has
clarified it, can we update the other page as well.
2.
Setting it a lower value than the current
+ number of tracked replication origins (reflected in
+ <link
linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>,
+ not <link
linkend="catalog-pg-replication-origin">pg_replication_origin</link>)
+ will prevent the server from starting.
+ </para>Why can't we just mention pg_replication_origin above?
[1]: https://www.postgresql.org/docs/13/logical-replication-config.html
--
With Regards,
Amit Kapila.
On Thu, Feb 25, 2021 at 5:31 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
For docs only patch, I have few suggestions:
1. On page [1], it is not very clear that we are suggesting to set
max_replication_slots for origins whereas your new doc patch has
clarified it, can we update the other page as well.
Sorry, what other page are you referring to?
2. Setting it a lower value than the current + number of tracked replication origins (reflected in + <link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>, + not <link linkend="catalog-pg-replication-origin">pg_replication_origin</link>) + will prevent the server from starting. + </para>Why can't we just mention pg_replication_origin above?
So this is slightly confusing:
pg_replication_origin just contains mappings from origin names to oids.
It is regular catalog table and has no limit on its size. Users can also
manually insert rows into this table.
https://www.postgresql.org/docs/13/catalog-pg-replication-origin.html
The view showing the in-memory information is actually
pg_replication_origin_status. The number of entries here is what is
actually constrained by the GUC parameter.
https://www.postgresql.org/docs/13/view-pg-replication-origin-status.html
I clarified pointing to pg_replication_origin_status because it could in
theory be out of sync with pg_replication_origin. I'm actually not sure
how entries there are managed. Perhaps if you were replicating from one
database and then stopped and started replicating from another database
you'd have two replication origins, but only one replication origin
status?
This also brings up a point regarding the naming of the added GUC.
max_replication_origins is cleanest, but has this confusion regarding
pg_replication_origin vs. pg_replication_origin_status.
max_replication_origin_statuses is weird (and long).
max_tracked_replication_origins is a possibility?
(One last bit of naming confusion; the internal code refers to them as
ReplicationStates, rather than ReplicationOrigins or
ReplicationOriginStatuses, or something like that.)
- Paul
On Fri, Feb 26, 2021 at 1:53 AM Paul Martinez <paulmtz@google.com> wrote:
On Thu, Feb 25, 2021 at 5:31 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
For docs only patch, I have few suggestions:
1. On page [1], it is not very clear that we are suggesting to set
max_replication_slots for origins whereas your new doc patch has
clarified it, can we update the other page as well.Sorry, what other page are you referring to?
https://www.postgresql.org/docs/devel/logical-replication-config.html
2. Setting it a lower value than the current + number of tracked replication origins (reflected in + <link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>, + not <link linkend="catalog-pg-replication-origin">pg_replication_origin</link>) + will prevent the server from starting. + </para>Why can't we just mention pg_replication_origin above?
So this is slightly confusing:
pg_replication_origin just contains mappings from origin names to oids.
It is regular catalog table and has no limit on its size. Users can also
manually insert rows into this table.https://www.postgresql.org/docs/13/catalog-pg-replication-origin.html
The view showing the in-memory information is actually
pg_replication_origin_status. The number of entries here is what is
actually constrained by the GUC parameter.
Okay, that makes sense. However, I have sent a patch today (see [1]/messages/by-id/CAA4eK1KkbppndxxRKbaT2sXrLkdPwy44F4pjEZ0EDrVjD9MPjQ@mail.gmail.com)
where I have slightly updated the subscriber-side configuration
paragraph. From PG-14 onwards, table synchronization workers also use
origins on subscribers, so you might want to adjust.
This also brings up a point regarding the naming of the added GUC.
max_replication_origins is cleanest, but has this confusion regarding
pg_replication_origin vs. pg_replication_origin_status.
max_replication_origin_statuses is weird (and long).
max_tracked_replication_origins is a possibility?
or maybe max_replication_origin_states. I guess we can leave adding
GUC to some other day as that might require a bit broader acceptance
and we are already near to the start of last CF. I think we can still
consider it if we few more people share the same opinion as yours.
[1]: /messages/by-id/CAA4eK1KkbppndxxRKbaT2sXrLkdPwy44F4pjEZ0EDrVjD9MPjQ@mail.gmail.com
--
With Regards,
Amit Kapila.
On Fri, Feb 26, 2021 at 5:22 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
https://www.postgresql.org/docs/devel/logical-replication-config.html
Ah, yep. I added a clause to the end of the sentence to clarify why we're
using max_replication_slots here:
- The subscriber also requires the max_replication_slots to be set.
+ The subscriber also requires that max_replication_slots be set to
+ configure how many replication origins can be tracked.Okay, that makes sense. However, I have sent a patch today (see [1])
where I have slightly updated the subscriber-side configuration
paragraph. From PG-14 onwards, table synchronization workers also use
origins on subscribers, so you might want to adjust....
I guess we can leave adding GUC to some other day as that might
require a bit broader acceptance and we are already near to the start
of last CF. I think we can still consider it if we few more people
share the same opinion as yours.
Great. I'll wait to update the GUC patch until your patch and/or my
doc-only patch get merged. Should I add it to the March CF?
Separate question: are documentation updates like these ever backported
to older versions that are still supported? And if so, would the changes
be reflected immediately, or would they require a minor point release?
When I was on an older release I found that I'd jump back and forth
between the version I was using and the latest version to see if
anything had changed.
- Paul
Attachments:
max_replication_slots_subscriber_doc_v01.diffapplication/octet-stream; name=max_replication_slots_subscriber_doc_v01.diffDownload
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index e81141e45c..ff0df2d5c6 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4001,6 +4001,17 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
+
+ <para>
+ On the subscriber side, specifies how many replication origins (see
+ <xref linkend="replication-origins"/>) can be tracked simultaneously,
+ effectively limiting how many logical replication subscriptions can
+ be created on the server. Setting it a lower value than the current
+ number of tracked replication origins (reflected in
+ <link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>,
+ not <link linkend="catalog-pg-replication-origin">pg_replication_origin</link>)
+ will prevent the server from starting.
+ </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index d0742f2c52..c025249a8c 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -602,8 +602,9 @@
</para>
<para>
- The subscriber also requires the <varname>max_replication_slots</varname>
- to be set. In this case it should be set to at least the number of
+ The subscriber also requires that <varname>max_replication_slots</varname>
+ be set to configure how many replication origins can be tracked.
+ In this case it should be set to at least the number of
subscriptions that will be added to the subscriber.
<varname>max_logical_replication_workers</varname> must be set to at
least the number of subscriptions, again plus some reserve for the table
On Sat, Feb 27, 2021 at 2:47 AM Paul Martinez <paulmtz@google.com> wrote:
On Fri, Feb 26, 2021 at 5:22 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
https://www.postgresql.org/docs/devel/logical-replication-config.html
Ah, yep. I added a clause to the end of the sentence to clarify why we're
using max_replication_slots here:- The subscriber also requires the max_replication_slots to be set.
+ The subscriber also requires that max_replication_slots be set to + configure how many replication origins can be tracked.
LGTM.
Okay, that makes sense. However, I have sent a patch today (see [1])
where I have slightly updated the subscriber-side configuration
paragraph. From PG-14 onwards, table synchronization workers also use
origins on subscribers, so you might want to adjust....
I guess we can leave adding GUC to some other day as that might
require a bit broader acceptance and we are already near to the start
of last CF. I think we can still consider it if we few more people
share the same opinion as yours.Great. I'll wait to update the GUC patch until your patch and/or my
doc-only patch get merged. Should I add it to the March CF?
Which patch are you asking about doc-patch or GUC one? If you are
asking for a doc-patch, then I don't think it is required, I'll take
care of this sometime next week. For the GUC patch, my suggestion
would be to propose for v15 with an appropriate use-case. At this
point (just before the last CF of release), people are mostly busy
with patches that are going on for a long time so this might not get
due attention unless few people show-up and say it is important.
However, it is up to you, if you want feel free to register your GUC
patch in the upcoming CF.
Separate question: are documentation updates like these ever backported
to older versions that are still supported?
Not every doc-change is back-ported but I think it is good to backport
the user-visible ones. It is on a case-by-case basis. For this, I
think we can backport unless you or others feel otherwise?
And if so, would the changes
be reflected immediately, or would they require a minor point release?
Where you are referring to the docs? If you are checking from code, it
will be reflected immediately.
--
With Regards,
Amit Kapila.
On Sat, Feb 27, 2021 at 2:35 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Sat, Feb 27, 2021 at 2:47 AM Paul Martinez <paulmtz@google.com> wrote:
On Fri, Feb 26, 2021 at 5:22 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
https://www.postgresql.org/docs/devel/logical-replication-config.html
Ah, yep. I added a clause to the end of the sentence to clarify why we're
using max_replication_slots here:- The subscriber also requires the max_replication_slots to be set.
+ The subscriber also requires that max_replication_slots be set to + configure how many replication origins can be tracked.LGTM.
The rebased version attached. As mentioned earlier, I think we can
backpatch this patch as this clarifies the already existing behavior.
Do let me know if you or others think otherwise?
--
With Regards,
Amit Kapila.
Attachments:
max_replication_slots_subscriber_doc_v02.patchapplication/octet-stream; name=max_replication_slots_subscriber_doc_v02.patchDownload
From 6283725f5ea91d0a153b840ff40823cd88d5dbf6 Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Mon, 1 Mar 2021 17:08:50 +0530
Subject: [PATCH v1] Clarify the usage of max_replication_slots on the
subscriber side.
It was not clear in the docs that the max_replication_slots is also used
to track replication origins on the subscriber side.
Author: Paul Martinez
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CACqFVBZgwCN_pHnW6dMNCrOS7tiHCw6Retf_=U2Vvj3aUSeATw@mail.gmail.com
---
doc/src/sgml/config.sgml | 11 +++++++++++
doc/src/sgml/logical-replication.sgml | 11 ++++++-----
2 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index b5718fc..967de73 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4001,6 +4001,17 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
to <literal>replica</literal> or higher to allow replication slots to
be used.
</para>
+
+ <para>
+ On the subscriber side, specifies how many replication origins (see
+ <xref linkend="replication-origins"/>) can be tracked simultaneously,
+ effectively limiting how many logical replication subscriptions can
+ be created on the server. Setting it a lower value than the current
+ number of tracked replication origins (reflected in
+ <link linkend="view-pg-replication-origin-status">pg_replication_origin_status</link>,
+ not <link linkend="catalog-pg-replication-origin">pg_replication_origin</link>)
+ will prevent the server from starting.
+ </para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml
index 9cd3f3f..e95d446 100644
--- a/doc/src/sgml/logical-replication.sgml
+++ b/doc/src/sgml/logical-replication.sgml
@@ -603,11 +603,12 @@
<para>
The subscriber also requires the <varname>max_replication_slots</varname>
- to be set. In this case it should be set to at least the number of
- subscriptions that will be added to the subscriber, plus some reserve for
- table synchronization. <varname>max_logical_replication_workers</varname>
- must be set to at least the number of subscriptions, again plus some reserve
- for the table synchronization. Additionally the <varname>max_worker_processes</varname>
+ be set to configure how many replication origins can be tracked. In this
+ case it should be set to at least the number of subscriptions that will be
+ added to the subscriber, plus some reserve for table synchronization.
+ <varname>max_logical_replication_workers</varname> must be set to at least
+ the number of subscriptions, again plus some reserve for the table
+ synchronization. Additionally the <varname>max_worker_processes</varname>
may need to be adjusted to accommodate for replication workers, at least
(<varname>max_logical_replication_workers</varname>
+ <literal>1</literal>). Note that some extensions and parallel queries
--
1.8.3.1
On Mon, Mar 1, 2021 at 5:32 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Sat, Feb 27, 2021 at 2:35 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Sat, Feb 27, 2021 at 2:47 AM Paul Martinez <paulmtz@google.com> wrote:
On Fri, Feb 26, 2021 at 5:22 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
https://www.postgresql.org/docs/devel/logical-replication-config.html
Ah, yep. I added a clause to the end of the sentence to clarify why we're
using max_replication_slots here:- The subscriber also requires the max_replication_slots to be set.
+ The subscriber also requires that max_replication_slots be set to + configure how many replication origins can be tracked.LGTM.
The rebased version attached.
Pushed!
--
With Regards,
Amit Kapila.