Doc about how to set max_wal_senders when setting minimal wal_level

On Wed, Mar 2, 2022 at 7:44 PM Japin Li <japinli@hotmail.com> wrote:

Hi, hackers

When I try to change wal_level to minimal and restart the database, it
complains
max_wal_senders > 0.

2022-03-03 10:10:16.938 CST [6389] FATAL: WAL streaming (max_wal_senders

0) requires wal_level "replica" or "logical"

However, the documentation about wal_level [1] doesn't mentation this.
How about adding a sentence to describe how to set max_wal_senders when
setting wal_level to minimal?

[1] https://www.postgresql.org/docs/devel/runtime-config-wal.html

I would suggest a wording more like:

"A precondition for using minimal WAL is to disable WAL archiving and
streaming replication by setting max_wal_senders to 0, and archive_mode to
off."

While accurate, the phrase "you must set" just doesn't feel right to me. I
also don't like how the proposed sentence (either one) is added separately
as opposed to being included in the immediately preceding paragraph. While
this limited patch is probably sufficient I would suggest trying to work
out a slightly larger patch the improves the wording on the entire existing
paragraph while incorporating the reference to max_wal_senders.

Note, we seem to be missing the documentation of the default setting for
archive_mode.

In addition, max_wal_senders could also be changed, adding a sentence like:

"If setting max_wal_senders to 0 consider also reducing the amount of WAL
produced by changing wal_level to minimal."

At least insofar as core is concerned without a wal sender the additional
wal of replica is not actually able to be leveraged as pg_basebackup will
not work (at noted in its own description).

David J.

On Thu, 03 Mar 2022 at 11:25, David G. Johnston <david.g.johnston@gmail.com> wrote:

I would suggest a wording more like:

"A precondition for using minimal WAL is to disable WAL archiving and
streaming replication by setting max_wal_senders to 0, and archive_mode to
off."

While accurate, the phrase "you must set" just doesn't feel right to me. I
also don't like how the proposed sentence (either one) is added separately
as opposed to being included in the immediately preceding paragraph. While
this limited patch is probably sufficient I would suggest trying to work
out a slightly larger patch the improves the wording on the entire existing
paragraph while incorporating the reference to max_wal_senders.

Thanks for your review. Modified as your suggestion.

Note, we seem to be missing the documentation of the default setting for
archive_mode.

Add the default value for archive_mode.

In addition, max_wal_senders could also be changed, adding a sentence like:

"If setting max_wal_senders to 0 consider also reducing the amount of WAL
produced by changing wal_level to minimal."

Modified.

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:

On Thu, 03 Mar 2022 at 11:25, David G. Johnston <david.g.johnston@gmail.com> wrote:

I would suggest a wording more like:

"A precondition for using minimal WAL is to disable WAL archiving and
streaming replication by setting max_wal_senders to 0, and archive_mode to
off."

While accurate, the phrase "you must set" just doesn't feel right to me. I
also don't like how the proposed sentence (either one) is added separately
as opposed to being included in the immediately preceding paragraph. While
this limited patch is probably sufficient I would suggest trying to work
out a slightly larger patch the improves the wording on the entire existing
paragraph while incorporating the reference to max_wal_senders.

Thanks for your review. Modified as your suggestion.

Note, we seem to be missing the documentation of the default setting for
archive_mode.

Add the default value for archive_mode.

In addition, max_wal_senders could also be changed, adding a sentence like:

"If setting max_wal_senders to 0 consider also reducing the amount of WAL
produced by changing wal_level to minimal."

Modified.

Attach v3 patch to fix missing close varname tag.

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

At Fri, 04 Mar 2022 12:18:29 +0800, Japin Li <japinli@hotmail.com> wrote in

On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:

Attach v3 patch to fix missing close varname tag.

+        A precondition for using minimal WAL is to disable WAL archiving and
+        streaming replication by setting <xref linkend="guc-max-wal-senders"/>
+        to <literal>0</literal>, and <varname>archive_mode</varname>
+        to <literal>off</literal>.

It is a bit odd that the features to stop and the corresponding GUCs
are written irrespectively. It would be better they're in the same
order.

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL produced
+        by changing <varname>wal_level</varname> to <literal>minimal</literal>.

Those who anively follow this suggestion may bump into failure when
arhive_mode is on. Thus archive_mode is also worth referred to. But,
anyway, IMHO, it is mere a performance tips that is not necessarily
required in this section, or even in this documentaiotn. Addtion to
that, if we write this for max_wal_senders, archive_mode will deserve
the similar tips but I think it is too verbose. In short, I think I
would not add that description at all.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

On Fri, 04 Mar 2022 at 14:05, Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:

At Fri, 04 Mar 2022 12:18:29 +0800, Japin Li <japinli@hotmail.com> wrote in

On Thu, 03 Mar 2022 at 12:10, Japin Li <japinli@hotmail.com> wrote:

Attach v3 patch to fix missing close varname tag.

+        A precondition for using minimal WAL is to disable WAL archiving and
+        streaming replication by setting <xref linkend="guc-max-wal-senders"/>
+        to <literal>0</literal>, and <varname>archive_mode</varname>
+        to <literal>off</literal>.

It is a bit odd that the features to stop and the corresponding GUCs
are written irrespectively. It would be better they're in the same
order.

Thanks for your review. Modified.

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL produced
+        by changing <varname>wal_level</varname> to <literal>minimal</literal>.

Those who anively follow this suggestion may bump into failure when
arhive_mode is on. Thus archive_mode is also worth referred to. But,
anyway, IMHO, it is mere a performance tips that is not necessarily
required in this section, or even in this documentaiotn. Addtion to
that, if we write this for max_wal_senders, archive_mode will deserve
the similar tips but I think it is too verbose. In short, I think I
would not add that description at all.

It already has a tip about wal_level for archive_mode [1]https://www.postgresql.org/docs/devel/runtime-config-wal.html#GUC-ARCHIVE-MODE, IIUC.

archive_mode cannot be enabled when wal_level is set to minimal.

[1]: https://www.postgresql.org/docs/devel/runtime-config-wal.html#GUC-ARCHIVE-MODE

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

On Thu, Mar 3, 2022 at 11:05 PM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:

But,
anyway, IMHO, it is mere a performance tips that is not necessarily
required in this section, or even in this documentaiotn. Addtion to
that, if we write this for max_wal_senders, archive_mode will deserve
the similar tips but I think it is too verbose. In short, I think I
would not add that description at all.

I wrote it as a performance tip but it is documenting that when set to 0 no
features of the server require more information than is captured in the
minimal wal. That fact seems worthy of noting. Even at the cost of a bit
of verbosity. These features interact with each other and that interaction
should be adequately described. While subjective, this dynamic seems to
warrant inclusion.

David J.

On Fri, Mar 4, 2022 at 2:49 AM Japin Li <japinli@hotmail.com> wrote:

Thanks for your review. Modified.

Works for me. I have some additional sparks of ideas but nothing that need
hold this up.

David J.

Japin Li <japinli@hotmail.com> writes:

[ v4-wal-level-documentation.patch ]

Hm, I don't care for the wording here:

+        A precondition for using minimal WAL is to disable WAL archiving and
+        streaming replication by setting <varname>archive_mode</varname> to
+        <literal>off</literal>, and <xref linkend="guc-max-wal-senders"/> to
+        <literal>0</literal>.

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL produced
+        by changing <varname>wal_level</varname> to <literal>minimal</literal>.

I don't think this is great advice. It will encourage people to use
wal_level = minimal even if they have other requirements that weigh
against it. If they feel that their system is producing too much
WAL, I doubt they'll have a hard time finding the wal_level knob.

TBH, I think the real problem with the docs in this area is that
the first para about wal_level is disjointed and unclear; we ought
to nuke and rewrite that. In particular it fails to provide adequate
context about what "logical decoding" means. I think possibly what
it needs to say is that replica mode supports *physical* replication
but if you want to use *logical* replication you need logical mode;
if you need neither, and are not doing WAL archiving either, you
can get away with minimal mode.

regards, tom lane

On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote:

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

I have created the attached patch to try to improve this text.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson

Sorry for the late reply.

On Wed, 06 Jul 2022 at 08:02, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Japin Li <japinli@hotmail.com> writes:

[ v4-wal-level-documentation.patch ]

Hm, I don't care for the wording here:

+        A precondition for using minimal WAL is to disable WAL archiving and
+        streaming replication by setting <varname>archive_mode</varname> to
+        <literal>off</literal>, and <xref linkend="guc-max-wal-senders"/> to
+        <literal>0</literal>.

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

Yeah, it's the former case.

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL produced
+        by changing <varname>wal_level</varname> to <literal>minimal</literal>.

I don't think this is great advice. It will encourage people to use
wal_level = minimal even if they have other requirements that weigh
against it. If they feel that their system is producing too much
WAL, I doubt they'll have a hard time finding the wal_level knob.

Agreed. It isn't good advice. We can remove the suggestion.

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote:

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

I have created the attached patch to try to improve this text.

IMO we can add the following sentence for wal_level description, since
if wal_level = minimal and max_wal_senders > 0, we cannot start the database.

To set wal_level to minimal, you must also set max_wal_senders to 0,
which has the effect of disabling both WAL archiving and streaming
replication.

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

On Fri, Jul 15, 2022 at 6:27 AM Japin Li <japinli@hotmail.com> wrote:

+        servers.  If setting <varname>max_wal_senders</varname> to
+        <literal>0</literal> consider also reducing the amount of WAL

produced

+ by changing <varname>wal_level</varname> to

<literal>minimal</literal>.

I don't think this is great advice. It will encourage people to use
wal_level = minimal even if they have other requirements that weigh
against it. If they feel that their system is producing too much
WAL, I doubt they'll have a hard time finding the wal_level knob.

Agreed. It isn't good advice. We can remove the suggestion.

Yeah, I wrote that thinking that max_wal_senders being set to 0 implied
archive_mode = off, but that isn't the case.

David J.

On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:

On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote:

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

I have created the attached patch to try to improve this text.

IMO we can add the following sentence for wal_level description, since
if wal_level = minimal and max_wal_senders > 0, we cannot start the database.

To set wal_level to minimal, you must also set max_wal_senders to 0,
which has the effect of disabling both WAL archiving and streaming
replication.

Okay, text added in the attached patch.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson

On Tue, 19 Jul 2022 at 03:58, Bruce Momjian <bruce@momjian.us> wrote:

On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:

On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote:

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to minimal,
you must also set [these variables], which has the effect of disabling
both WAL archiving and streaming replication."

I have created the attached patch to try to improve this text.

IMO we can add the following sentence for wal_level description, since
if wal_level = minimal and max_wal_senders > 0, we cannot start the database.

To set wal_level to minimal, you must also set max_wal_senders to 0,
which has the effect of disabling both WAL archiving and streaming
replication.

Okay, text added in the attached patch.

Thanks for updating the patch! LGTM.

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:

On Tue, 19 Jul 2022 at 03:58, Bruce Momjian <bruce@momjian.us> wrote:

On Fri, Jul 15, 2022 at 09:29:20PM +0800, Japin Li wrote:

On Fri, 15 Jul 2022 at 08:49, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Jul 5, 2022 at 08:02:33PM -0400, Tom Lane wrote:

"Precondition" is an overly fancy word that makes things less clear
not more so. Does it mean that setting wal_level = minimal will fail
if you don't do these other things, or does it just mean that you
won't be getting the absolute minimum WAL volume? If the former,
I think it'd be better to say something like "To set wal_level to

minimal,

you must also set [these variables], which has the effect of

disabling

both WAL archiving and streaming replication."

I have created the attached patch to try to improve this text.

IMO we can add the following sentence for wal_level description, since
if wal_level = minimal and max_wal_senders > 0, we cannot start the

database.

To set wal_level to minimal, you must also set max_wal_senders to 0,
which has the effect of disabling both WAL archiving and streaming
replication.

Okay, text added in the attached patch.

Thanks for updating the patch! LGTM.

+0.90

Consider changing:

"makes any base backups taken before this unusable"

to:

"makes existing base backups unusable"

As I try to justify this, though, it isn't quite true, maybe:

"makes point-in-time recovery, using existing base backups, unable to
replay future WAL."

David J.

On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:

On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:

+0.90

Consider changing:

"makes any base backups taken before this unusable"

to:

"makes existing base backups unusable"

As I try to justify this, though, it isn't quite true, maybe:

"makes point-in-time recovery, using existing base backups, unable to replay
future WAL."

I went with simpler wording.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson

On Mon, Jul 18, 2022 at 8:16 PM Bruce Momjian <bruce@momjian.us> wrote:

On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:

On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:

+0.90

Consider changing:

"makes any base backups taken before this unusable"

to:

"makes existing base backups unusable"

As I try to justify this, though, it isn't quite true, maybe:

"makes point-in-time recovery, using existing base backups, unable to

replay

future WAL."

I went with simpler wording.

+1

Thanks!

David J.

On Mon, Jul 18, 2022 at 11:16:15PM -0400, Bruce Momjian wrote:

On Mon, Jul 18, 2022 at 07:39:55PM -0700, David G. Johnston wrote:

On Mon, Jul 18, 2022 at 6:27 PM Japin Li <japinli@hotmail.com> wrote:

+0.90

Consider changing:

"makes any base backups taken before this unusable"

to:

"makes existing base backups unusable"

As I try to justify this, though, it isn't quite true, maybe:

"makes point-in-time recovery, using existing base backups, unable to replay
future WAL."

I went with simpler wording.

Patch applied back to PG 14, and partial to PG 13. Docs before that
were too different to be safe.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson