max_worker_processes on the standby
Hi,
"25.5.3. Administrator's Overview" in the document
-----------------------------------------------------
The setting of some parameters on the standby will need reconfiguration
if they have been changed on the primary. For these parameters,
the value on the standby must be equal to or greater than the value
on the primary. If these parameters are not set high enough then
the standby will refuse to start. Higher values can then be supplied
and the server restarted to begin recovery again. These parameters are:
max_connections
max_prepared_transactions
max_locks_per_transaction
-----------------------------------------------------
I found that the value of max_worker_processes on the standby also
must be equal to or greater than the value on the master. So we should
just add max_worker_processes to this paragraph. Patch attached.
Regards,
--
Fujii Masao
Attachments:
doc.patchtext/x-patch; charset=US-ASCII; name=doc.patchDownload
*** a/doc/src/sgml/high-availability.sgml
--- b/doc/src/sgml/high-availability.sgml
***************
*** 2035,2040 **** LOG: database system is ready to accept read only connections
--- 2035,2045 ----
<varname>max_locks_per_transaction</>
</para>
</listitem>
+ <listitem>
+ <para>
+ <varname>max_worker_processes</>
+ </para>
+ </listitem>
</itemizedlist>
</para>
Fujii Masao wrote:
Hi,
"25.5.3. Administrator's Overview" in the document
-----------------------------------------------------
The setting of some parameters on the standby will need reconfiguration
if they have been changed on the primary. For these parameters,
the value on the standby must be equal to or greater than the value
on the primary. If these parameters are not set high enough then
the standby will refuse to start. Higher values can then be supplied
and the server restarted to begin recovery again. These parameters are:
max_connections
max_prepared_transactions
max_locks_per_transaction
-----------------------------------------------------I found that the value of max_worker_processes on the standby also
must be equal to or greater than the value on the master. So we should
just add max_worker_processes to this paragraph. Patch attached.
True. Also track_commit_timestamp. Can you add a comment somewhere in
CheckRequiredParameterValues(void) that the set of parameters is listed
in section such-and-such in the docs, so that next time there's a higher
chance that the docs are kept up to date?
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
On Wed, Jul 15, 2015 at 5:57 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
Fujii Masao wrote:
Hi,
"25.5.3. Administrator's Overview" in the document
-----------------------------------------------------
The setting of some parameters on the standby will need reconfiguration
if they have been changed on the primary. For these parameters,
the value on the standby must be equal to or greater than the value
on the primary. If these parameters are not set high enough then
the standby will refuse to start. Higher values can then be supplied
and the server restarted to begin recovery again. These parameters are:
max_connections
max_prepared_transactions
max_locks_per_transaction
-----------------------------------------------------I found that the value of max_worker_processes on the standby also
must be equal to or greater than the value on the master. So we should
just add max_worker_processes to this paragraph. Patch attached.True. Also track_commit_timestamp.
Yes, but I intentionally did not include track_commit_timestamp in
the patch because it's not easy for me to document the hot standby
condition of track_commit_timestamp unless I read the code more.
One example which makes me a bit confusing is; both master and
standby are running fine with track_commit_timestamp disabled,
then I enable it only on the master. That is, the value of
track_commit_timestamp is not the same between master and standby.
No error happens in this case. According to the code of xlog_redo(),
the commit timestamp tracking mechanism is activated in this case.
However, after that, if standby is restarted, standby emits an error
because the value of track_commit_timestamp is not the same between
master and standby. Simple question is; why do we need to cause
the standby fail in this case? Since I'm not familiar with the code of
track_commit_timestamp yet, I'm not sure whether this behavior is
valid or not.
Can you add a comment somewhere in
CheckRequiredParameterValues(void) that the set of parameters is listed
in section such-and-such in the docs, so that next time there's a higher
chance that the docs are kept up to date?
+1
What about the attached patch?
Regards,
--
Fujii Masao
Attachments:
doc_v2.patchtext/x-patch; charset=US-ASCII; name=doc_v2.patchDownload
*** a/doc/src/sgml/high-availability.sgml
--- b/doc/src/sgml/high-availability.sgml
***************
*** 2035,2040 **** LOG: database system is ready to accept read only connections
--- 2035,2045 ----
<varname>max_locks_per_transaction</>
</para>
</listitem>
+ <listitem>
+ <para>
+ <varname>max_worker_processes</>
+ </para>
+ </listitem>
</itemizedlist>
</para>
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 5817,5822 **** do { \
--- 5817,5826 ----
/*
* Check to see if required parameters are set high enough on this server
* for various aspects of recovery operation.
+ *
+ * Note that all the parameters which this function tests need to be
+ * listed in Administrator's Overview section in high-availability.sgml.
+ * If you change them, don't forget to update the list.
*/
static void
CheckRequiredParameterValues(void)
On Thu, Jul 16, 2015 at 12:07 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
One example which makes me a bit confusing is; both master and
standby are running fine with track_commit_timestamp disabled,
then I enable it only on the master. That is, the value of
track_commit_timestamp is not the same between master and standby.
No error happens in this case. According to the code of xlog_redo(),
the commit timestamp tracking mechanism is activated in this case.
However, after that, if standby is restarted, standby emits an error
because the value of track_commit_timestamp is not the same between
master and standby. Simple question is; why do we need to cause
the standby fail in this case? Since I'm not familiar with the code of
track_commit_timestamp yet, I'm not sure whether this behavior is
valid or not.
Hmm, that seems like awfully weird behavior.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
Robert Haas wrote:
On Thu, Jul 16, 2015 at 12:07 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
One example which makes me a bit confusing is; both master and
standby are running fine with track_commit_timestamp disabled,
then I enable it only on the master. That is, the value of
track_commit_timestamp is not the same between master and standby.
No error happens in this case. According to the code of xlog_redo(),
the commit timestamp tracking mechanism is activated in this case.
Well. We could have the standby fail (i.e. stop replication) when it
receives the WAL record indicating that the master turned it on. But
that would be very unfriendly, so we chose to make it follow the
master config instead. We're okay with this part, yes?
However, after that, if standby is restarted, standby emits an error
because the value of track_commit_timestamp is not the same between
master and standby. Simple question is; why do we need to cause
the standby fail in this case? Since I'm not familiar with the code of
track_commit_timestamp yet, I'm not sure whether this behavior is
valid or not.Hmm, that seems like awfully weird behavior.
The alternative is to turn the feature on automatically if it sees that
the master also has it on, i.e. the value would not be what the config
file says it is. Doing this would be a bit surprising IMO, but given
the behavior above maybe it's better than the current behavior.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
On Tue, Aug 4, 2015 at 12:41 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
Robert Haas wrote:
On Thu, Jul 16, 2015 at 12:07 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
One example which makes me a bit confusing is; both master and
standby are running fine with track_commit_timestamp disabled,
then I enable it only on the master. That is, the value of
track_commit_timestamp is not the same between master and standby.
No error happens in this case. According to the code of xlog_redo(),
the commit timestamp tracking mechanism is activated in this case.Well. We could have the standby fail (i.e. stop replication) when it
receives the WAL record indicating that the master turned it on. But
that would be very unfriendly, so we chose to make it follow the
master config instead. We're okay with this part, yes?However, after that, if standby is restarted, standby emits an error
because the value of track_commit_timestamp is not the same between
master and standby. Simple question is; why do we need to cause
the standby fail in this case? Since I'm not familiar with the code of
track_commit_timestamp yet, I'm not sure whether this behavior is
valid or not.Hmm, that seems like awfully weird behavior.
The alternative is to turn the feature on automatically if it sees that
the master also has it on, i.e. the value would not be what the config
file says it is. Doing this would be a bit surprising IMO, but given
the behavior above maybe it's better than the current behavior.
I think it's totally reasonable for the standby to follow the master's
behavior rather than the config file. That should be documented, but
otherwise, no problem. If it were technologically possible for the
standby to follow the config file rather than the master in all cases,
that would be fine, too. But the current behavior is somewhere in the
middle, and that doesn't seem like a good plan.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
Adding CC to hackers, since this is clearly not just a docs issue. Also
CCing Petr and Craig since they are the ones that know how this is used
in BDR.
Robert Haas wrote:
On Tue, Aug 4, 2015 at 12:41 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
The alternative is to turn the feature on automatically if it sees that
the master also has it on, i.e. the value would not be what the config
file says it is. Doing this would be a bit surprising IMO, but given
the behavior above maybe it's better than the current behavior.I think it's totally reasonable for the standby to follow the master's
behavior rather than the config file. That should be documented, but
otherwise, no problem. If it were technologically possible for the
standby to follow the config file rather than the master in all cases,
that would be fine, too. But the current behavior is somewhere in the
middle, and that doesn't seem like a good plan.
So I discussed this with Petr. He points out that if we make the
standby follows the master, then the problem would be the misbehavior
that results once the standby is promoted: at that point the standby
would no longer "follow the master" and would start with the feature
turned off, which could be disastrous (depending on what are you using
the commit timestamps for).
To solve that problem, you could suggest that if we see the setting
turned on in pg_control then we should follow that instead of the config
file; but then the problem is that there's no way to turn the feature
off. And things are real crazy by then.
Given this, we're leaning towards the idea that the standby should not
try to follow the master at all. Instead, an extension that wants to
use this stuff can check the value for itself, and raise a fatal error
if it's not already turned on the config file. That way, a lot of the
strange corner cases disappear.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
On 2015-08-05 00:13, Alvaro Herrera wrote:
Robert Haas wrote:
On Tue, Aug 4, 2015 at 12:41 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:The alternative is to turn the feature on automatically if it sees that
the master also has it on, i.e. the value would not be what the config
file says it is. Doing this would be a bit surprising IMO, but given
the behavior above maybe it's better than the current behavior.I think it's totally reasonable for the standby to follow the master's
behavior rather than the config file. That should be documented, but
otherwise, no problem. If it were technologically possible for the
standby to follow the config file rather than the master in all cases,
that would be fine, too. But the current behavior is somewhere in the
middle, and that doesn't seem like a good plan.So I discussed this with Petr. He points out that if we make the
standby follows the master, then the problem would be the misbehavior
that results once the standby is promoted: at that point the standby
would no longer "follow the master" and would start with the feature
turned off, which could be disastrous (depending on what are you using
the commit timestamps for).Given this, we're leaning towards the idea that the standby should not
try to follow the master at all. Instead, an extension that wants to
use this stuff can check the value for itself, and raise a fatal error
if it's not already turned on the config file. That way, a lot of the
strange corner cases disappear.
Actually, after thinking bit more about this I think the behavior of
these two will be similar - you suddenly lose the commit timestamp info.
The difference is that with fist option you'll lose it after restart
while with second one you lose it immediately after promotion since
there was never any info on the slave.
Extensions can do sanity checking in both scenarios.
The way I see it the first option has following advantages:
- it's smaller change
- it's more consistent with how wal_log_hints behaves
- fixing the config does not require server restart since the in-memory
state was set from WAL record automatically
However the second option has also some:
- one can have slave which doesn't have overhead of the commit timestamp
SLRU if they don't need it there
- it's theoretically easier to notice that the track_commit_timestamps
is off in config because the the SQL interface will complain if called
on the slave
So +0.5 from me towards following master and removing the error message
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
On Tue, Aug 4, 2015 at 6:13 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
I think it's totally reasonable for the standby to follow the master's
behavior rather than the config file. That should be documented, but
otherwise, no problem. If it were technologically possible for the
standby to follow the config file rather than the master in all cases,
that would be fine, too. But the current behavior is somewhere in the
middle, and that doesn't seem like a good plan.So I discussed this with Petr. He points out that if we make the
standby follows the master, then the problem would be the misbehavior
that results once the standby is promoted: at that point the standby
would no longer "follow the master" and would start with the feature
turned off, which could be disastrous (depending on what are you using
the commit timestamps for).
That seems like an imaginary problem. If it's critical to have commit
timestamps, don't turn them off on the standby.
To solve that problem, you could suggest that if we see the setting
turned on in pg_control then we should follow that instead of the config
file; but then the problem is that there's no way to turn the feature
off. And things are real crazy by then.
There's no existing precedent for a feature that lets the standby be
different from the master *in any way*. So I don't see why we should
start here. I think the reasonable definition is that the GUC
controls whether the master tries to update the SLRU (and generate
appropriate WAL records, presumably). The standby should not get a
choice about whether to replay those WAL records.
Note that if you do allow the standby to decide not to replay the WAL
records for this feature, then the data on the standby could be
partially there but not completely there after promotion, because the
DBA might have flipped the switch on and off at different times.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs
On Thu, Jul 16, 2015 at 1:07 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Wed, Jul 15, 2015 at 5:57 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:Fujii Masao wrote:
Hi,
"25.5.3. Administrator's Overview" in the document
-----------------------------------------------------
The setting of some parameters on the standby will need reconfiguration
if they have been changed on the primary. For these parameters,
the value on the standby must be equal to or greater than the value
on the primary. If these parameters are not set high enough then
the standby will refuse to start. Higher values can then be supplied
and the server restarted to begin recovery again. These parameters are:
max_connections
max_prepared_transactions
max_locks_per_transaction
-----------------------------------------------------I found that the value of max_worker_processes on the standby also
must be equal to or greater than the value on the master. So we should
just add max_worker_processes to this paragraph. Patch attached.True. Also track_commit_timestamp.
Yes, but I intentionally did not include track_commit_timestamp in
the patch because it's not easy for me to document the hot standby
condition of track_commit_timestamp unless I read the code more.One example which makes me a bit confusing is; both master and
standby are running fine with track_commit_timestamp disabled,
then I enable it only on the master. That is, the value of
track_commit_timestamp is not the same between master and standby.
No error happens in this case. According to the code of xlog_redo(),
the commit timestamp tracking mechanism is activated in this case.
However, after that, if standby is restarted, standby emits an error
because the value of track_commit_timestamp is not the same between
master and standby. Simple question is; why do we need to cause
the standby fail in this case? Since I'm not familiar with the code of
track_commit_timestamp yet, I'm not sure whether this behavior is
valid or not.Can you add a comment somewhere in
CheckRequiredParameterValues(void) that the set of parameters is listed
in section such-and-such in the docs, so that next time there's a higher
chance that the docs are kept up to date?+1
What about the attached patch?
Applied the patch.
Regards,
--
Fujii Masao
--
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs