pgsql: Make CheckRequiredParameterValues() depend upon correct

Started by Simon Riggsalmost 16 years ago95 messageshackers
Jump to latest
#1Simon Riggs
simon@2ndQuadrant.com

Log Message:
-----------
Make CheckRequiredParameterValues() depend upon correct combination
of parameters. Fix bug report by Robert Haas that error message and
hint was incorrect if wrong mode parameters specified on master.
Internal changes only. Proposals for parameter simplification on
master/primary still under way.

Modified Files:
--------------
pgsql/src/backend/access/transam:
xlog.c (r1.401 -> r1.402)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xlog.c?r1=1.401&r2=1.402)
pgsql/src/include/catalog:
pg_control.h (r1.51 -> r1.52)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_control.h?r1=1.51&r2=1.52)

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#1)
Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

sriggs@postgresql.org (Simon Riggs) writes:

Log Message:
-----------
Make CheckRequiredParameterValues() depend upon correct combination
of parameters. Fix bug report by Robert Haas that error message and
hint was incorrect if wrong mode parameters specified on master.
Internal changes only. Proposals for parameter simplification on
master/primary still under way.

Modified Files:
--------------
pgsql/src/backend/access/transam:
xlog.c (r1.401 -> r1.402)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xlog.c?r1=1.401&r2=1.402)
pgsql/src/include/catalog:
pg_control.h (r1.51 -> r1.52)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_control.h?r1=1.51&r2=1.52)

This is a change in pg_control layout and requires a bump to the
pg_control version number (and hence forced initdb's all round).

I think it was quite premature to commit this when the design is
still under active discussion --- you may be forcing two rounds
of initdb on testers, when maybe only one or none would be enough.
Especially when you appear to be in the minority about what the design
should be.

regards, tom lane

#3Robert Haas
robertmhaas@gmail.com
In reply to: Simon Riggs (#1)
Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

On Fri, Apr 23, 2010 at 3:57 PM, Simon Riggs <sriggs@postgresql.org> wrote:

Log Message:
-----------
Make CheckRequiredParameterValues() depend upon correct combination
of parameters. Fix bug report by Robert Haas that error message and
hint was incorrect if wrong mode parameters specified on master.
Internal changes only. Proposals for parameter simplification on
master/primary still under way.

This is (1) premature, (2) not the consensus position, and unless I
mistaken, (3) wrong.

...Robert

#4Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#2)
Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

On Fri, 2010-04-23 at 16:04 -0400, Tom Lane wrote:

sriggs@postgresql.org (Simon Riggs) writes:

Log Message:
-----------
Make CheckRequiredParameterValues() depend upon correct combination
of parameters. Fix bug report by Robert Haas that error message and
hint was incorrect if wrong mode parameters specified on master.
Internal changes only. Proposals for parameter simplification on
master/primary still under way.

Modified Files:
--------------
pgsql/src/backend/access/transam:
xlog.c (r1.401 -> r1.402)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xlog.c?r1=1.401&amp;r2=1.402)
pgsql/src/include/catalog:
pg_control.h (r1.51 -> r1.52)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/include/catalog/pg_control.h?r1=1.51&amp;r2=1.52)

This is a change in pg_control layout and requires a bump to the
pg_control version number (and hence forced initdb's all round).

OK

I think it was quite premature to commit this when the design is
still under active discussion --- you may be forcing two rounds
of initdb on testers, when maybe only one or none would be enough.
Especially when you appear to be in the minority about what the design
should be.

No intention of doing that. This change allows people to see what the
dependency actually is once the bug has been fixed. Change needs to
start from here, not from where we were before.

--
Simon Riggs www.2ndQuadrant.com

#5Simon Riggs
simon@2ndQuadrant.com
In reply to: Robert Haas (#3)
Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

On Fri, 2010-04-23 at 16:05 -0400, Robert Haas wrote:

On Fri, Apr 23, 2010 at 3:57 PM, Simon Riggs <sriggs@postgresql.org> wrote:

Log Message:
-----------
Make CheckRequiredParameterValues() depend upon correct combination
of parameters. Fix bug report by Robert Haas that error message and
hint was incorrect if wrong mode parameters specified on master.
Internal changes only. Proposals for parameter simplification on
master/primary still under way.

This is (1) premature, (2) not the consensus position, and unless I
mistaken,

As I've said elsewhere it was intended to assist and clarify. Those
changes have nothing to do with the current discussion as I understood
it, which was about simplifying the user interface.

I will revoke.

--
Simon Riggs www.2ndQuadrant.com

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#4)
Re: [HACKERS] Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

Simon Riggs <simon@2ndQuadrant.com> writes:

No intention of doing that. This change allows people to see what the
dependency actually is once the bug has been fixed. Change needs to
start from here, not from where we were before.

Well, actually, now that I've looked at the patch I think it's starting
from a fundamentally wrong position anyway. Checkpoint records are a
completely wrong mechanism for transmitting this data to slaves, because
a checkpoint is emitted *after* we do something, not *before* we do it.
In particular it's ludicrous to be looking at shutdown checkpoints to
try to determine whether the subsequent WAL will meet the slave's
requirements. There's no connection at all between what the GUC state
was at shutdown and what it might be after starting again.

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Then, slaves could check whether the master's wal_mode is high enough
by looking at pg_control when they start plus any wal_mode_change
records they come across.

If we did this then we could get rid of those WAL record types that were
added to signify that information had been omitted from WAL at specific
times.

regards, tom lane

#7Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#6)
Re: [HACKERS] Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

On Fri, Apr 23, 2010 at 4:44 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Simon Riggs <simon@2ndQuadrant.com> writes:

No intention of doing that. This change allows people to see what the
dependency actually is once the bug has been fixed. Change needs to
start from here, not from where we were before.

Well, actually, now that I've looked at the patch I think it's starting
from a fundamentally wrong position anyway.  Checkpoint records are a
completely wrong mechanism for transmitting this data to slaves, because
a checkpoint is emitted *after* we do something, not *before* we do it.
In particular it's ludicrous to be looking at shutdown checkpoints to
try to determine whether the subsequent WAL will meet the slave's
requirements.  There's no connection at all between what the GUC state
was at shutdown and what it might be after starting again.

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Well, right now wal_mode would only be able to be changed at server
restart. Eventually we might relax that, but I think there are some
restrictions on how we can do it - like maybe needing to wait until
all the transactions running at the time the change was decided on
have committed, or, well, I'm not sure.

...Robert

#8Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#6)
Re: [HACKERS] Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

On Fri, 2010-04-23 at 16:44 -0400, Tom Lane wrote:

There's no connection at all between what the GUC state
was at shutdown and what it might be after starting again.

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Then, slaves could check whether the master's wal_mode is high enough
by looking at pg_control when they start plus any wal_mode_change
records they come across.

Seems OK on standby side. On the primary there are some other points,
mentioned on other thread as to when we can change wal_mode.

If we did this then we could get rid of those WAL record types that were
added to signify that information had been omitted from WAL at specific
times.

Please.

--
Simon Riggs www.2ndQuadrant.com

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#7)
Re: [HACKERS] Re: pgsql: Make CheckRequiredParameterValues() depend upon correct

Robert Haas <robertmhaas@gmail.com> writes:

On Fri, Apr 23, 2010 at 4:44 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Well, right now wal_mode would only be able to be changed at server
restart.

Right, but slave servers won't find out about the change until the first
checkpoint after the start. Which is Too Late.

regards, tom lane

#10Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#6)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

On Fri, Apr 23, 2010 at 4:44 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Simon Riggs <simon@2ndQuadrant.com> writes:

No intention of doing that. This change allows people to see what the
dependency actually is once the bug has been fixed. Change needs to
start from here, not from where we were before.

Well, actually, now that I've looked at the patch I think it's starting
from a fundamentally wrong position anyway.  Checkpoint records are a
completely wrong mechanism for transmitting this data to slaves, because
a checkpoint is emitted *after* we do something, not *before* we do it.
In particular it's ludicrous to be looking at shutdown checkpoints to
try to determine whether the subsequent WAL will meet the slave's
requirements.  There's no connection at all between what the GUC state
was at shutdown and what it might be after starting again.

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Then, slaves could check whether the master's wal_mode is high enough
by looking at pg_control when they start plus any wal_mode_change
records they come across.

If we did this then we could get rid of those WAL record types that were
added to signify that information had been omitted from WAL at specific
times.

<dons project manager hat>

I notice that Heikki's patch doesn't include doing the above. Should
we? If so, who's going to do it?

...Robert

#11Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Robert Haas (#10)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

Robert Haas wrote:

On Fri, Apr 23, 2010 at 4:44 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Well, actually, now that I've looked at the patch I think it's starting
from a fundamentally wrong position anyway. Checkpoint records are a
completely wrong mechanism for transmitting this data to slaves, because
a checkpoint is emitted *after* we do something, not *before* we do it.
In particular it's ludicrous to be looking at shutdown checkpoints to
try to determine whether the subsequent WAL will meet the slave's
requirements. There's no connection at all between what the GUC state
was at shutdown and what it might be after starting again.

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Then, slaves could check whether the master's wal_mode is high enough
by looking at pg_control when they start plus any wal_mode_change
records they come across.

If we did this then we could get rid of those WAL record types that were
added to signify that information had been omitted from WAL at specific
times.

<dons project manager hat>

I notice that Heikki's patch doesn't include doing the above. Should
we? If so, who's going to do it?

I'll give it a shot.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#12Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Heikki Linnakangas (#11)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

Heikki Linnakangas wrote:

Robert Haas wrote:

On Fri, Apr 23, 2010 at 4:44 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Well, actually, now that I've looked at the patch I think it's starting
from a fundamentally wrong position anyway. Checkpoint records are a
completely wrong mechanism for transmitting this data to slaves, because
a checkpoint is emitted *after* we do something, not *before* we do it.
In particular it's ludicrous to be looking at shutdown checkpoints to
try to determine whether the subsequent WAL will meet the slave's
requirements. There's no connection at all between what the GUC state
was at shutdown and what it might be after starting again.

A design that might work is
(1) store the active value of wal_mode in pg_control (but NOT as part of
the last-checkpoint-record image).
(2) invent a new WAL record type that is transmitted when we change
wal_mode.

Then, slaves could check whether the master's wal_mode is high enough
by looking at pg_control when they start plus any wal_mode_change
records they come across.

If we did this then we could get rid of those WAL record types that were
added to signify that information had been omitted from WAL at specific
times.

<dons project manager hat>

I notice that Heikki's patch doesn't include doing the above. Should
we? If so, who's going to do it?

I'll give it a shot.

Ok, here's a patch that includes the changes to add new wal_mode GUC
(http://archives.postgresql.org/message-id/4BD581A6.60602@enterprisedb.com),
and implements Tom's design to keep a copy of wal_mode and the
max_connections, max_prepared_xacts and max_locks_per_xact settings in
pg_control.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Attachments:

wal_mode-3.patchtext/x-diff; name=wal_mode-3.patchDownload+305-191
#13Fujii Masao
masao.fujii@gmail.com
In reply to: Heikki Linnakangas (#12)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

On Tue, Apr 27, 2010 at 5:09 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

Ok, here's a patch that includes the changes to add new wal_mode GUC
(http://archives.postgresql.org/message-id/4BD581A6.60602@enterprisedb.com),
and implements Tom's design to keep a copy of wal_mode and the
max_connections, max_prepared_xacts and max_locks_per_xact settings in
pg_control.

I have some comments:

config.sgml

<literal>on</literal>. It is thought that there is little
measurable difference in performance from using this feature, so
feedback is welcome if any production impacts are noticeable.
It is likely that this parameter will be removed in later releases.

Is this description still required for recovery_connections?

if (!XLogArchivingActive())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_mode must be enabled at server start.")));

You need to change the error messages which refer to archive_mode,
like the above.

+ /*
+  * For Hot Standby, the WAL must be generated with 'hot_standby' mode,
+  * and we must have at least as many backend slots as the primary.
+  */
+ if (InArchiveRecovery && XLogRequestRecoveryConnections)
+ {
+   if (ControlFile->wal_mode < WAL_MODE_HOT_STANDBY)
+       ereport(ERROR,
+              (errmsg("recovery connections cannot start because
wal_mode was not set to 'hot_standby' on the WAL source server")));

This seems to always prevent the server from doing an archive recovery
since wal_mode is expected to be WAL_MODE_ARCHIVE in that case.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

#14Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Fujii Masao (#13)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

Fujii Masao wrote:

config.sgml

<literal>on</literal>. It is thought that there is little
measurable difference in performance from using this feature, so
feedback is welcome if any production impacts are noticeable.
It is likely that this parameter will be removed in later releases.

Is this description still required for recovery_connections?

Hmm, I guess it was referring to setting recovery_connections in the
master, I don't see us removing that option from the standby in the
future. recovery_connections in the master is being replaced with the
wal_mode setting, so I guess that's not required anymore.

if (!XLogArchivingActive())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_mode must be enabled at server start.")));

You need to change the error messages which refer to archive_mode,
like the above.

Hmm, I think we should change not only the error message, but the logic
too. There's two related checks there:

if (!XLogArchivingActive())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_mode must be enabled at server start.")));

if (!XLogArchiveCommandSet())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_command must be defined before "
"online backups can be made safely.")));

You can use streaming replication too to transport the WAL generated
during the backup, so I think we should just check that wal_mode>='archive'.

+ /*
+  * For Hot Standby, the WAL must be generated with 'hot_standby' mode,
+  * and we must have at least as many backend slots as the primary.
+  */
+ if (InArchiveRecovery && XLogRequestRecoveryConnections)
+ {
+   if (ControlFile->wal_mode < WAL_MODE_HOT_STANDBY)
+       ereport(ERROR,
+              (errmsg("recovery connections cannot start because
wal_mode was not set to 'hot_standby' on the WAL source server")));

This seems to always prevent the server from doing an archive recovery
since wal_mode is expected to be WAL_MODE_ARCHIVE in that case.

No, it doesn't prevent archive recovery. It only prevents hot standby if
wal_mode was not 'hot_standby' in the master. I think you missed the "&&
XLogRequestRecoveryConnections" condition above.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#15Fujii Masao
masao.fujii@gmail.com
In reply to: Heikki Linnakangas (#14)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

On Tue, Apr 27, 2010 at 6:49 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

Fujii Masao wrote:

config.sgml

<literal>on</literal>.  It is thought that there is little
measurable difference in performance from using this feature, so
feedback is welcome if any production impacts are noticeable.
It is likely that this parameter will be removed in later releases.

Is this description still required for recovery_connections?

Hmm, I guess it was referring to setting recovery_connections in the
master, I don't see us removing that option from the standby in the
future. recovery_connections in the master is being replaced with the
wal_mode setting, so I guess that's not required anymore.

Agreed.

if (!XLogArchivingActive())
  ereport(ERROR,
    (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
     errmsg("WAL archiving is not active"),
     errhint("archive_mode must be enabled at server start.")));

You need to change the error messages which refer to archive_mode,
like the above.

Hmm, I think we should change not only the error message, but the logic
too. There's two related checks there:

      if (!XLogArchivingActive())
              ereport(ERROR,
                              (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
                               errmsg("WAL archiving is not active"),
                               errhint("archive_mode must be enabled at server start.")));

      if (!XLogArchiveCommandSet())
              ereport(ERROR,
                              (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
                               errmsg("WAL archiving is not active"),
                               errhint("archive_command must be defined before "
                                               "online backups can be made safely.")));

You can use streaming replication too to transport the WAL generated
during the backup, so I think we should just check that wal_mode>='archive'.

It's OK in pg_start_backup(), but seems NG in pg_stop_backup() since
it waits until some WAL files have been archived by the archiver. No?

+ /*
+  * For Hot Standby, the WAL must be generated with 'hot_standby' mode,
+  * and we must have at least as many backend slots as the primary.
+  */
+ if (InArchiveRecovery && XLogRequestRecoveryConnections)
+ {
+   if (ControlFile->wal_mode < WAL_MODE_HOT_STANDBY)
+       ereport(ERROR,
+              (errmsg("recovery connections cannot start because
wal_mode was not set to 'hot_standby' on the WAL source server")));

This seems to always prevent the server from doing an archive recovery
since wal_mode is expected to be WAL_MODE_ARCHIVE in that case.

No, it doesn't prevent archive recovery. It only prevents hot standby if
wal_mode was not 'hot_standby' in the master. I think you missed the "&&
XLogRequestRecoveryConnections" condition above.

Even if we do only archive recovery, XLogRequestRecoveryConnections
might be TRUE. Or we need to ensure that the recovery_connection is
FALSE in the postgresql.conf before starting archive recovery?

And I tried archive recovery, and encountered the following error.

LOG: starting archive recovery
LOG: restored log file "000000010000000000000001" from archive
FATAL: recovery connections cannot start because wal_mode was not
set to 'hot_standby' on the WAL source server
LOG: startup process (PID 32512) exited with exit code 1
LOG: aborting startup due to startup process failure

XLOG-related parameters in postgresql.conf
archive_mode = on
archive_command = 'cp %p ../data.arh/%f'
wal_mode = archive
recovery_connections = on

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

#16Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Fujii Masao (#15)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

Fujii Masao wrote:

On Tue, Apr 27, 2010 at 6:49 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

Fujii Masao wrote:

if (!XLogArchivingActive())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_mode must be enabled at server start.")));

You need to change the error messages which refer to archive_mode,
like the above.

Hmm, I think we should change not only the error message, but the logic
too. There's two related checks there:

if (!XLogArchivingActive())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_mode must be enabled at server start.")));

if (!XLogArchiveCommandSet())
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("WAL archiving is not active"),
errhint("archive_command must be defined before "
"online backups can be made safely.")));

You can use streaming replication too to transport the WAL generated
during the backup, so I think we should just check that wal_mode>='archive'.

It's OK in pg_start_backup(), but seems NG in pg_stop_backup() since
it waits until some WAL files have been archived by the archiver. No?

Good point, that logic would need to be changed too. Should it simply
return immediately if archive_mode=off?

+ /*
+  * For Hot Standby, the WAL must be generated with 'hot_standby' mode,
+  * and we must have at least as many backend slots as the primary.
+  */
+ if (InArchiveRecovery && XLogRequestRecoveryConnections)
+ {
+   if (ControlFile->wal_mode < WAL_MODE_HOT_STANDBY)
+       ereport(ERROR,
+              (errmsg("recovery connections cannot start because
wal_mode was not set to 'hot_standby' on the WAL source server")));

This seems to always prevent the server from doing an archive recovery
since wal_mode is expected to be WAL_MODE_ARCHIVE in that case.

No, it doesn't prevent archive recovery. It only prevents hot standby if
wal_mode was not 'hot_standby' in the master. I think you missed the "&&
XLogRequestRecoveryConnections" condition above.

Even if we do only archive recovery, XLogRequestRecoveryConnections
might be TRUE. Or we need to ensure that the recovery_connection is
FALSE in the postgresql.conf before starting archive recovery?

Umm, yes, if you have recovery_connnections=on, it means you want hot
standby. And for that you need wal_mode='hot_standby'.

By "it doesn't prevent archive recovery" I meant "you can do traditional
archive recovery without hot standby". It doesn't matter how the WAL is
transported, via the archive or via streaming replication.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#17Fujii Masao
masao.fujii@gmail.com
In reply to: Heikki Linnakangas (#16)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

On Tue, Apr 27, 2010 at 7:50 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

It's OK in pg_start_backup(), but seems NG in pg_stop_backup() since
it waits until some WAL files have been archived by the archiver. No?

Good point, that logic would need to be changed too. Should it simply
return immediately if archive_mode=off?

What if we wrongly set archive_mode to on and wal_mode to minimal?
I think that checking XLogArchivingActive() in pg_stop_backup() is
adequate.

+ /*
+  * For Hot Standby, the WAL must be generated with 'hot_standby' mode,
+  * and we must have at least as many backend slots as the primary.
+  */
+ if (InArchiveRecovery && XLogRequestRecoveryConnections)
+ {
+   if (ControlFile->wal_mode < WAL_MODE_HOT_STANDBY)
+       ereport(ERROR,
+              (errmsg("recovery connections cannot start because
wal_mode was not set to 'hot_standby' on the WAL source server")));

This seems to always prevent the server from doing an archive recovery
since wal_mode is expected to be WAL_MODE_ARCHIVE in that case.

No, it doesn't prevent archive recovery. It only prevents hot standby if
wal_mode was not 'hot_standby' in the master. I think you missed the "&&
XLogRequestRecoveryConnections" condition above.

Even if we do only archive recovery, XLogRequestRecoveryConnections
might be TRUE. Or we need to ensure that the recovery_connection is
FALSE in the postgresql.conf before starting archive recovery?

Umm, yes, if you have recovery_connnections=on, it means you want hot
standby. And for that you need wal_mode='hot_standby'.

Since the default value of recovery_connections is TRUE, I think that
the trouble which I encountered would often happen. We should disable
recovery_connections by default? Furthermore should move it from
postgresql.conf to recovery.conf?

On the other hand, I feel that recovery_connections=on in an archive
recovery is valid configuration *until* any read only connections are
requested. How about moving the above check to postmaster or backend?

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

#18Robert Haas
robertmhaas@gmail.com
In reply to: Fujii Masao (#17)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

On Tue, Apr 27, 2010 at 7:24 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Tue, Apr 27, 2010 at 7:50 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

It's OK in pg_start_backup(), but seems NG in pg_stop_backup() since
it waits until some WAL files have been archived by the archiver. No?

Good point, that logic would need to be changed too. Should it simply
return immediately if archive_mode=off?

What if we wrongly set archive_mode to on and wal_mode to minimal?
I think that checking XLogArchivingActive() in pg_stop_backup() is
adequate.

That case should be rejected at primary startup.

+ /*
+  * For Hot Standby, the WAL must be generated with 'hot_standby' mode,
+  * and we must have at least as many backend slots as the primary.
+  */
+ if (InArchiveRecovery && XLogRequestRecoveryConnections)
+ {
+   if (ControlFile->wal_mode < WAL_MODE_HOT_STANDBY)
+       ereport(ERROR,
+              (errmsg("recovery connections cannot start because
wal_mode was not set to 'hot_standby' on the WAL source server")));

This seems to always prevent the server from doing an archive recovery
since wal_mode is expected to be WAL_MODE_ARCHIVE in that case.

No, it doesn't prevent archive recovery. It only prevents hot standby if
wal_mode was not 'hot_standby' in the master. I think you missed the "&&
XLogRequestRecoveryConnections" condition above.

Even if we do only archive recovery, XLogRequestRecoveryConnections
might be TRUE. Or we need to ensure that the recovery_connection is
FALSE in the postgresql.conf before starting archive recovery?

Umm, yes, if you have recovery_connnections=on, it means you want hot
standby. And for that you need wal_mode='hot_standby'.

Since the default value of recovery_connections is TRUE, I think that
the trouble which I encountered would often happen. We should disable
recovery_connections by default? Furthermore should move it from
postgresql.conf to recovery.conf?

On the other hand, I feel that recovery_connections=on in an archive
recovery is valid configuration *until* any read only connections are
requested. How about moving the above check to postmaster or backend?

Or just not starting recovery connections, but still doing archive
recovery? I think in this case a WARNING might be adequate.

...Robert

#19Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#12)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Ok, here's a patch that includes the changes to add new wal_mode GUC
(http://archives.postgresql.org/message-id/4BD581A6.60602@enterprisedb.com),

I haven't read this in any detail, but why does it add inclusion of
pg_control.h to xlog.h? I don't see any reason for that in the actual
changes in xlog.h.

regards, tom lane

#20Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#19)
Re: Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Ok, here's a patch that includes the changes to add new wal_mode GUC
(http://archives.postgresql.org/message-id/4BD581A6.60602@enterprisedb.com),

I haven't read this in any detail, but why does it add inclusion of
pg_control.h to xlog.h? I don't see any reason for that in the actual
changes in xlog.h.

I put the enum for wal_mode to pg_control.h, so that it's available to
pg_controlinfo.c without #including xlog.h there. The
XLogArchivingActive() macro in xlog.h needs the enum values:

#define XLogArchivingActive() (XLogArchiveMode && wal_mode >=
WAL_MODE_ARCHIVE

I'm all ears for better suggestions, I didn't like that much either.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#21Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#20)
#22Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#21)
#23Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Heikki Linnakangas (#22)
#24Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#23)
#25Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Tom Lane (#24)
#26Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#24)
#27Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#26)
#28Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#27)
#29Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#28)
#30Fujii Masao
masao.fujii@gmail.com
In reply to: Heikki Linnakangas (#26)
#31Simon Riggs
simon@2ndQuadrant.com
In reply to: Fujii Masao (#30)
#32Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Fujii Masao (#30)
#33Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#26)
#34Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#27)
#35Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#34)
#36Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#33)
#37Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#36)
#38Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#37)
#39Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#34)
#40Fujii Masao
masao.fujii@gmail.com
In reply to: Heikki Linnakangas (#39)
#41Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Fujii Masao (#40)
#42Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#41)
#43Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#42)
#44Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#43)
#45Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#44)
#46Robert Haas
robertmhaas@gmail.com
In reply to: Heikki Linnakangas (#43)
#47Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#45)
#48Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Robert Haas (#46)
#49Robert Haas
robertmhaas@gmail.com
In reply to: Heikki Linnakangas (#48)
#50Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#47)
#51Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#50)
#52Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#51)
#53Robert Haas
robertmhaas@gmail.com
In reply to: Heikki Linnakangas (#50)
#54Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Robert Haas (#49)
#55Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Robert Haas (#53)
#56Robert Haas
robertmhaas@gmail.com
In reply to: Heikki Linnakangas (#54)
#57Robert Haas
robertmhaas@gmail.com
In reply to: Heikki Linnakangas (#55)
#58Dimitri Fontaine
dimitri@2ndQuadrant.fr
In reply to: Robert Haas (#49)
#59Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#51)
#60Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#59)
#61Aidan Van Dyk
aidan@highrise.ca
In reply to: Simon Riggs (#60)
#62Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#60)
#63Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#62)
#64Simon Riggs
simon@2ndQuadrant.com
In reply to: Aidan Van Dyk (#61)
#65Aidan Van Dyk
aidan@highrise.ca
In reply to: Simon Riggs (#63)
#66Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#63)
#67Aidan Van Dyk
aidan@highrise.ca
In reply to: Simon Riggs (#64)
#68Robert Haas
robertmhaas@gmail.com
In reply to: Aidan Van Dyk (#65)
#69Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Simon Riggs (#64)
#70Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#64)
#71Simon Riggs
simon@2ndQuadrant.com
In reply to: Kevin Grittner (#69)
#72Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#68)
#73Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Simon Riggs (#71)
#74Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#72)
#75Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#72)
#76Simon Riggs
simon@2ndQuadrant.com
In reply to: Kevin Grittner (#73)
#77Greg Smith
gsmith@gregsmith.com
In reply to: Aidan Van Dyk (#61)
#78Joshua D. Drake
jd@commandprompt.com
In reply to: Tom Lane (#72)
#79Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#75)
#80Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#70)
#81Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Simon Riggs (#75)
#82Simon Riggs
simon@2ndQuadrant.com
In reply to: Robert Haas (#74)
#83Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#75)
#84Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Simon Riggs (#82)
#85Simon Riggs
simon@2ndQuadrant.com
In reply to: Kevin Grittner (#81)
#86Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#80)
#87Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#82)
#88Joshua D. Drake
jd@commandprompt.com
In reply to: Heikki Linnakangas (#86)
#89Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kevin Grittner (#84)
#90Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Simon Riggs (#85)
#91Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kevin Grittner (#90)
#92Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#91)
#93Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#91)
#94Bruce Momjian
bruce@momjian.us
In reply to: Heikki Linnakangas (#38)
#95Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Bruce Momjian (#94)