unite recovery.conf and postgresql.conf
Hi,
http://archives.postgresql.org/pgsql-hackers/2010-12/msg02343.php
In previous discussion, we've reached the consensus that we should unite
recovery.conf and postgresql.conf. The attached patch does that. The
patch is WIP, I'll have to update the document, but if you notice something,
please feel free to comment.
This patch allows us to specify recovery parameters like primary_conninfo
in postgresql.conf. You can change some of them without restarting
the server (i.e., by using "pg_ctl reload") and see the current settings by
SHOW command. But note that recovery.conf is still required as a status
file for archive recovery and standby server even if you put all the settings
in postgresql.conf. Recovery parameters in postgresql.conf only have effect
when recovery.conf exists.
For backward compatibility, this patch still allows us to specify recovery
parameters in recovery.conf. But, as in past years, you cannot reload
recovery.conf and see the current settings by SHOW command. We should
recommend to put all the settings in postgresql.conf and empty recovery.conf,
but you can also put all recovery parameters in recovery.conf.
If the same parameter is specified in both file, the setting in recovery.conf
overrides that in postgresql.conf. In this case, SHOW command displays
the settings in postgresql.conf even though they are not used at all. Even if
you change the settings in postgresql.conf and reload it, they have no effect
because the settings in recovery.conf are used preferentially.
These limitations on recovery.conf might confuse users. But since most
users will put all the settings in postgresql.conf if we recommend to do that,
I don't think that it's worth complicating the source code for getting rid of
those limitations.
Thought?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Attachments:
unite_recoveryconf_postgresqlconf_v0.patchtext/x-patch; charset=US-ASCII; name=unite_recoveryconf_postgresqlconf_v0.patchDownload
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 79,84 **** bool fullPageWrites = true;
--- 79,98 ----
bool log_checkpoints = false;
int sync_method = DEFAULT_SYNC_METHOD;
int wal_level = WAL_LEVEL_MINIMAL;
+ char *restore_command = NULL;
+ char *archive_cleanup_command = NULL;
+ char *recovery_end_command = NULL;
+ bool standby_mode = false;
+ char *primary_conninfo = NULL;
+ char *trigger_file = NULL;
+ RecoveryTargetType recovery_target = RECOVERY_TARGET_UNSET;
+ TransactionId recovery_target_xid = InvalidTransactionId;
+ TimestampTz recovery_target_time = 0;
+ char *recovery_target_name = NULL;
+ bool recovery_target_inclusive = true;
+ bool pause_at_recovery_target = true;
+ char *recovery_target_timeline_string = NULL;
+ TimeLineID recovery_target_timeline = 0;
#ifdef WAL_DEBUG
bool XLOG_DEBUG = false;
***************
*** 185,207 **** static bool InArchiveRecovery = false;
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* options taken from recovery.conf for archive recovery */
! static char *recoveryRestoreCommand = NULL;
! static char *recoveryEndCommand = NULL;
! static char *archiveCleanupCommand = NULL;
! static RecoveryTargetType recoveryTarget = RECOVERY_TARGET_UNSET;
! static bool recoveryTargetInclusive = true;
! static bool recoveryPauseAtTarget = true;
! static TransactionId recoveryTargetXid;
! static TimestampTz recoveryTargetTime;
! static char *recoveryTargetName;
!
! /* options taken from recovery.conf for XLOG streaming */
! static bool StandbyMode = false;
! static char *PrimaryConnInfo = NULL;
! static char *TriggerFile = NULL;
!
! /* if recoveryStopsHere returns true, it saves actual stop xid/time/name here */
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
--- 199,206 ----
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* if recoveryStopsHere returns true, it saves actual stop type/xid/time/name here */
! static RecoveryTargetType recoveryStopTarget;
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
***************
*** 407,416 **** typedef struct XLogCtlData
TimeLineID RecoveryTargetTLI;
/*
! * archiveCleanupCommand is read from recovery.conf but needs to be in
! * shared memory so that the bgwriter process can access it.
*/
! char archiveCleanupCommand[MAXPGPATH];
/*
* SharedRecoveryInProgress indicates if we're still in crash or archive
--- 406,415 ----
TimeLineID RecoveryTargetTLI;
/*
! * archive_cleanup_command can be read from recovery.conf but needs
! * to be in shared memory so that the bgwriter process can access it.
*/
! char archive_cleanup_command[MAXPGPATH];
/*
* SharedRecoveryInProgress indicates if we're still in crash or archive
***************
*** 611,616 **** static void SetRecoveryPause(bool recoveryPause);
--- 610,616 ----
static void SetLatestXTime(TimestampTz xtime);
static TimestampTz GetLatestXTime(void);
static void CheckRequiredParameterValues(void);
+ static void CheckRestoreCommandSet(void);
static void XLogReportParameters(void);
static void LocalSetXLogInsertAllowed(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
***************
*** 2931,2937 **** RestoreArchivedFile(char *path, const char *xlogfname,
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (recoveryRestoreCommand == NULL)
goto not_available;
/*
--- 2931,2937 ----
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (!restore_command[0])
goto not_available;
/*
***************
*** 2951,2957 **** RestoreArchivedFile(char *path, const char *xlogfname,
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the recoveryRestoreCommand may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
--- 2951,2957 ----
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the restore_command may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
***************
*** 3015,3021 **** RestoreArchivedFile(char *path, const char *xlogfname,
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = recoveryRestoreCommand; *sp; sp++)
{
if (*sp == '%')
{
--- 3015,3021 ----
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = restore_command; *sp; sp++)
{
if (*sp == '%')
{
***************
*** 3106,3112 **** RestoreArchivedFile(char *path, const char *xlogfname,
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (StandbyMode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
--- 3106,3112 ----
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (standby_mode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
***************
*** 4063,4069 **** next_record_is_invalid:
}
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return NULL;
--- 4063,4069 ----
}
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return NULL;
***************
*** 4522,4528 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
--- 4522,4528 ----
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryStopTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4530,4536 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
--- 4530,4536 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryStopTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4538,4544 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
--- 4538,4544 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryStopTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
***************
*** 5276,5283 **** static void
readRecoveryCommandFile(void)
{
FILE *fd;
- TimeLineID rtli = 0;
- bool rtliGiven = false;
ConfigVariable *item,
*head = NULL,
*tail = NULL;
--- 5276,5281 ----
***************
*** 5302,5480 **** readRecoveryCommandFile(void)
for (item = head; item; item = item->next)
{
if (strcmp(item->name, "restore_command") == 0)
! {
! recoveryRestoreCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("restore_command = '%s'",
! recoveryRestoreCommand)));
! }
else if (strcmp(item->name, "recovery_end_command") == 0)
! {
! recoveryEndCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("recovery_end_command = '%s'",
! recoveryEndCommand)));
! }
else if (strcmp(item->name, "archive_cleanup_command") == 0)
! {
! archiveCleanupCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("archive_cleanup_command = '%s'",
! archiveCleanupCommand)));
! }
else if (strcmp(item->name, "pause_at_recovery_target") == 0)
! {
! if (!parse_bool(item->value, &recoveryPauseAtTarget))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value", "pause_at_recovery_target")));
! ereport(DEBUG2,
! (errmsg_internal("pause_at_recovery_target = '%s'",
! item->value)));
! }
else if (strcmp(item->name, "recovery_target_timeline") == 0)
! {
! rtliGiven = true;
! if (strcmp(item->value, "latest") == 0)
! rtli = 0;
! else
! {
! errno = 0;
! rtli = (TimeLineID) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_timeline is not a valid number: \"%s\"",
! item->value)));
! }
! if (rtli)
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = %u", rtli)));
! else
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = latest")));
! }
else if (strcmp(item->name, "recovery_target_xid") == 0)
! {
! errno = 0;
! recoveryTargetXid = (TransactionId) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_xid is not a valid number: \"%s\"",
! item->value)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_xid = %u",
! recoveryTargetXid)));
! recoveryTarget = RECOVERY_TARGET_XID;
! }
else if (strcmp(item->name, "recovery_target_time") == 0)
! {
! /*
! * if recovery_target_xid or recovery_target_name specified, then
! * this overrides recovery_target_time
! */
! if (recoveryTarget == RECOVERY_TARGET_XID ||
! recoveryTarget == RECOVERY_TARGET_NAME)
! continue;
! recoveryTarget = RECOVERY_TARGET_TIME;
!
! /*
! * Convert the time string given by the user to TimestampTz form.
! */
! recoveryTargetTime =
! DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
! CStringGetDatum(item->value),
! ObjectIdGetDatum(InvalidOid),
! Int32GetDatum(-1)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_time = '%s'",
! timestamptz_to_str(recoveryTargetTime))));
! }
else if (strcmp(item->name, "recovery_target_name") == 0)
! {
! /*
! * if recovery_target_xid specified, then this overrides
! * recovery_target_name
! */
! if (recoveryTarget == RECOVERY_TARGET_XID)
! continue;
! recoveryTarget = RECOVERY_TARGET_NAME;
!
! recoveryTargetName = pstrdup(item->value);
! if (strlen(recoveryTargetName) >= MAXFNAMELEN)
! ereport(FATAL,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("recovery_target_name is too long (maximum %d characters)",
! MAXFNAMELEN - 1)));
!
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_name = '%s'",
! recoveryTargetName)));
! }
else if (strcmp(item->name, "recovery_target_inclusive") == 0)
! {
! /*
! * does nothing if a recovery_target is not also set
! */
! if (!parse_bool(item->value, &recoveryTargetInclusive))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "recovery_target_inclusive")));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_inclusive = %s",
! item->value)));
! }
else if (strcmp(item->name, "standby_mode") == 0)
! {
! if (!parse_bool(item->value, &StandbyMode))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "standby_mode")));
! ereport(DEBUG2,
! (errmsg_internal("standby_mode = '%s'", item->value)));
! }
else if (strcmp(item->name, "primary_conninfo") == 0)
! {
! PrimaryConnInfo = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("primary_conninfo = '%s'",
! PrimaryConnInfo)));
! }
else if (strcmp(item->name, "trigger_file") == 0)
! {
! TriggerFile = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("trigger_file = '%s'",
! TriggerFile)));
! }
else
ereport(FATAL,
(errmsg("unrecognized recovery parameter \"%s\"",
item->name)));
}
/*
* Check for compulsory parameters
*/
! if (StandbyMode)
! {
! if (PrimaryConnInfo == NULL && recoveryRestoreCommand == NULL)
! ereport(WARNING,
! (errmsg("recovery command file \"%s\" specified neither primary_conninfo nor restore_command",
! RECOVERY_COMMAND_FILE),
! errhint("The database server will regularly poll the pg_xlog subdirectory to check for files placed there.")));
! }
! else
! {
! if (recoveryRestoreCommand == NULL)
! ereport(FATAL,
! (errmsg("recovery command file \"%s\" must specify restore_command when standby mode is not enabled",
! RECOVERY_COMMAND_FILE)));
! }
!
! /* Enable fetching from archive recovery area */
! InArchiveRecovery = true;
/*
* If user specified recovery_target_timeline, validate it or compute the
--- 5300,5346 ----
for (item = head; item; item = item->next)
{
if (strcmp(item->name, "restore_command") == 0)
! SetConfigOption("restore_command", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "recovery_end_command") == 0)
! SetConfigOption("recovery_end_command", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "archive_cleanup_command") == 0)
! SetConfigOption("archive_cleanup_command", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "pause_at_recovery_target") == 0)
! SetConfigOption("pause_at_recovery_target", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "recovery_target_timeline") == 0)
! SetConfigOption("recovery_target_timeline", item->value, PGC_POSTMASTER, PGC_S_OVERRIDE);
else if (strcmp(item->name, "recovery_target_xid") == 0)
! SetConfigOption("recovery_target_xid", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "recovery_target_time") == 0)
! SetConfigOption("recovery_target_time", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "recovery_target_name") == 0)
! SetConfigOption("recovery_target_name", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "recovery_target_inclusive") == 0)
! SetConfigOption("recovery_target_inclusive", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "standby_mode") == 0)
! SetConfigOption("standby_mode", item->value, PGC_POSTMASTER, PGC_S_OVERRIDE);
else if (strcmp(item->name, "primary_conninfo") == 0)
! SetConfigOption("primary_conninfo", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else if (strcmp(item->name, "trigger_file") == 0)
! SetConfigOption("trigger_file", item->value, PGC_SIGHUP, PGC_S_OVERRIDE);
else
ereport(FATAL,
(errmsg("unrecognized recovery parameter \"%s\"",
item->name)));
}
+ /* Enable fetching from archive recovery area */
+ InArchiveRecovery = true;
+
/*
* Check for compulsory parameters
*/
! if (standby_mode && !restore_command[0] && !primary_conninfo[0])
! ereport(WARNING,
! (errmsg("neither primary_conninfo nor restore_command is specified"),
! errhint("The database server will regularly poll the pg_xlog subdirectory to "
! "check for files placed there until either of them is set in postgresql.conf.")));
! CheckRestoreCommandSet();
/*
* If user specified recovery_target_timeline, validate it or compute the
***************
*** 5482,5497 **** readRecoveryCommandFile(void)
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (rtliGiven)
{
! if (rtli)
{
/* Timeline 1 does not have a history file, all else should */
! if (rtli != 1 && !existsTimeLineHistory(rtli))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! rtli)));
! recoveryTargetTLI = rtli;
recoveryTargetIsLatest = false;
}
else
--- 5348,5363 ----
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (strcmp(recovery_target_timeline_string, "") != 0)
{
! if (recovery_target_timeline)
{
/* Timeline 1 does not have a history file, all else should */
! if (recovery_target_timeline != 1 && !existsTimeLineHistory(recovery_target_timeline))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! recovery_target_timeline)));
! recoveryTargetTLI = recovery_target_timeline;
recoveryTargetIsLatest = false;
}
else
***************
*** 5646,5652 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
/* Do we have a PITR target at all? */
! if (recoveryTarget == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
--- 5512,5518 ----
return false;
/* Do we have a PITR target at all? */
! if (recovery_target == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
***************
*** 5657,5663 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
}
! if (recoveryTarget == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
--- 5523,5529 ----
return false;
}
! if (recovery_target == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
***************
*** 5668,5687 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recoveryTargetXid);
if (stopsHere)
! *includeThis = recoveryTargetInclusive;
}
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recoveryTargetName) == 0);
/*
! * Ignore recoveryTargetInclusive because this is not a transaction
* record
*/
*includeThis = false;
--- 5534,5553 ----
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recovery_target_xid);
if (stopsHere)
! *includeThis = recovery_target_inclusive;
}
! else if (recovery_target == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recovery_target_name) == 0);
/*
! * Ignore recovery_target_inclusive because this is not a transaction
* record
*/
*includeThis = false;
***************
*** 5693,5708 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recoveryTargetInclusive)
! stopsHere = (recordXtime > recoveryTargetTime);
else
! stopsHere = (recordXtime >= recoveryTargetTime);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
--- 5559,5575 ----
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recovery_target_inclusive)
! stopsHere = (recordXtime > recovery_target_time);
else
! stopsHere = (recordXtime >= recovery_target_time);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
+ recoveryStopTarget = recovery_target;
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
***************
*** 6003,6008 **** CheckRequiredParameterValues(void)
--- 5870,5888 ----
}
/*
+ * Check to see if restore_command must be set for archive recovery
+ * when standby mode is not enabled
+ */
+ static void
+ CheckRestoreCommandSet(void)
+ {
+ if (InArchiveRecovery && !standby_mode && !restore_command[0])
+ ereport(FATAL,
+ (errmsg("restore_command must be specified for archive recovery "
+ "when standby mode is not enabled")));
+ }
+
+ /*
* This must be called ONCE during postmaster or standalone-backend startup
*/
void
***************
*** 6122,6148 **** StartupXLOG(void)
* see them
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
! strncpy(XLogCtl->archiveCleanupCommand,
! archiveCleanupCommand ? archiveCleanupCommand : "",
! sizeof(XLogCtl->archiveCleanupCommand));
if (InArchiveRecovery)
{
! if (StandbyMode)
ereport(LOG,
(errmsg("entering standby mode")));
- else if (recoveryTarget == RECOVERY_TARGET_XID)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to XID %u",
- recoveryTargetXid)));
- else if (recoveryTarget == RECOVERY_TARGET_TIME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to %s",
- timestamptz_to_str(recoveryTargetTime))));
- else if (recoveryTarget == RECOVERY_TARGET_NAME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to \"%s\"",
- recoveryTargetName)));
else
ereport(LOG,
(errmsg("starting archive recovery")));
--- 6002,6016 ----
* see them
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
! strncpy(XLogCtl->archive_cleanup_command,
! archive_cleanup_command,
! sizeof(XLogCtl->archive_cleanup_command));
if (InArchiveRecovery)
{
! if (standby_mode)
ereport(LOG,
(errmsg("entering standby mode")));
else
ereport(LOG,
(errmsg("starting archive recovery")));
***************
*** 6152,6158 **** StartupXLOG(void)
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (StandbyMode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
--- 6020,6026 ----
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (standby_mode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
***************
*** 6210,6216 **** StartupXLOG(void)
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (StandbyMode)
{
/*
* The last valid checkpoint record required for a streaming
--- 6078,6084 ----
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (standby_mode)
{
/*
* The last valid checkpoint record required for a streaming
***************
*** 6563,6569 **** StartupXLOG(void)
* Pause only if users can connect to send a resume
* message
*/
! if (recoveryPauseAtTarget && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
--- 6431,6437 ----
* Pause only if users can connect to send a resume
* message
*/
! if (pause_at_recovery_target && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
***************
*** 6662,6668 **** StartupXLOG(void)
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (StandbyMode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
--- 6530,6536 ----
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (standby_mode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
***************
*** 6670,6676 **** StartupXLOG(void)
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! StandbyMode = false;
/*
* Re-fetch the last valid or last applied record, so we can identify the
--- 6538,6544 ----
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! SetConfigOption("standby_mode", "false", PGC_POSTMASTER, PGC_S_OVERRIDE);
/*
* Re-fetch the last valid or last applied record, so we can identify the
***************
*** 6863,6870 **** StartupXLOG(void)
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recoveryEndCommand)
! ExecuteRecoveryCommand(recoveryEndCommand,
"recovery_end_command",
true);
}
--- 6731,6738 ----
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recovery_end_command[0])
! ExecuteRecoveryCommand(recovery_end_command,
"recovery_end_command",
true);
}
***************
*** 8187,8194 **** CreateRestartPoint(int flags)
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (XLogCtl->archiveCleanupCommand[0])
! ExecuteRecoveryCommand(XLogCtl->archiveCleanupCommand,
"archive_cleanup_command",
false);
--- 8055,8062 ----
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (XLogCtl->archive_cleanup_command[0])
! ExecuteRecoveryCommand(XLogCtl->archive_cleanup_command,
"archive_cleanup_command",
false);
***************
*** 10019,10024 **** HandleStartupProcInterrupts(void)
--- 9887,9903 ----
{
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /* Check for compulsory parameter */
+ CheckRestoreCommandSet();
+
+ /*
+ * If primary_conninfo has been changed while walreceiver
+ * is running, stop walreceiver so that it restarts replication
+ * using new primary_conninfo.
+ */
+ if (IsPrimaryConninfoChanged() && WalRcvInProgress())
+ ShutdownWalRcv();
}
/*
***************
*** 10143,10149 **** XLogPageRead(XLogRecPtr *RecPtr, int emode, bool fetching_ckpt,
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (StandbyMode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
--- 10022,10028 ----
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (standby_mode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
***************
*** 10165,10171 **** retry:
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (StandbyMode)
{
/*
* In standby mode, wait for the requested record to become
--- 10044,10050 ----
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (standby_mode)
{
/*
* In standby mode, wait for the requested record to become
***************
*** 10329,10339 **** retry:
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (PrimaryConnInfo)
{
RequestXLogStreaming(
fetching_ckpt ? RedoStartLSN : *RecPtr,
! PrimaryConnInfo);
continue;
}
}
--- 10208,10218 ----
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (primary_conninfo[0])
{
RequestXLogStreaming(
fetching_ckpt ? RedoStartLSN : *RecPtr,
! primary_conninfo);
continue;
}
}
***************
*** 10476,10482 **** next_record_is_invalid:
readSource = 0;
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return false;
--- 10355,10361 ----
readSource = 0;
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return false;
***************
*** 10548,10562 **** CheckForStandbyTrigger(void)
return true;
}
! if (TriggerFile == NULL)
return false;
! if (stat(TriggerFile, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", TriggerFile)));
ShutdownWalRcv();
! unlink(TriggerFile);
triggered = true;
return true;
}
--- 10427,10441 ----
return true;
}
! if (!trigger_file[0])
return false;
! if (stat(trigger_file, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", trigger_file)));
ShutdownWalRcv();
! unlink(trigger_file);
triggered = true;
return true;
}
*** a/src/backend/replication/walreceiverfuncs.c
--- b/src/backend/replication/walreceiverfuncs.c
***************
*** 109,114 **** WalRcvInProgress(void)
--- 109,131 ----
}
/*
+ * Is the current primary_conninfo different from that which walreceiver is using?
+ */
+ bool
+ IsPrimaryConninfoChanged(void)
+ {
+ /* use volatile pointer to prevent code rearrangement */
+ volatile WalRcvData *walrcv = WalRcv;
+ char conninfo[MAXCONNINFO];
+
+ SpinLockAcquire(&walrcv->mutex);
+ strlcpy(conninfo, (char *) walrcv->conninfo, MAXCONNINFO);
+ SpinLockRelease(&walrcv->mutex);
+
+ return strncmp(conninfo, primary_conninfo, MAXCONNINFO) != 0;
+ }
+
+ /*
* Stop walreceiver (if running) and wait for it to die.
*/
void
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 30,35 ****
--- 30,36 ----
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
+ #include "access/xlog_internal.h"
#include "catalog/namespace.h"
#include "commands/async.h"
#include "commands/prepare.h"
***************
*** 205,210 **** static bool check_application_name(char **newval, void **extra, GucSource source
--- 206,219 ----
static void assign_application_name(const char *newval, void *extra);
static const char *show_unix_socket_permissions(void);
static const char *show_log_file_mode(void);
+ static bool check_recovery_target_xid(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_xid(const char *newval, void *extra);
+ static bool check_recovery_target_name(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_name(const char *newval, void *extra);
+ static bool check_recovery_target_time(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_time(const char *newval, void *extra);
+ static bool check_recovery_target_timeline(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_timeline(const char *newval, void *extra);
static char *config_enum_get_options(struct config_enum * record,
const char *prefix, const char *suffix,
***************
*** 476,481 **** static int wal_block_size;
--- 485,492 ----
static int wal_segment_size;
static bool integer_datetimes;
static int effective_io_concurrency;
+ static char *recovery_target_xid_string;
+ static char *recovery_target_time_string;
/* should be static, but commands/variable.c needs to get at this */
char *role_string;
***************
*** 555,560 **** const char *const config_group_names[] =
--- 566,575 ----
gettext_noop("Write-Ahead Log / Checkpoints"),
/* WAL_ARCHIVING */
gettext_noop("Write-Ahead Log / Archiving"),
+ /* WAL_ARCHIVE_RECOVERY */
+ gettext_noop("Write-Ahead Log / Archive Recovery"),
+ /* WAL_RECOVERY_TARGET */
+ gettext_noop("Write-Ahead Log / Recovery Target"),
/* REPLICATION */
gettext_noop("Replication"),
/* REPLICATION_SENDING */
***************
*** 1365,1370 **** static struct config_bool ConfigureNamesBool[] =
--- 1380,1415 ----
},
{
+ {"recovery_target_inclusive", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether to include or exclude transaction with recovery target."),
+ NULL
+ },
+ &recovery_target_inclusive,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"pause_at_recovery_target", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether recovery should pause when the recovery target is reached."),
+ NULL
+ },
+ &pause_at_recovery_target,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"standby_mode", PGC_POSTMASTER, REPLICATION_STANDBY,
+ gettext_noop("Sets whether to start the server as a standby."),
+ NULL
+ },
+ &standby_mode,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"hot_standby", PGC_POSTMASTER, REPLICATION_STANDBY,
gettext_noop("Allows connections and queries during recovery."),
NULL
***************
*** 2521,2526 **** static struct config_string ConfigureNamesString[] =
--- 2566,2661 ----
},
{
+ {"restore_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will retrieve an archived WAL file."),
+ NULL
+ },
+ &restore_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"archive_cleanup_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed at every restartpoint."),
+ NULL
+ },
+ &archive_cleanup_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_end_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed once only at the end of recovery."),
+ NULL
+ },
+ &recovery_end_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_target_xid", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the transaction ID up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_xid_string,
+ "",
+ check_recovery_target_xid, assign_recovery_target_xid, NULL
+ },
+
+ {
+ {"recovery_target_name", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the named restore point."),
+ NULL
+ },
+ &recovery_target_name,
+ "",
+ check_recovery_target_name, assign_recovery_target_name, NULL
+ },
+
+ {
+ {"recovery_target_time", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the time stamp up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_time_string,
+ "",
+ check_recovery_target_time, assign_recovery_target_time, NULL
+ },
+
+ {
+ {"recovery_target_timeline", PGC_POSTMASTER, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets recoverying into a particular timeline."),
+ NULL
+ },
+ &recovery_target_timeline_string,
+ "",
+ check_recovery_target_timeline, assign_recovery_target_timeline, NULL
+ },
+
+ {
+ {"primary_conninfo", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the connection string to be used to connect with the primary."),
+ NULL
+ },
+ &primary_conninfo,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"trigger_file", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the trigger file whose presence ends recovery in the standby."),
+ NULL
+ },
+ &trigger_file,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
{"client_encoding", PGC_USERSET, CLIENT_CONN_LOCALE,
gettext_noop("Sets the client's character set encoding."),
NULL,
***************
*** 8678,8681 **** show_log_file_mode(void)
--- 8813,8965 ----
return buf;
}
+ static bool
+ check_recovery_target_xid(char **newval, void **extra, GucSource source)
+ {
+ TransactionId xid;
+ TransactionId *myextra;
+
+ errno = 0;
+ xid = (TransactionId) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_xid is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+
+ myextra = (TransactionId *) guc_malloc(ERROR, sizeof(TransactionId));
+ *myextra = xid;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_xid(const char *newval, void *extra)
+ {
+ recovery_target_xid = *((TransactionId *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_name(char **newval, void **extra, GucSource source)
+ {
+ if (strlen(*newval) >= MAXFNAMELEN)
+ {
+ GUC_check_errdetail("\"recovery_target_name\" is too long (maximum %d characters)",
+ MAXFNAMELEN - 1);
+ return false;
+ }
+ return true;
+ }
+
+ static void
+ assign_recovery_target_name(const char *newval, void *extra)
+ {
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (newval[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_time(char **newval, void **extra, GucSource source)
+ {
+ TimestampTz time;
+ TimestampTz *myextra;
+ MemoryContext oldcontext = CurrentMemoryContext;
+
+ PG_TRY();
+ {
+ time = (strcmp(*newval, "") == 0) ?
+ 0 :
+ DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
+ CStringGetDatum(*newval),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ }
+ PG_CATCH();
+ {
+ ErrorData *edata;
+
+ /* Save error info */
+ MemoryContextSwitchTo(oldcontext);
+ edata = CopyErrorData();
+ FlushErrorState();
+
+ /* Pass the error message */
+ GUC_check_errdetail("%s", edata->message);
+ FreeErrorData(edata);
+ return false;
+ }
+ PG_END_TRY();
+
+ myextra = (TimestampTz *) guc_malloc(ERROR, sizeof(TimestampTz));
+ *myextra = time;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_time(const char *newval, void *extra)
+ {
+ recovery_target_time = *((TimestampTz *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_timeline(char **newval, void **extra, GucSource source)
+ {
+ TimeLineID tli = 0;
+ TimeLineID *myextra;
+
+ if (strcmp(*newval, "latest") == 0)
+ tli = 0;
+ else
+ {
+ errno = 0;
+ tli = (TimeLineID) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_timeline is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+ }
+
+ myextra = (TimeLineID *) guc_malloc(ERROR, sizeof(TimeLineID));
+ *myextra = tli;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_timeline(const char *newval, void *extra)
+ {
+ recovery_target_timeline = *((TimeLineID *) extra);
+ }
+
#include "guc-file.c"
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 192,197 ****
--- 192,213 ----
#archive_timeout = 0 # force a logfile segment switch after this
# number of seconds; 0 disables
+ # - Archive Recovery -
+ #restore_command = '' # command to use to restore an archived logfile segment
+ # placeholders: %p = path of file to restore
+ # %f = file name only
+ # e.g. 'cp /mnt/server/archivedir/%f %p'
+ #archive_cleanup_command = '' # command to execute at every restartpoint
+ #recovery_end_command = '' # command to execute at completion of recovery
+
+ # - Recovery Target -
+ #recovery_target_xid = ''
+ #recovery_target_name = ''
+ #recovery_target_time = ''
+ #recovery_target_inclusive = on
+ #pause_at_recovery_target = on
+ #recovery_target_timeline = ''
+
#------------------------------------------------------------------------------
# REPLICATION
***************
*** 219,224 ****
--- 235,244 ----
# These settings are ignored on a master server
+ #standby_mode = off # "on" starts the server as a standby
+ # (change requires restart)
+ #primary_conninfo = '' # connection string to connect to the master
+ #trigger_file = '' # trigger file to promote the standby
#hot_standby = off # "on" allows queries during recovery
# (change requires restart)
#max_standby_archive_delay = 30s # max delay before canceling queries
*** a/src/include/access/xlog.h
--- b/src/include/access/xlog.h
***************
*** 198,203 **** extern bool XLogArchiveMode;
--- 198,217 ----
extern char *XLogArchiveCommand;
extern bool EnableHotStandby;
extern bool log_checkpoints;
+ extern char *restore_command;
+ extern char *archive_cleanup_command;
+ extern char *recovery_end_command;
+ extern bool standby_mode;
+ extern char *primary_conninfo;
+ extern char *trigger_file;
+ extern RecoveryTargetType recovery_target;
+ extern TransactionId recovery_target_xid;
+ extern TimestampTz recovery_target_time;
+ extern char *recovery_target_name;
+ extern bool recovery_target_inclusive;
+ extern bool pause_at_recovery_target;
+ extern char *recovery_target_timeline_string;
+ extern TimeLineID recovery_target_timeline;
/* WAL levels */
typedef enum WalLevel
*** a/src/include/replication/walreceiver.h
--- b/src/include/replication/walreceiver.h
***************
*** 110,115 **** extern Size WalRcvShmemSize(void);
--- 110,116 ----
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
+ extern bool IsPrimaryConninfoChanged(void);
extern void RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
*** a/src/include/utils/guc_tables.h
--- b/src/include/utils/guc_tables.h
***************
*** 68,73 **** enum config_group
--- 68,75 ----
WAL_SETTINGS,
WAL_CHECKPOINTS,
WAL_ARCHIVING,
+ WAL_ARCHIVE_RECOVERY,
+ WAL_RECOVERY_TARGET,
REPLICATION,
REPLICATION_SENDING,
REPLICATION_MASTER,
On Fri, Sep 9, 2011 at 11:56, Fujii Masao <masao.fujii@gmail.com> wrote:
Hi,
http://archives.postgresql.org/pgsql-hackers/2010-12/msg02343.php
In previous discussion, we've reached the consensus that we should unite
recovery.conf and postgresql.conf. The attached patch does that. The
patch is WIP, I'll have to update the document, but if you notice something,
please feel free to comment.This patch allows us to specify recovery parameters like primary_conninfo
in postgresql.conf. You can change some of them without restarting
the server (i.e., by using "pg_ctl reload") and see the current settings by
SHOW command. But note that recovery.conf is still required as a status
file for archive recovery and standby server even if you put all the settings
in postgresql.conf. Recovery parameters in postgresql.conf only have effect
when recovery.conf exists.
Why exactly do we need to keep recovery.conf in this case?
For backward compatibility, this patch still allows us to specify recovery
parameters in recovery.conf. But, as in past years, you cannot reload
recovery.conf and see the current settings by SHOW command. We should
recommend to put all the settings in postgresql.conf and empty recovery.conf,
but you can also put all recovery parameters in recovery.conf.
Perhaps we should just have people use recovery.conf as an include
file in this case?
As a whole, I'm not sure I like the idea of having such critical
parameters in two different places. Do we really need to care about
the backwards compatibility of something like that this much?
If the same parameter is specified in both file, the setting in recovery.conf
overrides that in postgresql.conf. In this case, SHOW command displays
the settings in postgresql.conf even though they are not used at all. Even if
Now, *that* is just broken, IMHO. The SHOW command should show what is
actually in use, period. Everything is guaranteed to confuse the hell
out of anybody trying to use it.
you change the settings in postgresql.conf and reload it, they have no effect
because the settings in recovery.conf are used preferentially.These limitations on recovery.conf might confuse users. But since most
users will put all the settings in postgresql.conf if we recommend to do that,
I don't think that it's worth complicating the source code for getting rid of
those limitations.
I think we should just get rid of supporting them in recovery.conf
completely. Recovery settings are critical, and making that confusing
to the user is not a good thing.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Fri, Sep 9, 2011 at 7:21 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Fri, Sep 9, 2011 at 11:56, Fujii Masao <masao.fujii@gmail.com> wrote:
Hi,
http://archives.postgresql.org/pgsql-hackers/2010-12/msg02343.php
In previous discussion, we've reached the consensus that we should unite
recovery.conf and postgresql.conf. The attached patch does that. The
patch is WIP, I'll have to update the document, but if you notice something,
please feel free to comment.This patch allows us to specify recovery parameters like primary_conninfo
in postgresql.conf. You can change some of them without restarting
the server (i.e., by using "pg_ctl reload") and see the current settings by
SHOW command. But note that recovery.conf is still required as a status
file for archive recovery and standby server even if you put all the settings
in postgresql.conf. Recovery parameters in postgresql.conf only have effect
when recovery.conf exists.Why exactly do we need to keep recovery.conf in this case?
PostgreSQL automatically reinitializes after a backend crash. Imagine the case
where this happens after archive recovery. Since restore_command is left set
in postgresql.conf and it still has effect (if we've completely removed
recovery.conf), the reinitialization will makes the server go into
archive recovery
mode again unexpectedly.
We can avoid this problem by using recovery.conf. At the end of
archive recovery,
recovery.conf is renamed to recovery.done. So when the reinitialization happens,
recovery.conf doesn't exist and restore_command has no effect. Which prevents
the server from going into archive recovery mode again.
For backward compatibility, this patch still allows us to specify recovery
parameters in recovery.conf. But, as in past years, you cannot reload
recovery.conf and see the current settings by SHOW command. We should
recommend to put all the settings in postgresql.conf and empty recovery.conf,
but you can also put all recovery parameters in recovery.conf.Perhaps we should just have people use recovery.conf as an include
file in this case?
Yeah, that's my first idea, but I've given up adopting that. The problem is that
recovery.conf is renamed to recovery.done at the end of recovery. This means
that all the settings in recovery.conf are reset when archive recovery ends.
If you run "pg_ctl reload" after archive recovery, you'll get something like the
following error message and the reload will always fail;
LOG: parameter "standby_mode" cannot be changed without
restarting the server
As a whole, I'm not sure I like the idea of having such critical
parameters in two different places. Do we really need to care about
the backwards compatibility of something like that this much?
I don't have strong opinion about this. But in previous discussion, Simon argued
that we should.
http://archives.postgresql.org/pgsql-hackers/2010-10/msg00017.php
If the same parameter is specified in both file, the setting in recovery.conf
overrides that in postgresql.conf. In this case, SHOW command displays
the settings in postgresql.conf even though they are not used at all. Even ifNow, *that* is just broken, IMHO. The SHOW command should show what is
actually in use, period. Everything is guaranteed to confuse the hell
out of anybody trying to use it.
I'm afraid that we have to add very complicated code to fix that problem.
Though of course we can easily fix the problem if we don't care about
the backward compatibility.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Fri, Sep 9, 2011 at 13:15, Fujii Masao <masao.fujii@gmail.com> wrote:
On Fri, Sep 9, 2011 at 7:21 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Fri, Sep 9, 2011 at 11:56, Fujii Masao <masao.fujii@gmail.com> wrote:
Hi,
http://archives.postgresql.org/pgsql-hackers/2010-12/msg02343.php
In previous discussion, we've reached the consensus that we should unite
recovery.conf and postgresql.conf. The attached patch does that. The
patch is WIP, I'll have to update the document, but if you notice something,
please feel free to comment.This patch allows us to specify recovery parameters like primary_conninfo
in postgresql.conf. You can change some of them without restarting
the server (i.e., by using "pg_ctl reload") and see the current settings by
SHOW command. But note that recovery.conf is still required as a status
file for archive recovery and standby server even if you put all the settings
in postgresql.conf. Recovery parameters in postgresql.conf only have effect
when recovery.conf exists.Why exactly do we need to keep recovery.conf in this case?
PostgreSQL automatically reinitializes after a backend crash. Imagine the case
where this happens after archive recovery. Since restore_command is left set
in postgresql.conf and it still has effect (if we've completely removed
recovery.conf), the reinitialization will makes the server go into
archive recovery
mode again unexpectedly.We can avoid this problem by using recovery.conf. At the end of
archive recovery,
recovery.conf is renamed to recovery.done. So when the reinitialization happens,
recovery.conf doesn't exist and restore_command has no effect. Which prevents
the server from going into archive recovery mode again.
Ah, ugh. Good point.
I have to wonder though, if it wouldn't be less confusing to just get
rid of recovery.conf and use a *different* file for this. Just to make
it clear it's not a config file, but just a boolean exists/notexists
state.
For backward compatibility, this patch still allows us to specify recovery
parameters in recovery.conf. But, as in past years, you cannot reload
recovery.conf and see the current settings by SHOW command. We should
recommend to put all the settings in postgresql.conf and empty recovery.conf,
but you can also put all recovery parameters in recovery.conf.Perhaps we should just have people use recovery.conf as an include
file in this case?Yeah, that's my first idea, but I've given up adopting that. The problem is that
recovery.conf is renamed to recovery.done at the end of recovery. This means
that all the settings in recovery.conf are reset when archive recovery ends.
If you run "pg_ctl reload" after archive recovery, you'll get something like the
following error message and the reload will always fail;LOG: parameter "standby_mode" cannot be changed without
restarting the server
Good point. And I believe another argument for actually using
completely different files ;)
As a whole, I'm not sure I like the idea of having such critical
parameters in two different places. Do we really need to care about
the backwards compatibility of something like that this much?I don't have strong opinion about this. But in previous discussion, Simon argued
that we should.
http://archives.postgresql.org/pgsql-hackers/2010-10/msg00017.php
I see his argument, but I think reduced confusion is a more important one.
If the same parameter is specified in both file, the setting in recovery.conf
overrides that in postgresql.conf. In this case, SHOW command displays
the settings in postgresql.conf even though they are not used at all. Even ifNow, *that* is just broken, IMHO. The SHOW command should show what is
actually in use, period. Everything is guaranteed to confuse the hell
out of anybody trying to use it.I'm afraid that we have to add very complicated code to fix that problem.
Though of course we can easily fix the problem if we don't care about
the backward compatibility.
That is an even bigger reason to drop backwards compatibility. Unless
someone else can come up with a neat way of fixing it.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
I have to wonder though, if it wouldn't be less confusing to just get
rid of recovery.conf and use a *different* file for this. Just to make
it clear it's not a config file, but just a boolean exists/notexists
state.
+1. If it's not a configuration file anymore, it shouldn't be called
one.
regards, tom lane
On Fri, Sep 9, 2011 at 3:05 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
I have to wonder though, if it wouldn't be less confusing to just get
rid of recovery.conf and use a *different* file for this. Just to make
it clear it's not a config file, but just a boolean exists/notexists
state.+1. If it's not a configuration file anymore, it shouldn't be called
one.
+1 to rename file
+1 to overall concept, just thinking same myself, not looked at patch yet
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 9/9/11 7:05 AM, Tom Lane wrote:
Magnus Hagander <magnus@hagander.net> writes:
I have to wonder though, if it wouldn't be less confusing to just get
rid of recovery.conf and use a *different* file for this. Just to make
it clear it's not a config file, but just a boolean exists/notexists
state.+1. If it's not a configuration file anymore, it shouldn't be called
one.
I'm in favor of this. People are sufficiently confused by the existing
behavior that we're not going to confuse them further by changing it.
In fact: currently we have a "trigger file" concept in recovery.conf.
Might we unite the trigger file concept with the concept of having a
file to indicate that server is a replica on startup?
i.e. in postgresql.conf we have two settings:
replication_status = {master,standby}
failover_trigger_file = '' #defaults to $PGDATA/master_trigger
If replication_status = master, then the server is a standalone or
master node.
If replication_status = standby, then the server is a PITR, warm
standby, or hot standby replica.
If the failover_trigger_file is created and detected, then PostgreSQL
will reload in master mode. replication_status will get set to "master"
so that it's easy to tell this from the command line.
The above would be a lot easier to comprehend than the current behavior,
and would allow us all of the same options the current behavoir does.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Fri, Sep 9, 2011 at 6:40 PM, Josh Berkus <josh@agliodbs.com> wrote:
I'm in favor of this. People are sufficiently confused by the existing
behavior that we're not going to confuse them further by changing it.
Fwiw as someone who *was* confused previously, it now makes perfect
sense to me. "We have postgres.conf which always applies and then
recovery.conf which can have all the same options but they only apply
during recover". That's much clearer than "we have two configuration
files with two disjoint sets of options and good luck remembering
which options belong in which file". And it still serves a useful
purpose if you have options like recovery_target that you only want to
apply during recovery and then plan to remove.
--
greg
On Sat, Sep 10, 2011 at 01:07, Greg Stark <stark@mit.edu> wrote:
On Fri, Sep 9, 2011 at 6:40 PM, Josh Berkus <josh@agliodbs.com> wrote:
I'm in favor of this. People are sufficiently confused by the existing
behavior that we're not going to confuse them further by changing it.Fwiw as someone who *was* confused previously, it now makes perfect
sense to me. "We have postgres.conf which always applies and then
recovery.conf which can have all the same options but they only apply
during recover". That's much clearer than "we have two configuration
But it only means that for *some* options. Which certainly doesn't
reduce confusion.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Fri, Sep 9, 2011 at 8:27 PM, Magnus Hagander <magnus@hagander.net> wrote:
If the same parameter is specified in both file, the setting in recovery.conf
overrides that in postgresql.conf. In this case, SHOW command displays
the settings in postgresql.conf even though they are not used at all. Even ifNow, *that* is just broken, IMHO. The SHOW command should show what is
actually in use, period. Everything is guaranteed to confuse the hell
out of anybody trying to use it.I'm afraid that we have to add very complicated code to fix that problem.
Though of course we can easily fix the problem if we don't care about
the backward compatibility.That is an even bigger reason to drop backwards compatibility. Unless
someone else can come up with a neat way of fixing it.
Agreed. After a little thought, it's came to mind that we might be
able to simply
fix that problem by making the postmaster instead of the startup process read
recovery.conf; The postmaster reads recovery.conf before it invokes the startup
process, and sets the global variables of GUC to the specified values. The child
processes including the startup process inherit the global variables of GUC, so
SHOW can return the exact setting value and the startup process can use them
to do a recovery. Even in EXEC_BACKEND case, we can adopt this idea because
those global variables are inherited from parent to children via the
save/restore_backend_variables functions.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Sat, Sep 10, 2011 at 12:18 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Fri, Sep 9, 2011 at 3:05 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
I have to wonder though, if it wouldn't be less confusing to just get
rid of recovery.conf and use a *different* file for this. Just to make
it clear it's not a config file, but just a boolean exists/notexists
state.+1. If it's not a configuration file anymore, it shouldn't be called
one.+1 to rename file
+1 to overall concept, just thinking same myself, not looked at patch yet
Are you still thinking the backward-compatibility (i.e., the
capability to specify
recovery parameters in recovery.conf) is required?
Even if we maintain the backward-compatibility, if we rename the file, ISTM
that the tools which depends on recovery.conf would need to be changed so
that <file-with-new-name> is used instead of recovery.conf. So I'm not sure
whether leaving the capability to set parameters in recovery.conf as it is is
really worth supporting or not. If we would like to make our effort for the
backward-compatibility more worthwhile, we might have to make the path of
the file configurable as a user can set it to "recovery.conf".
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On tis, 2011-09-13 at 14:46 +0900, Fujii Masao wrote:
Are you still thinking the backward-compatibility (i.e., the
capability to specify recovery parameters in recovery.conf) is
required?
I think parameters related to a particular recovery, e.g.,
recovery_target_time, fit better into a recovery.conf that is renamed
after the recovery is complete. That was the original idea, after all.
Everything that is a permanent setting across multiple recovery
attempts, and anything related to replication, better fits elsewhere.
On Tue, Sep 13, 2011 at 2:51 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On tis, 2011-09-13 at 14:46 +0900, Fujii Masao wrote:
Are you still thinking the backward-compatibility (i.e., the
capability to specify recovery parameters in recovery.conf) is
required?I think parameters related to a particular recovery, e.g.,
recovery_target_time, fit better into a recovery.conf that is renamed
after the recovery is complete. That was the original idea, after all.Everything that is a permanent setting across multiple recovery
attempts, and anything related to replication, better fits elsewhere.
I've just been thinking that a better way would be to make
recovery.conf an extension of postgresql.conf when we are in archive
recovery.
So treat postgresql.conf as if it has an automatic "include
recovery.conf" in it. The file format is the same.
That way we don't need to change existing behaviour, so any software
that relies upon this will still work, but we gain the additional
ability to reload values in recovery,conf (where appropriate).
We can change the .sample files to show parameters that make more
sense in one or the other file, rather than making it a hard
requirement for them to appear in specific files which will be a real
pain in the ass.
Internal changes would then be to move existing recovery.conf
parameters into guc.c and revise the manual accordingly.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Sep 14, 2011 at 1:10 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Sep 13, 2011 at 2:51 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On tis, 2011-09-13 at 14:46 +0900, Fujii Masao wrote:
Are you still thinking the backward-compatibility (i.e., the
capability to specify recovery parameters in recovery.conf) is
required?I think parameters related to a particular recovery, e.g.,
recovery_target_time, fit better into a recovery.conf that is renamed
after the recovery is complete. That was the original idea, after all.Everything that is a permanent setting across multiple recovery
attempts, and anything related to replication, better fits elsewhere.I've just been thinking that a better way would be to make
recovery.conf an extension of postgresql.conf when we are in archive
recovery.So treat postgresql.conf as if it has an automatic "include
recovery.conf" in it. The file format is the same.That way we don't need to change existing behaviour, so any software
that relies upon this will still work, but we gain the additional
ability to reload values in recovery,conf (where appropriate).We can change the .sample files to show parameters that make more
sense in one or the other file, rather than making it a hard
requirement for them to appear in specific files which will be a real
pain in the ass.Internal changes would then be to move existing recovery.conf
parameters into guc.c and revise the manual accordingly.
Sounds reasonable. I'll revise the patch that way.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On tis, 2011-09-13 at 17:10 +0100, Simon Riggs wrote:
So treat postgresql.conf as if it has an automatic "include
recovery.conf" in it. The file format is the same.
Sounds good. That would also have the merit that you could use, say,
different memory settings during recovery.
On Wed, Sep 14, 2011 at 6:33 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On tis, 2011-09-13 at 17:10 +0100, Simon Riggs wrote:
So treat postgresql.conf as if it has an automatic "include
recovery.conf" in it. The file format is the same.Sounds good. That would also have the merit that you could use, say,
different memory settings during recovery.
One problem of this is that recovery.conf is relative to the configuration
file directory instead of data directory if we treat it as an "include" file.
If your configuration file directory is the same as the data directory, you
don't need to worry about this problem. But if you set data_directory to
the directory other than ConfigDir, the problem might break your tool
which depends on recovery.conf under the data directory.
If we'd like to treat recovery.conf as if it's under the data directory, I'm
afraid that we should add complicated code to parse recovery.conf after
the value of data_directory has been determined from postgresql.conf.
Furthermore, what if recovery.conf has another setting of data_directory?
Since recovery.conf is a configuration file, it's intuitive for me to put it
in configuration file directory rather than data directory. So I'm not inclined
to treat recovery.conf as if it's under data directory. Is this OK?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Fujii Masao <masao.fujii@gmail.com> writes:
If we'd like to treat recovery.conf as if it's under the data directory, I'm
afraid that we should add complicated code to parse recovery.conf after
the value of data_directory has been determined from postgresql.conf.
Furthermore, what if recovery.conf has another setting of data_directory?
We already do this dance for custom_variable_classes. IIRC the only
thing you have to care about is setting the GUC before any other one. I
guess you could just process the data_directory the same way, before the
main loop, and be done with it.
Since recovery.conf is a configuration file, it's intuitive for me to put it
in configuration file directory rather than data directory. So I'm not inclined
to treat recovery.conf as if it's under data directory. Is this OK?
I think that if we keep some compatibility with recovery.conf, then we
need to include finding it in the data_directory or the configuration
file directory, in that order, and with a LOG message that says where we
did find it.
I think we should *NOT* load both of them with some priority rules.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On tor, 2011-09-15 at 16:54 +0900, Fujii Masao wrote:
On Wed, Sep 14, 2011 at 6:33 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On tis, 2011-09-13 at 17:10 +0100, Simon Riggs wrote:
So treat postgresql.conf as if it has an automatic "include
recovery.conf" in it. The file format is the same.Sounds good. That would also have the merit that you could use, say,
different memory settings during recovery.One problem of this is that recovery.conf is relative to the configuration
file directory instead of data directory if we treat it as an "include" file.
Treat it *as if* it were an included file. A special included file if
you will.
If we'd like to treat recovery.conf as if it's under the data directory, I'm
afraid that we should add complicated code to parse recovery.conf after
the value of data_directory has been determined from postgresql.conf.
Furthermore, what if recovery.conf has another setting of data_directory?
Perhaps that could be addresses by inventing a new context setting
PGC_RECOVERY so that you can only set sane values.
Since recovery.conf is a configuration file, it's intuitive for me to put it
in configuration file directory rather than data directory. So I'm not inclined
to treat recovery.conf as if it's under data directory. Is this OK?
It's not a configuration file, because it disappears after use. (And a
lot of configuration file management systems would be really upset if we
had a configuration file that behaved that way.) The whole point of
this exercise is allowing the permanent configuration file parameters to
be moved to a real configuration file (postgresql.conf), while leaving
the temporary settings separately.
Alternatively, we could just forget about the whole thing and move
everything to postgresql.conf and treat recovery.conf as a simple empty
signal file. I don't know if that's necessarily better.
On Thu, Sep 15, 2011 at 5:58 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
Alternatively, we could just forget about the whole thing and move
everything to postgresql.conf and treat recovery.conf as a simple empty
signal file. I don't know if that's necessarily better.
Seems like it might be simpler.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Thu, Sep 15, 2011 at 8:04 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Sep 15, 2011 at 5:58 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
Alternatively, we could just forget about the whole thing and move
everything to postgresql.conf and treat recovery.conf as a simple empty
signal file. I don't know if that's necessarily better.Seems like it might be simpler.
It seems to need a bit more time until we've reached a consensus about
the treatment of recovery.conf. How about committing the core patch
first, and addressing the recovery.conf issue as a different patch later?
The attached patch provides a core part of the feature, i.e., it moves
every recovery parameters from recovery.conf to postgresql.conf.
Even if you create recovery.conf, the server doesn't read it automatically.
The patch renames recovery.conf to recovery.ready, so if you want to
enter archive recovery or standby mode, you need to create
recovery.ready file in the cluster data directory. Since recovery.ready is
just a signal file, its contents have no effect.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Attachments:
unite_recoveryconf_postgresqlconf_v1.patchtext/x-patch; charset=US-ASCII; name=unite_recoveryconf_postgresqlconf_v1.patchDownload
*** a/contrib/pg_archivecleanup/pg_archivecleanup.c
--- b/contrib/pg_archivecleanup/pg_archivecleanup.c
***************
*** 208,214 **** usage(void)
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "For use as archive_cleanup_command in recovery.conf when standby_mode = on:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
"e.g.\n"
" archive_cleanup_command = 'pg_archivecleanup /mnt/server/archiverdir %%r'\n");
--- 208,214 ----
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "For use as archive_cleanup_command in postgresql.conf when standby_mode = on:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
"e.g.\n"
" archive_cleanup_command = 'pg_archivecleanup /mnt/server/archiverdir %%r'\n");
*** a/contrib/pg_standby/pg_standby.c
--- b/contrib/pg_standby/pg_standby.c
***************
*** 531,537 **** usage(void)
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "Main intended use as restore_command in recovery.conf:\n"
" restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
"e.g.\n"
" restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
--- 531,537 ----
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "Main intended use as restore_command in postgresql.conf:\n"
" restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
"e.g.\n"
" restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
*** a/doc/src/sgml/backup.sgml
--- b/doc/src/sgml/backup.sgml
***************
*** 995,1002 **** SELECT pg_stop_backup();
</listitem>
<listitem>
<para>
! Create a recovery command file <filename>recovery.conf</> in the cluster
! data directory (see <xref linkend="recovery-config">). You might
also want to temporarily modify <filename>pg_hba.conf</> to prevent
ordinary users from connecting until you are sure the recovery was successful.
</para>
--- 995,1009 ----
</listitem>
<listitem>
<para>
! Set up recovery parameters in <filename>postgresql.conf</> (see
! <xref linkend="runtime-config-wal-archive-recovery"> and
! <xref linkend="runtime-config-wal-recovery-target">).
! </para>
! </listitem>
! <listitem>
! <para>
! Create a recovery status file <filename>recovery.ready</>
! in the cluster data directory. You might
also want to temporarily modify <filename>pg_hba.conf</> to prevent
ordinary users from connecting until you are sure the recovery was successful.
</para>
***************
*** 1008,1014 **** SELECT pg_stop_backup();
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
! <filename>recovery.conf</> to <filename>recovery.done</> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
</para>
--- 1015,1021 ----
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
! <filename>recovery.ready</> to <filename>recovery.done</> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
</para>
***************
*** 1024,1035 **** SELECT pg_stop_backup();
</para>
<para>
! The key part of all this is to set up a recovery configuration file that
! describes how you want to recover and how far the recovery should
! run. You can use <filename>recovery.conf.sample</> (normally
! located in the installation's <filename>share/</> directory) as a
! prototype. The one thing that you absolutely must specify in
! <filename>recovery.conf</> is the <varname>restore_command</>,
which tells <productname>PostgreSQL</> how to retrieve archived
WAL file segments. Like the <varname>archive_command</>, this is
a shell command string. It can contain <literal>%f</>, which is
--- 1031,1041 ----
</para>
<para>
! The key part of all this is to set up recovery parameters that
! specify how you want to recover and how far the recovery should
! run. The one thing that you absolutely must specify in
! <filename>postgresql.conf</> to recover from the backup is
! the <varname>restore_command</>,
which tells <productname>PostgreSQL</> how to retrieve archived
WAL file segments. Like the <varname>archive_command</>, this is
a shell command string. It can contain <literal>%f</>, which is
***************
*** 1085,1091 **** restore_command = 'cp /mnt/server/archivedir/%f %p'
<para>
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
! required stopping point in <filename>recovery.conf</>. You can specify
the stop point, known as the <quote>recovery target</>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
--- 1091,1097 ----
<para>
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
! required stopping point in <filename>postgresql.conf</>. You can specify
the stop point, known as the <quote>recovery target</>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
***************
*** 1182,1189 **** restore_command = 'cp /mnt/server/archivedir/%f %p'
The default behavior of recovery is to recover along the same timeline
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
! was itself generated after a recovery attempt), you need to specify the
! target timeline ID in <filename>recovery.conf</>. You cannot recover into
timelines that branched off earlier than the base backup.
</para>
</sect2>
--- 1188,1196 ----
The default behavior of recovery is to recover along the same timeline
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
! was itself generated after a recovery attempt), you need to set
! <xref linkend="guc-recovery-target-timeline"> to the
! target timeline ID in <filename>postgresql.conf</>. You cannot recover into
timelines that branched off earlier than the base backup.
</para>
</sect2>
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 1961,1966 **** SET ENABLE_SEQSCAN TO OFF;
--- 1961,2238 ----
</variablelist>
</sect2>
+ <sect2 id="runtime-config-wal-archive-recovery">
+ <title>Archive Recovery</title>
+
+ <variablelist>
+ <varlistentry id="guc-restore-command" xreflabel="restore_command">
+ <term><varname>restore_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>restore_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command to execute to retrieve an archived segment of
+ the WAL file series. This parameter is required for archive recovery,
+ but optional for streaming replication.
+ Any <literal>%f</> in the string is
+ replaced by the name of the file to retrieve from the archive,
+ and any <literal>%p</> is replaced by the copy destination path name
+ on the server.
+ (The path name is relative to the current working directory,
+ i.e., the cluster's data directory.)
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point. That is the earliest file that must be kept
+ to allow a restore to be restartable, so this information can be used
+ to truncate the archive to just the minimum required to support
+ restarting from the current restore. <literal>%r</> is typically only
+ used by warm-standby configurations
+ (see <xref linkend="warm-standby">).
+ Write <literal>%%</> to embed an actual <literal>%</> character.
+ </para>
+ <para>
+ It is important for the command to return a zero exit status
+ only if it succeeds. The command <emphasis>will</> be asked for file
+ names that are not present in the archive; it must return nonzero
+ when so asked. Examples:
+ <programlisting>
+ restore_command = 'cp /mnt/server/archivedir/%f "%p"'
+ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
+ </programlisting>
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-archive-cleanup-command" xreflabel="archive_cleanup_command">
+ <term><varname>archive_cleanup_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>archive_cleanup_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command that will be executed at every restartpoint.
+ The purpose of <varname>archive_cleanup_command</> is to
+ provide a mechanism for cleaning up old archived WAL files that
+ are no longer needed by the standby server.
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point.
+ That is the earliest file that must be <emphasis>kept</> to allow a
+ restore to be restartable, and so all files earlier than <literal>%r</>
+ may be safely removed.
+ This information can be used to truncate the archive to just the
+ minimum required to support restart from the current restore.
+ The <xref linkend="pgarchivecleanup"> module
+ is often used in <varname>archive_cleanup_command</> for
+ single-standby configurations, for example:
+ <programlisting>archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'</programlisting>
+ Note however that if multiple standby servers are restoring from the
+ same archive directory, you will need to ensure that you do not delete
+ WAL files until they are no longer needed by any of the servers.
+ <varname>archive_cleanup_command</> would typically be used in a
+ warm-standby configuration (see <xref linkend="warm-standby">).
+ Write <literal>%%</> to embed an actual <literal>%</> character in the
+ command.
+ </para>
+ <para>
+ If the command returns a non-zero exit status then a WARNING log
+ message will be written.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-end-command" xreflabel="recovery_end_command">
+ <term><varname>recovery_end_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_end_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command that will be executed once only
+ at the end of recovery. This parameter is optional. The purpose of the
+ <varname>recovery_end_command</> is to provide a mechanism for cleanup
+ following replication or recovery.
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point, like in <varname>archive_cleanup_command</>.
+ </para>
+ <para>
+ If the command returns a non-zero exit status then a WARNING log
+ message will be written and the database will proceed to start up
+ anyway. An exception is that if the command was terminated by a
+ signal, the database will not proceed with startup.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="runtime-config-wal-recovery-target">
+ <title>Recovery Target</title>
+
+ <variablelist>
+ <varlistentry id="guc-recovery-target-name" xreflabel="recovery_target_name">
+ <term><varname>recovery_target_name</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_name</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the named restore point, created with
+ <function>pg_create_restore_point()</> to which recovery will proceed.
+ At most one of <varname>recovery_target_name</>,
+ <varname>recovery_target_time</> or
+ <varname>recovery_target_xid</> can be specified. The default
+ value is an empty string, which will recover to the end of the WAL log.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-time" xreflabel="recovery_target_time">
+ <term><varname>recovery_target_time</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_time</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the time stamp up to which recovery will proceed.
+ This parameter must be specified in the date/time format
+ (see <xref linkend="datatype-datetime-input"> for details).
+ At most one of <varname>recovery_target_time</>,
+ <varname>recovery_target_name</> or
+ <varname>recovery_target_xid</> can be specified.
+ The default value is an empty string, which will recover to
+ the end of the WAL log. The precise stopping point is also
+ influenced by <varname>recovery_target_inclusive</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-xid" xreflabel="recovery_target_xid">
+ <term><varname>recovery_target_xid</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_xid</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the transaction ID up to which recovery will proceed.
+ Keep in mind that while transaction IDs are assigned sequentially
+ at transaction start, transactions can complete in a different
+ numeric order. The transactions that will be recovered are
+ those that committed before (and optionally including)
+ the specified one. At most one of <varname>recovery_target_xid</>,
+ <varname>recovery_target_name</> or
+ <varname>recovery_target_time</> can be specified.
+ The default value is an empty string, which will recover to the end of
+ the WAL log. The precise stopping point is also influenced by
+ <varname>recovery_target_inclusive</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-inclusive" xreflabel="recovery_target_inclusive">
+ <term><varname>recovery_target_inclusive</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_inclusive</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether we stop just after the specified recovery target
+ (<literal>on</>), or just before the recovery target (<literal>off</>).
+ Applies to both <varname>recovery_target_time</>
+ and <varname>recovery_target_xid</>, whichever one is
+ specified for this recovery. This indicates whether transactions
+ having exactly the target commit time or ID, respectively, will
+ be included in the recovery. Default is <literal>on</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-timeline" xreflabel="recovery_target_timeline">
+ <term><varname>recovery_target_timeline</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_timeline</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies recovering into a particular timeline. The default value is
+ an empty string, which will recover along the same timeline that was
+ current when the base backup was taken. Setting this to
+ <literal>latest</> recovers to the latest timeline found in the archive,
+ which is useful in a standby server. Other than that you only need to
+ set this parameter in complex re-recovery situations, where you need
+ to return to a state that itself was reached after a point-in-time
+ recovery. See <xref linkend="backup-timelines"> for discussion.
+ </para>
+ <para>
+ This parameter can only be set at server start. It only has effect
+ during archive recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-pause-at-recovery-target" xreflabel="pause_at_recovery_target">
+ <term><varname>pause_at_recovery_target</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>pause_at_recovery_target</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether recovery should pause when the recovery target
+ is reached. The default is <literal>on</>.
+ This is intended to allow queries to be executed against the
+ database to check if this recovery target is the most desirable
+ point for recovery. The paused state can be resumed by using
+ <function>pg_xlog_replay_resume()</> (See
+ <xref linkend="functions-recovery-control-table">), which then
+ causes recovery to end. If this recovery target is not the
+ desired stopping point, then shutdown the server, change the
+ recovery target settings to a later target and restart to
+ continue recovery.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode if recovery target is set.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
</sect1>
<sect1 id="runtime-config-replication">
***************
*** 2182,2187 **** SET ENABLE_SEQSCAN TO OFF;
--- 2454,2546 ----
<variablelist>
+ <varlistentry id="guc-standby-mode" xreflabel="standby_mode">
+ <term><varname>standby_mode</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>standby_mode</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether to start the <productname>PostgreSQL</> server as
+ a standby when recovery status file <filename>recovery.ready</> exists.
+ The default value is <literal>off</>.
+ If this parameter is <literal>on</>, the server will not
+ stop recovery when the end of archived WAL is reached,
+ but will keep trying to continue recovery by fetching new WAL segments
+ using <varname>restore_command</> and/or by connecting to
+ the primary server as specified by the <varname>primary_conninfo</>
+ setting.
+ </para>
+ <para>
+ This parameter can only be set at server start. It only has effect
+ if recovery status file <filename>recovery.ready</> exists.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-primary-conninfo" xreflabel="primary_conninfo">
+ <term><varname>primary_conninfo</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>primary_conninfo</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies a connection string to be used for the standby server
+ to connect with the primary. This string is in the format
+ accepted by the libpq <function>PQconnectdb</function> function,
+ described in <xref linkend="libpq-connect">. If any option is
+ unspecified in this string, then the corresponding environment
+ variable (see <xref linkend="libpq-envars">) is checked. If the
+ environment variable is not set either, then defaults are used.
+ If this parameter is an empty string (the default), no attempt is
+ made to connect to the master.
+ </para>
+ <para>
+ The connection string should specify the host name (or address)
+ of the primary server, as well as the port number if it is not
+ the same as the standby server's default.
+ Also specify a user name corresponding to a role that has the
+ <literal>REPLICATION</> and <literal>LOGIN</> privileges on the
+ primary (see
+ <xref linkend="streaming-replication-authentication">).
+ A password needs to be provided too, if the primary demands password
+ authentication. It can be provided in the
+ <varname>primary_conninfo</varname> string, or in a separate
+ <filename>~/.pgpass</> file on the standby server (use
+ <literal>replication</> as the database name).
+ Do not specify a database name in the
+ <varname>primary_conninfo</varname> string.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect in standby mode.
+ </para>
+ <para>
+ If this parameter is changed while replication is in progress,
+ the standby terminates replication, and then tries to restart
+ replication with new setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-trigger-file" xreflabel="trigger_file">
+ <term><varname>trigger_file</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>trigger_file</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies a trigger file whose presence ends recovery in the
+ standby. Even if this value is not set, you can still promote
+ the standby using <command>pg_ctl promote</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-hot-standby" xreflabel="hot_standby">
<term><varname>hot_standby</varname> (<type>boolean</type>)</term>
<indexterm>
*** a/doc/src/sgml/filelist.sgml
--- b/doc/src/sgml/filelist.sgml
***************
*** 42,48 ****
<!ENTITY manage-ag SYSTEM "manage-ag.sgml">
<!ENTITY monitoring SYSTEM "monitoring.sgml">
<!ENTITY regress SYSTEM "regress.sgml">
- <!ENTITY recovery-config SYSTEM "recovery-config.sgml">
<!ENTITY runtime SYSTEM "runtime.sgml">
<!ENTITY config SYSTEM "config.sgml">
<!ENTITY user-manag SYSTEM "user-manag.sgml">
--- 42,47 ----
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***************
*** 14134,14140 **** postgres=# select pg_start_backup('label_goes_here');
<function>pg_create_restore_point</> creates a named transaction log
record that can be used as recovery target, and returns the corresponding
transaction log location. The given name can then be used with
! <xref linkend="recovery-target-name"> to specify the point up to which
recovery will proceed. Avoid creating multiple restore points with the
same name, since recovery will stop at the first one whose name matches
the recovery target.
--- 14134,14140 ----
<function>pg_create_restore_point</> creates a named transaction log
record that can be used as recovery target, and returns the corresponding
transaction log location. The given name can then be used with
! <xref linkend="guc-recovery-target-name"> to specify the point up to which
recovery will proceed. Avoid creating multiple restore points with the
same name, since recovery will stop at the first one whose name matches
the recovery target.
*** a/doc/src/sgml/high-availability.sgml
--- b/doc/src/sgml/high-availability.sgml
***************
*** 591,597 **** protocol to make nodes agree on a serializable transactional order.
<para>
In standby mode, the server continuously applies WAL received from the
master server. The standby server can read WAL from a WAL archive
! (see <xref linkend="restore-command">) or directly from the master
over a TCP connection (streaming replication). The standby server will
also attempt to restore any WAL found in the standby cluster's
<filename>pg_xlog</> directory. That typically happens after a server
--- 591,597 ----
<para>
In standby mode, the server continuously applies WAL received from the
master server. The standby server can read WAL from a WAL archive
! (see <xref linkend="guc-restore-command">) or directly from the master
over a TCP connection (streaming replication). The standby server will
also attempt to restore any WAL found in the standby cluster's
<filename>pg_xlog</> directory. That typically happens after a server
***************
*** 658,665 **** protocol to make nodes agree on a serializable transactional order.
<para>
To set up the standby server, restore the base backup taken from primary
server (see <xref linkend="backup-pitr-recovery">). Create a recovery
! command file <filename>recovery.conf</> in the standby's cluster data
! directory, and turn on <varname>standby_mode</>. Set
<varname>restore_command</> to a simple command to copy files from
the WAL archive. If you plan to have multiple standby servers for high
availability purposes, set <varname>recovery_target_timeline</> to
--- 658,665 ----
<para>
To set up the standby server, restore the base backup taken from primary
server (see <xref linkend="backup-pitr-recovery">). Create a recovery
! status file <filename>recovery.ready</> in the standby's cluster data
! directory. Turn on <varname>standby_mode</> and set
<varname>restore_command</> to a simple command to copy files from
the WAL archive. If you plan to have multiple standby servers for high
availability purposes, set <varname>recovery_target_timeline</> to
***************
*** 695,701 **** protocol to make nodes agree on a serializable transactional order.
<para>
If you're using a WAL archive, its size can be minimized using the <xref
! linkend="archive-cleanup-command"> parameter to remove files that are no
longer required by the standby server.
The <application>pg_archivecleanup</> utility is designed specifically to
be used with <varname>archive_cleanup_command</> in typical single-standby
--- 695,701 ----
<para>
If you're using a WAL archive, its size can be minimized using the <xref
! linkend="guc-archive-cleanup-command"> parameter to remove files that are no
longer required by the standby server.
The <application>pg_archivecleanup</> utility is designed specifically to
be used with <varname>archive_cleanup_command</> in typical single-standby
***************
*** 706,712 **** protocol to make nodes agree on a serializable transactional order.
</para>
<para>
! A simple example of a <filename>recovery.conf</> is:
<programlisting>
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
--- 706,712 ----
</para>
<para>
! A simple example of standby settings is:
<programlisting>
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
***************
*** 762,769 **** archive_cleanup_command = 'pg_archivecleanup /path/to/archive %r'
To use streaming replication, set up a file-based log-shipping standby
server as described in <xref linkend="warm-standby">. The step that
turns a file-based log-shipping standby into streaming replication
! standby is setting <varname>primary_conninfo</> setting in the
! <filename>recovery.conf</> file to point to the primary server. Set
<xref linkend="guc-listen-addresses"> and authentication options
(see <filename>pg_hba.conf</>) on the primary so that the standby server
can connect to the <literal>replication</> pseudo-database on the primary
--- 762,769 ----
To use streaming replication, set up a file-based log-shipping standby
server as described in <xref linkend="warm-standby">. The step that
turns a file-based log-shipping standby into streaming replication
! standby is setting <varname>primary_conninfo</> to
! point to the primary server. Set
<xref linkend="guc-listen-addresses"> and authentication options
(see <filename>pg_hba.conf</>) on the primary so that the standby server
can connect to the <literal>replication</> pseudo-database on the primary
***************
*** 832,846 **** host replication foo 192.168.1.100/32 md5
</para>
<para>
The host name and port number of the primary, connection user name,
! and password are specified in the <filename>recovery.conf</> file.
The password can also be set in the <filename>~/.pgpass</> file on the
standby (specify <literal>replication</> in the <replaceable>database</>
field).
For example, if the primary is running on host IP <literal>192.168.1.50</>,
port <literal>5432</literal>, the account name for replication is
<literal>foo</>, and the password is <literal>foopass</>, the administrator
! can add the following line to the <filename>recovery.conf</> file on the
! standby:
<programlisting>
# The standby connects to the primary that is running on host 192.168.1.50
--- 832,845 ----
</para>
<para>
The host name and port number of the primary, connection user name,
! and password are specified in <varname>primary_conninfo</>.
The password can also be set in the <filename>~/.pgpass</> file on the
standby (specify <literal>replication</> in the <replaceable>database</>
field).
For example, if the primary is running on host IP <literal>192.168.1.50</>,
port <literal>5432</literal>, the account name for replication is
<literal>foo</>, and the password is <literal>foopass</>, the administrator
! can set <varname>primary_conninfo</> on the standby like this:
<programlisting>
# The standby connects to the primary that is running on host 192.168.1.50
***************
*** 1204,1211 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<para>
To trigger failover of a log-shipping standby server,
run <command>pg_ctl promote</> or create a trigger
! file with the file name and path specified by the <varname>trigger_file</>
! setting in <filename>recovery.conf</>. If you're planning to use
<command>pg_ctl promote</> to fail over, <varname>trigger_file</> is
not required. If you're setting up the reporting servers that are
only used to offload read-only queries from the primary, not for high
--- 1203,1210 ----
<para>
To trigger failover of a log-shipping standby server,
run <command>pg_ctl promote</> or create a trigger
! file with the file name and path specified by the <varname>trigger_file</>.
! If you're planning to use
<command>pg_ctl promote</> to fail over, <varname>trigger_file</> is
not required. If you're setting up the reporting servers that are
only used to offload read-only queries from the primary, not for high
***************
*** 1250,1257 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
The magic that makes the two loosely coupled servers work together is
simply a <varname>restore_command</> used on the standby that,
when asked for the next WAL file, waits for it to become available from
! the primary. The <varname>restore_command</> is specified in the
! <filename>recovery.conf</> file on the standby server. Normal recovery
processing would request a file from the WAL archive, reporting failure
if the file was unavailable. For standby processing it is normal for
the next WAL file to be unavailable, so the standby must wait for
--- 1249,1255 ----
The magic that makes the two loosely coupled servers work together is
simply a <varname>restore_command</> used on the standby that,
when asked for the next WAL file, waits for it to become available from
! the primary. Normal recovery
processing would request a file from the WAL archive, reporting failure
if the file was unavailable. For standby processing it is normal for
the next WAL file to be unavailable, so the standby must wait for
***************
*** 1338,1345 **** if (!triggered)
</listitem>
<listitem>
<para>
Begin recovery on the standby server from the local WAL
! archive, using a <filename>recovery.conf</> that specifies a
<varname>restore_command</> that waits as described
previously (see <xref linkend="backup-pitr-recovery">).
</para>
--- 1336,1349 ----
</listitem>
<listitem>
<para>
+ Create a recovery status file <filename>recovery.ready</>
+ in the standby's cluster data directory.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
Begin recovery on the standby server from the local WAL
! archive, specifying a
<varname>restore_command</> that waits as described
previously (see <xref linkend="backup-pitr-recovery">).
</para>
***************
*** 1830,1838 **** if (!triggered)
<title>Administrator's Overview</title>
<para>
! If <varname>hot_standby</> is turned <literal>on</> in
! <filename>postgresql.conf</> and there is a <filename>recovery.conf</>
! file present, the server will run in Hot Standby mode.
However, it may take some time for Hot Standby connections to be allowed,
because the server will not accept connections until it has completed
sufficient recovery to provide a consistent state against which queries
--- 1834,1842 ----
<title>Administrator's Overview</title>
<para>
! If <varname>hot_standby</> is turned <literal>on</>
! and there is a recovery status file
! <filename>recovery.ready</> present, the server will run in Hot Standby mode.
However, it may take some time for Hot Standby connections to be allowed,
because the server will not accept connections until it has completed
sufficient recovery to provide a consistent state against which queries
*** a/doc/src/sgml/pgarchivecleanup.sgml
--- b/doc/src/sgml/pgarchivecleanup.sgml
***************
*** 37,44 ****
<para>
To configure a standby
! server to use <application>pg_archivecleanup</>, put this into its
! <filename>recovery.conf</filename> configuration file:
<programlisting>
archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
</programlisting>
--- 37,44 ----
<para>
To configure a standby
! server to use <application>pg_archivecleanup</>, specify
! <xref linkend="guc-archive-cleanup-command"> like this:
<programlisting>
archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
</programlisting>
***************
*** 46,52 **** archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
files should be removed.
</para>
<para>
! When used within <xref linkend="archive-cleanup-command">, all WAL files
logically preceding the value of the <literal>%r</> argument will be removed
from <replaceable>archivelocation</>. This minimizes the number of files
that need to be retained, while preserving crash-restart capability. Use of
--- 46,52 ----
files should be removed.
</para>
<para>
! When used within <varname>archive_cleanup_command</>, all WAL files
logically preceding the value of the <literal>%r</> argument will be removed
from <replaceable>archivelocation</>. This minimizes the number of files
that need to be retained, while preserving crash-restart capability. Use of
*** a/doc/src/sgml/pgstandby.sgml
--- b/doc/src/sgml/pgstandby.sgml
***************
*** 48,55 ****
<para>
To configure a standby
! server to use <application>pg_standby</>, put this into its
! <filename>recovery.conf</filename> configuration file:
<programlisting>
restore_command = 'pg_standby <replaceable>archiveDir</> %f %p %r'
</programlisting>
--- 48,55 ----
<para>
To configure a standby
! server to use <application>pg_standby</>, specify
! <xref linkend="guc-restore-command"> like this:
<programlisting>
restore_command = 'pg_standby <replaceable>archiveDir</> %f %p %r'
</programlisting>
*** a/doc/src/sgml/postgres.sgml
--- b/doc/src/sgml/postgres.sgml
***************
*** 155,161 ****
&maintenance;
&backup;
&high-availability;
- &recovery-config;
&monitoring;
&diskusage;
&wal;
--- 155,160 ----
*** a/doc/src/sgml/recovery-config.sgml
--- /dev/null
***************
*** 1,363 ****
- <!-- doc/src/sgml/recovery-config.sgml -->
-
- <chapter id="recovery-config">
- <title>Recovery Configuration</title>
-
- <indexterm>
- <primary>configuration</primary>
- <secondary>of recovery</secondary>
- <tertiary>of a standby server</tertiary>
- </indexterm>
-
- <para>
- This chapter describes the settings available in the
- <filename>recovery.conf</><indexterm><primary>recovery.conf</></>
- file. They apply only for the duration of the
- recovery. They must be reset for any subsequent recovery you wish to
- perform. They cannot be changed once recovery has begun.
- </para>
-
- <para>
- Settings in <filename>recovery.conf</> are specified in the format
- <literal>name = 'value'</>. One parameter is specified per line.
- Hash marks (<literal>#</literal>) designate the rest of the
- line as a comment. To embed a single quote in a parameter
- value, write two quotes (<literal>''</>).
- </para>
-
- <para>
- A sample file, <filename>share/recovery.conf.sample</>,
- is provided in the installation's <filename>share/</> directory.
- </para>
-
- <sect1 id="archive-recovery-settings">
-
- <title>Archive Recovery Settings</title>
- <variablelist>
-
- <varlistentry id="restore-command" xreflabel="restore_command">
- <term><varname>restore_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>restore_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- The shell command to execute to retrieve an archived segment of
- the WAL file series. This parameter is required for archive recovery,
- but optional for streaming replication.
- Any <literal>%f</> in the string is
- replaced by the name of the file to retrieve from the archive,
- and any <literal>%p</> is replaced by the copy destination path name
- on the server.
- (The path name is relative to the current working directory,
- i.e., the cluster's data directory.)
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point. That is the earliest file that must be kept
- to allow a restore to be restartable, so this information can be used
- to truncate the archive to just the minimum required to support
- restarting from the current restore. <literal>%r</> is typically only
- used by warm-standby configurations
- (see <xref linkend="warm-standby">).
- Write <literal>%%</> to embed an actual <literal>%</> character.
- </para>
-
- <para>
- It is important for the command to return a zero exit status
- only if it succeeds. The command <emphasis>will</> be asked for file
- names that are not present in the archive; it must return nonzero
- when so asked. Examples:
- <programlisting>
- restore_command = 'cp /mnt/server/archivedir/%f "%p"'
- restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
- </programlisting>
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="archive-cleanup-command" xreflabel="archive_cleanup_command">
- <term><varname>archive_cleanup_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>archive_cleanup_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This optional parameter specifies a shell command that will be executed
- at every restartpoint. The purpose of
- <varname>archive_cleanup_command</> is to provide a mechanism for
- cleaning up old archived WAL files that are no longer needed by the
- standby server.
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point.
- That is the earliest file that must be <emphasis>kept</> to allow a
- restore to be restartable, and so all files earlier than <literal>%r</>
- may be safely removed.
- This information can be used to truncate the archive to just the
- minimum required to support restart from the current restore.
- The <xref linkend="pgarchivecleanup"> module
- is often used in <varname>archive_cleanup_command</> for
- single-standby configurations, for example:
- <programlisting>archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'</programlisting>
- Note however that if multiple standby servers are restoring from the
- same archive directory, you will need to ensure that you do not delete
- WAL files until they are no longer needed by any of the servers.
- <varname>archive_cleanup_command</> would typically be used in a
- warm-standby configuration (see <xref linkend="warm-standby">).
- Write <literal>%%</> to embed an actual <literal>%</> character in the
- command.
- </para>
- <para>
- If the command returns a non-zero exit status then a WARNING log
- message will be written.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-end-command" xreflabel="recovery_end_command">
- <term><varname>recovery_end_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>recovery_end_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies a shell command that will be executed once only
- at the end of recovery. This parameter is optional. The purpose of the
- <varname>recovery_end_command</> is to provide a mechanism for cleanup
- following replication or recovery.
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point, like in <xref linkend="archive-cleanup-command">.
- </para>
- <para>
- If the command returns a non-zero exit status then a WARNING log
- message will be written and the database will proceed to start up
- anyway. An exception is that if the command was terminated by a
- signal, the database will not proceed with startup.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
-
- </sect1>
-
- <sect1 id="recovery-target-settings">
-
- <title>Recovery Target Settings</title>
- <variablelist>
-
- <varlistentry id="recovery-target-name" xreflabel="recovery_target_name">
- <term><varname>recovery_target_name</varname>
- (<type>string</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_name</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the named restore point, created with
- <function>pg_create_restore_point()</> to which recovery will proceed.
- At most one of <varname>recovery_target_name</>,
- <xref linkend="recovery-target-time"> or
- <xref linkend="recovery-target-xid"> can be specified. The default is to
- recover to the end of the WAL log.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-time" xreflabel="recovery_target_time">
- <term><varname>recovery_target_time</varname>
- (<type>timestamp</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_time</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the time stamp up to which recovery
- will proceed.
- At most one of <varname>recovery_target_time</>,
- <xref linkend="recovery-target-name"> or
- <xref linkend="recovery-target-xid"> can be specified.
- The default is to recover to the end of the WAL log.
- The precise stopping point is also influenced by
- <xref linkend="recovery-target-inclusive">.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-xid" xreflabel="recovery_target_xid">
- <term><varname>recovery_target_xid</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>recovery_target_xid</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the transaction ID up to which recovery
- will proceed. Keep in mind
- that while transaction IDs are assigned sequentially at transaction
- start, transactions can complete in a different numeric order.
- The transactions that will be recovered are those that committed
- before (and optionally including) the specified one.
- At most one of <varname>recovery_target_xid</>,
- <xref linkend="recovery-target-name"> or
- <xref linkend="recovery-target-time"> can be specified.
- The default is to recover to the end of the WAL log.
- The precise stopping point is also influenced by
- <xref linkend="recovery-target-inclusive">.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-inclusive"
- xreflabel="recovery_target_inclusive">
- <term><varname>recovery_target_inclusive</varname>
- (<type>boolean</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_inclusive</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether we stop just after the specified recovery target
- (<literal>true</literal>), or just before the recovery target
- (<literal>false</literal>).
- Applies to both <xref linkend="recovery-target-time">
- and <xref linkend="recovery-target-xid">, whichever one is
- specified for this recovery. This indicates whether transactions
- having exactly the target commit time or ID, respectively, will
- be included in the recovery. Default is <literal>true</>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-timeline"
- xreflabel="recovery_target_timeline">
- <term><varname>recovery_target_timeline</varname>
- (<type>string</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_timeline</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies recovering into a particular timeline. The default is
- to recover along the same timeline that was current when the
- base backup was taken. Setting this to <literal>latest</> recovers
- to the latest timeline found in the archive, which is useful in
- a standby server. Other than that you only need to set this parameter
- in complex re-recovery situations, where you need to return to
- a state that itself was reached after a point-in-time recovery.
- See <xref linkend="backup-timelines"> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="pause-at-recovery-target"
- xreflabel="pause_at_recovery_target">
- <term><varname>pause_at_recovery_target</varname>
- (<type>boolean</type>)
- </term>
- <indexterm>
- <primary><varname>pause_at_recovery_target</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether recovery should pause when the recovery target
- is reached. The default is true.
- This is intended to allow queries to be executed against the
- database to check if this recovery target is the most desirable
- point for recovery. The paused state can be resumed by using
- <function>pg_xlog_replay_resume()</> (See
- <xref linkend="functions-recovery-control-table">), which then
- causes recovery to end. If this recovery target is not the
- desired stopping point, then shutdown the server, change the
- recovery target settings to a later target and restart to
- continue recovery.
- </para>
- <para>
- This setting has no effect if <xref linkend="guc-hot-standby"> is not
- enabled, or if no recovery target is set.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect1>
-
- <sect1 id="standby-settings">
-
- <title>Standby Server Settings</title>
- <variablelist>
-
- <varlistentry id="standby-mode" xreflabel="standby_mode">
- <term><varname>standby_mode</varname> (<type>boolean</type>)</term>
- <indexterm>
- <primary><varname>standby_mode</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether to start the <productname>PostgreSQL</> server as
- a standby. If this parameter is <literal>on</>, the server will
- not stop recovery when the end of archived WAL is reached, but
- will keep trying to continue recovery by fetching new WAL segments
- using <varname>restore_command</>
- and/or by connecting to the primary server as specified by the
- <varname>primary_conninfo</> setting.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry id="primary-conninfo" xreflabel="primary_conninfo">
- <term><varname>primary_conninfo</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>primary_conninfo</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies a connection string to be used for the standby server
- to connect with the primary. This string is in the format
- accepted by the libpq <function>PQconnectdb</function> function,
- described in <xref linkend="libpq-connect">. If any option is
- unspecified in this string, then the corresponding environment
- variable (see <xref linkend="libpq-envars">) is checked. If the
- environment variable is not set either, then
- defaults are used.
- </para>
- <para>
- The connection string should specify the host name (or address)
- of the primary server, as well as the port number if it is not
- the same as the standby server's default.
- Also specify a user name corresponding to a role that has the
- <literal>REPLICATION</> and <literal>LOGIN</> privileges on the
- primary (see
- <xref linkend="streaming-replication-authentication">).
- A password needs to be provided too, if the primary demands password
- authentication. It can be provided in the
- <varname>primary_conninfo</varname> string, or in a separate
- <filename>~/.pgpass</> file on the standby server (use
- <literal>replication</> as the database name).
- Do not specify a database name in the
- <varname>primary_conninfo</varname> string.
- </para>
- <para>
- This setting has no effect if <varname>standby_mode</> is <literal>off</>.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry id="trigger-file" xreflabel="trigger_file">
- <term><varname>trigger_file</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>trigger_file</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies a trigger file whose presence ends recovery in the
- standby. Even if this value is not set, you can still promote
- the standby using <command>pg_ctl promote</>.
- This setting has no effect if <varname>standby_mode</> is <literal>off</>.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect1>
-
- </chapter>
--- 0 ----
*** a/doc/src/sgml/release-9.1.sgml
--- b/doc/src/sgml/release-9.1.sgml
***************
*** 1002,1008 ****
<listitem>
<para>
Add <filename>recovery.conf</> setting <link
! linkend="pause-at-recovery-target"><varname>pause_at_recovery_target</></link>
to pause recovery at target (Simon Riggs)
</para>
--- 1002,1008 ----
<listitem>
<para>
Add <filename>recovery.conf</> setting <link
! linkend="guc-pause-at-recovery-target"><varname>pause_at_recovery_target</></link>
to pause recovery at target (Simon Riggs)
</para>
***************
*** 1022,1028 ****
<para>
These named restore points can be specified as recovery
targets using the new <filename>recovery.conf</> setting
! <link linkend="recovery-target-name"><varname>recovery_target_name</></link>.
</para>
</listitem>
--- 1022,1028 ----
<para>
These named restore points can be specified as recovery
targets using the new <filename>recovery.conf</> setting
! <link linkend="guc-recovery-target-name"><varname>recovery_target_name</></link>.
</para>
</listitem>
***************
*** 1054,1061 ****
<listitem>
<para>
! Allow <link
! linkend="recovery-config"><filename>recovery.conf</></link>
to use the same quoting behavior as <filename>postgresql.conf</>
(Dimitri Fontaine)
</para>
--- 1054,1060 ----
<listitem>
<para>
! Allow <filename>recovery.conf</>
to use the same quoting behavior as <filename>postgresql.conf</>
(Dimitri Fontaine)
</para>
*** a/doc/src/sgml/release.sgml
--- b/doc/src/sgml/release.sgml
***************
*** 5,12 **** Typical markup:
&<> use & escapes
PostgreSQL <productname>
! postgresql.conf, pg_hba.conf,
! recovery.conf <filename>
[A-Z][A-Z_ ]+[A-Z_] <command>, <literal>, <envar>
[A-Za-z_][A-Za-z0-9_]+() <function>
-[-A-Za-z_]+ <option>
--- 5,12 ----
&<> use & escapes
PostgreSQL <productname>
! postgresql.conf, pg_hba.conf
! recovery.ready <filename>
[A-Z][A-Z_ ]+[A-Z_] <command>, <literal>, <envar>
[A-Za-z_][A-Za-z0-9_]+() <function>
-[-A-Za-z_]+ <option>
*** a/src/backend/access/transam/recovery.conf.sample
--- b/src/backend/access/transam/recovery.conf.sample
***************
*** 2,24 ****
# PostgreSQL recovery config file
# -------------------------------
#
! # Edit this file to provide the parameters that PostgreSQL needs to
! # perform an archive recovery of a database, or to act as a replication
! # standby.
#
! # If "recovery.conf" is present in the PostgreSQL data directory, it is
! # read on postmaster startup. After successful recovery, it is renamed
! # to "recovery.done" to ensure that we do not accidentally re-enter
! # archive recovery or standby mode.
#
! # This file consists of lines of the form:
! #
! # name = value
! #
! # Comments are introduced with '#'.
! #
! # The complete list of option names and allowed values can be found
! # in the PostgreSQL documentation.
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
--- 2,15 ----
# PostgreSQL recovery config file
# -------------------------------
#
! # PostgreSQL 9.2 or later, recovery.conf is no longer used. Instead,
! # specify all recovery parameters in postgresql.conf and create
! # recovery.ready to enter archive recovery or standby mode.
#
! # If you must use recovery.conf, specify "include directives" in
! # postgresql.conf like this:
#
! # include 'recovery.conf'
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 63,69 ****
/* File path names (all relative to $PGDATA) */
! #define RECOVERY_COMMAND_FILE "recovery.conf"
#define RECOVERY_COMMAND_DONE "recovery.done"
#define PROMOTE_SIGNAL_FILE "promote"
--- 63,69 ----
/* File path names (all relative to $PGDATA) */
! #define RECOVERY_COMMAND_READY "recovery.ready"
#define RECOVERY_COMMAND_DONE "recovery.done"
#define PROMOTE_SIGNAL_FILE "promote"
***************
*** 80,85 **** bool fullPageWrites = true;
--- 80,99 ----
bool log_checkpoints = false;
int sync_method = DEFAULT_SYNC_METHOD;
int wal_level = WAL_LEVEL_MINIMAL;
+ char *restore_command = NULL;
+ char *archive_cleanup_command = NULL;
+ char *recovery_end_command = NULL;
+ bool standby_mode = false;
+ char *primary_conninfo = NULL;
+ char *trigger_file = NULL;
+ RecoveryTargetType recovery_target = RECOVERY_TARGET_UNSET;
+ TransactionId recovery_target_xid = InvalidTransactionId;
+ TimestampTz recovery_target_time = 0;
+ char *recovery_target_name = NULL;
+ bool recovery_target_inclusive = true;
+ bool pause_at_recovery_target = true;
+ char *recovery_target_timeline_string = NULL;
+ TimeLineID recovery_target_timeline = 0;
#ifdef WAL_DEBUG
bool XLOG_DEBUG = false;
***************
*** 186,208 **** static bool InArchiveRecovery = false;
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* options taken from recovery.conf for archive recovery */
! static char *recoveryRestoreCommand = NULL;
! static char *recoveryEndCommand = NULL;
! static char *archiveCleanupCommand = NULL;
! static RecoveryTargetType recoveryTarget = RECOVERY_TARGET_UNSET;
! static bool recoveryTargetInclusive = true;
! static bool recoveryPauseAtTarget = true;
! static TransactionId recoveryTargetXid;
! static TimestampTz recoveryTargetTime;
! static char *recoveryTargetName;
!
! /* options taken from recovery.conf for XLOG streaming */
! static bool StandbyMode = false;
! static char *PrimaryConnInfo = NULL;
! static char *TriggerFile = NULL;
!
! /* if recoveryStopsHere returns true, it saves actual stop xid/time/name here */
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
--- 200,207 ----
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* if recoveryStopsHere returns true, it saves actual stop type/xid/time/name here */
! static RecoveryTargetType recoveryStopTarget;
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
***************
*** 408,419 **** typedef struct XLogCtlData
TimeLineID RecoveryTargetTLI;
/*
- * archiveCleanupCommand is read from recovery.conf but needs to be in
- * shared memory so that the bgwriter process can access it.
- */
- char archiveCleanupCommand[MAXPGPATH];
-
- /*
* SharedRecoveryInProgress indicates if we're still in crash or archive
* recovery. Protected by info_lck.
*/
--- 407,412 ----
***************
*** 602,608 **** static void XLogArchiveNotifySeg(uint32 log, uint32 seg);
static bool XLogArchiveCheckDone(const char *xlog);
static bool XLogArchiveIsBusy(const char *xlog);
static void XLogArchiveCleanup(const char *xlog);
! static void readRecoveryCommandFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
--- 595,601 ----
static bool XLogArchiveCheckDone(const char *xlog);
static bool XLogArchiveIsBusy(const char *xlog);
static void XLogArchiveCleanup(const char *xlog);
! static void CheckRecoveryReadyFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
***************
*** 612,617 **** static void SetRecoveryPause(bool recoveryPause);
--- 605,611 ----
static void SetLatestXTime(TimestampTz xtime);
static TimestampTz GetLatestXTime(void);
static void CheckRequiredParameterValues(void);
+ static void CheckRestoreCommandSet(void);
static void XLogReportParameters(void);
static void LocalSetXLogInsertAllowed(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
***************
*** 2932,2938 **** RestoreArchivedFile(char *path, const char *xlogfname,
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (recoveryRestoreCommand == NULL)
goto not_available;
/*
--- 2926,2932 ----
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (!restore_command[0])
goto not_available;
/*
***************
*** 2952,2958 **** RestoreArchivedFile(char *path, const char *xlogfname,
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the recoveryRestoreCommand may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
--- 2946,2952 ----
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the restore_command may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
***************
*** 3016,3022 **** RestoreArchivedFile(char *path, const char *xlogfname,
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = recoveryRestoreCommand; *sp; sp++)
{
if (*sp == '%')
{
--- 3010,3016 ----
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = restore_command; *sp; sp++)
{
if (*sp == '%')
{
***************
*** 3107,3113 **** RestoreArchivedFile(char *path, const char *xlogfname,
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (StandbyMode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
--- 3101,3107 ----
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (standby_mode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
***************
*** 3283,3289 **** ExecuteRecoveryCommand(char *command, char *commandName, bool failOnSignal)
ereport((signaled && failOnSignal) ? FATAL : WARNING,
/*------
! translator: First %s represents a recovery.conf parameter name like
"recovery_end_command", and the 2nd is the value of that parameter. */
(errmsg("%s \"%s\": return code %d", commandName,
command, rc)));
--- 3277,3283 ----
ereport((signaled && failOnSignal) ? FATAL : WARNING,
/*------
! translator: First %s represents a recovery parameter name like
"recovery_end_command", and the 2nd is the value of that parameter. */
(errmsg("%s \"%s\": return code %d", commandName,
command, rc)));
***************
*** 4064,4070 **** next_record_is_invalid:
}
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return NULL;
--- 4058,4064 ----
}
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return NULL;
***************
*** 4523,4529 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
--- 4517,4523 ----
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryStopTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4531,4537 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
--- 4525,4531 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryStopTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4539,4545 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
--- 4533,4539 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryStopTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
***************
*** 5268,5498 **** str_time(pg_time_t tnow)
}
/*
! * See if there is a recovery command file (recovery.conf), and if so
! * read in parameters for archive recovery and XLOG streaming.
! *
! * The file is parsed using the main configuration parser.
*/
static void
! readRecoveryCommandFile(void)
{
! FILE *fd;
! TimeLineID rtli = 0;
! bool rtliGiven = false;
! ConfigVariable *item,
! *head = NULL,
! *tail = NULL;
!
! fd = AllocateFile(RECOVERY_COMMAND_FILE, "r");
! if (fd == NULL)
! {
! if (errno == ENOENT)
! return; /* not there, so no archive recovery */
! ereport(FATAL,
! (errcode_for_file_access(),
! errmsg("could not open recovery command file \"%s\": %m",
! RECOVERY_COMMAND_FILE)));
! }
!
! /*
! * Since we're asking ParseConfigFp() to error out at FATAL, there's no
! * need to check the return value.
! */
! ParseConfigFp(fd, RECOVERY_COMMAND_FILE, 0, FATAL, &head, &tail);
!
! for (item = head; item; item = item->next)
! {
! if (strcmp(item->name, "restore_command") == 0)
! {
! recoveryRestoreCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("restore_command = '%s'",
! recoveryRestoreCommand)));
! }
! else if (strcmp(item->name, "recovery_end_command") == 0)
! {
! recoveryEndCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("recovery_end_command = '%s'",
! recoveryEndCommand)));
! }
! else if (strcmp(item->name, "archive_cleanup_command") == 0)
! {
! archiveCleanupCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("archive_cleanup_command = '%s'",
! archiveCleanupCommand)));
! }
! else if (strcmp(item->name, "pause_at_recovery_target") == 0)
! {
! if (!parse_bool(item->value, &recoveryPauseAtTarget))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value", "pause_at_recovery_target")));
! ereport(DEBUG2,
! (errmsg_internal("pause_at_recovery_target = '%s'",
! item->value)));
! }
! else if (strcmp(item->name, "recovery_target_timeline") == 0)
! {
! rtliGiven = true;
! if (strcmp(item->value, "latest") == 0)
! rtli = 0;
! else
! {
! errno = 0;
! rtli = (TimeLineID) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_timeline is not a valid number: \"%s\"",
! item->value)));
! }
! if (rtli)
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = %u", rtli)));
! else
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = latest")));
! }
! else if (strcmp(item->name, "recovery_target_xid") == 0)
! {
! errno = 0;
! recoveryTargetXid = (TransactionId) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_xid is not a valid number: \"%s\"",
! item->value)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_xid = %u",
! recoveryTargetXid)));
! recoveryTarget = RECOVERY_TARGET_XID;
! }
! else if (strcmp(item->name, "recovery_target_time") == 0)
! {
! /*
! * if recovery_target_xid or recovery_target_name specified, then
! * this overrides recovery_target_time
! */
! if (recoveryTarget == RECOVERY_TARGET_XID ||
! recoveryTarget == RECOVERY_TARGET_NAME)
! continue;
! recoveryTarget = RECOVERY_TARGET_TIME;
!
! /*
! * Convert the time string given by the user to TimestampTz form.
! */
! recoveryTargetTime =
! DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
! CStringGetDatum(item->value),
! ObjectIdGetDatum(InvalidOid),
! Int32GetDatum(-1)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_time = '%s'",
! timestamptz_to_str(recoveryTargetTime))));
! }
! else if (strcmp(item->name, "recovery_target_name") == 0)
! {
! /*
! * if recovery_target_xid specified, then this overrides
! * recovery_target_name
! */
! if (recoveryTarget == RECOVERY_TARGET_XID)
! continue;
! recoveryTarget = RECOVERY_TARGET_NAME;
!
! recoveryTargetName = pstrdup(item->value);
! if (strlen(recoveryTargetName) >= MAXFNAMELEN)
! ereport(FATAL,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("recovery_target_name is too long (maximum %d characters)",
! MAXFNAMELEN - 1)));
!
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_name = '%s'",
! recoveryTargetName)));
! }
! else if (strcmp(item->name, "recovery_target_inclusive") == 0)
! {
! /*
! * does nothing if a recovery_target is not also set
! */
! if (!parse_bool(item->value, &recoveryTargetInclusive))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "recovery_target_inclusive")));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_inclusive = %s",
! item->value)));
! }
! else if (strcmp(item->name, "standby_mode") == 0)
! {
! if (!parse_bool(item->value, &StandbyMode))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "standby_mode")));
! ereport(DEBUG2,
! (errmsg_internal("standby_mode = '%s'", item->value)));
! }
! else if (strcmp(item->name, "primary_conninfo") == 0)
! {
! PrimaryConnInfo = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("primary_conninfo = '%s'",
! PrimaryConnInfo)));
! }
! else if (strcmp(item->name, "trigger_file") == 0)
! {
! TriggerFile = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("trigger_file = '%s'",
! TriggerFile)));
! }
! else
! ereport(FATAL,
! (errmsg("unrecognized recovery parameter \"%s\"",
! item->name)));
! }
! /*
! * Check for compulsory parameters
! */
! if (StandbyMode)
! {
! if (PrimaryConnInfo == NULL && recoveryRestoreCommand == NULL)
! ereport(WARNING,
! (errmsg("recovery command file \"%s\" specified neither primary_conninfo nor restore_command",
! RECOVERY_COMMAND_FILE),
! errhint("The database server will regularly poll the pg_xlog subdirectory to check for files placed there.")));
! }
! else
! {
! if (recoveryRestoreCommand == NULL)
! ereport(FATAL,
! (errmsg("recovery command file \"%s\" must specify restore_command when standby mode is not enabled",
! RECOVERY_COMMAND_FILE)));
! }
/* Enable fetching from archive recovery area */
InArchiveRecovery = true;
/*
* If user specified recovery_target_timeline, validate it or compute the
* "latest" value. We can't do this until after we've gotten the restore
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (rtliGiven)
{
! if (rtli)
{
/* Timeline 1 does not have a history file, all else should */
! if (rtli != 1 && !existsTimeLineHistory(rtli))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! rtli)));
! recoveryTargetTLI = rtli;
recoveryTargetIsLatest = false;
}
else
--- 5262,5305 ----
}
/*
! * Check to see if there is a recovery status file (recovery.ready), and if so
! * validate recovery parameters and determine recovery target timeline
*/
static void
! CheckRecoveryReadyFile(void)
{
! struct stat stat_buf;
! if (stat(RECOVERY_COMMAND_READY, &stat_buf) != 0)
! return; /* not there, so no archive recovery */
/* Enable fetching from archive recovery area */
InArchiveRecovery = true;
+ /* Check for compulsory parameters */
+ if (standby_mode && !restore_command[0] && !primary_conninfo[0])
+ ereport(WARNING,
+ (errmsg("neither primary_conninfo nor restore_command is specified"),
+ errhint("The database server will regularly poll the pg_xlog subdirectory to "
+ "check for files placed there until either of them is set in postgresql.conf.")));
+ CheckRestoreCommandSet();
+
/*
* If user specified recovery_target_timeline, validate it or compute the
* "latest" value. We can't do this until after we've gotten the restore
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (strcmp(recovery_target_timeline_string, "") != 0)
{
! if (recovery_target_timeline)
{
/* Timeline 1 does not have a history file, all else should */
! if (recovery_target_timeline != 1 && !existsTimeLineHistory(recovery_target_timeline))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! recovery_target_timeline)));
! recoveryTargetTLI = recovery_target_timeline;
recoveryTargetIsLatest = false;
}
else
***************
*** 5502,5510 **** readRecoveryCommandFile(void)
recoveryTargetIsLatest = true;
}
}
-
- FreeConfigVariables(head);
- FreeFile(fd);
}
/*
--- 5309,5314 ----
***************
*** 5580,5590 **** exitArchiveRecovery(TimeLineID endTLI, uint32 endLogId, uint32 endLogSeg)
* re-enter archive recovery mode in a subsequent crash.
*/
unlink(RECOVERY_COMMAND_DONE);
! if (rename(RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE) != 0)
ereport(FATAL,
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
! RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
--- 5384,5394 ----
* re-enter archive recovery mode in a subsequent crash.
*/
unlink(RECOVERY_COMMAND_DONE);
! if (rename(RECOVERY_COMMAND_READY, RECOVERY_COMMAND_DONE) != 0)
ereport(FATAL,
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
! RECOVERY_COMMAND_READY, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
***************
*** 5647,5653 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
/* Do we have a PITR target at all? */
! if (recoveryTarget == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
--- 5451,5457 ----
return false;
/* Do we have a PITR target at all? */
! if (recovery_target == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
***************
*** 5658,5664 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
}
! if (recoveryTarget == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
--- 5462,5468 ----
return false;
}
! if (recovery_target == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
***************
*** 5669,5688 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recoveryTargetXid);
if (stopsHere)
! *includeThis = recoveryTargetInclusive;
}
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recoveryTargetName) == 0);
/*
! * Ignore recoveryTargetInclusive because this is not a transaction
* record
*/
*includeThis = false;
--- 5473,5492 ----
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recovery_target_xid);
if (stopsHere)
! *includeThis = recovery_target_inclusive;
}
! else if (recovery_target == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recovery_target_name) == 0);
/*
! * Ignore recovery_target_inclusive because this is not a transaction
* record
*/
*includeThis = false;
***************
*** 5694,5709 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recoveryTargetInclusive)
! stopsHere = (recordXtime > recoveryTargetTime);
else
! stopsHere = (recordXtime >= recoveryTargetTime);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
--- 5498,5514 ----
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recovery_target_inclusive)
! stopsHere = (recordXtime > recovery_target_time);
else
! stopsHere = (recordXtime >= recovery_target_time);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
+ recoveryStopTarget = recovery_target;
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
***************
*** 6004,6009 **** CheckRequiredParameterValues(void)
--- 5809,5827 ----
}
/*
+ * Check to see if restore_command must be set for archive recovery
+ * when standby mode is not enabled
+ */
+ static void
+ CheckRestoreCommandSet(void)
+ {
+ if (InArchiveRecovery && !standby_mode && !restore_command[0])
+ ereport(FATAL,
+ (errmsg("restore_command must be specified for archive recovery "
+ "when standby mode is not enabled")));
+ }
+
+ /*
* This must be called ONCE during postmaster or standalone-backend startup
*/
void
***************
*** 6097,6106 **** StartupXLOG(void)
recoveryTargetTLI = ControlFile->checkPointCopy.ThisTimeLineID;
/*
! * Check for recovery control file, and if so set up state for offline
* recovery
*/
! readRecoveryCommandFile();
/* Now we can determine the list of expected TLIs */
expectedTLIs = readTimeLineHistory(recoveryTargetTLI);
--- 5915,5924 ----
recoveryTargetTLI = ControlFile->checkPointCopy.ThisTimeLineID;
/*
! * Check for recovery status file, and if so set up state for offline
* recovery
*/
! CheckRecoveryReadyFile();
/* Now we can determine the list of expected TLIs */
expectedTLIs = readTimeLineHistory(recoveryTargetTLI);
***************
*** 6118,6149 **** StartupXLOG(void)
ControlFile->checkPointCopy.ThisTimeLineID)));
/*
! * Save the selected recovery target timeline ID and
! * archive_cleanup_command in shared memory so that other processes can
! * see them
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
- strncpy(XLogCtl->archiveCleanupCommand,
- archiveCleanupCommand ? archiveCleanupCommand : "",
- sizeof(XLogCtl->archiveCleanupCommand));
if (InArchiveRecovery)
{
! if (StandbyMode)
ereport(LOG,
(errmsg("entering standby mode")));
- else if (recoveryTarget == RECOVERY_TARGET_XID)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to XID %u",
- recoveryTargetXid)));
- else if (recoveryTarget == RECOVERY_TARGET_TIME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to %s",
- timestamptz_to_str(recoveryTargetTime))));
- else if (recoveryTarget == RECOVERY_TARGET_NAME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to \"%s\"",
- recoveryTargetName)));
else
ereport(LOG,
(errmsg("starting archive recovery")));
--- 5936,5951 ----
ControlFile->checkPointCopy.ThisTimeLineID)));
/*
! * Save the selected recovery target timeline ID in shared memory
! * so that other processes can see it
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
if (InArchiveRecovery)
{
! if (standby_mode)
ereport(LOG,
(errmsg("entering standby mode")));
else
ereport(LOG,
(errmsg("starting archive recovery")));
***************
*** 6153,6159 **** StartupXLOG(void)
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (StandbyMode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
--- 5955,5961 ----
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (standby_mode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
***************
*** 6211,6217 **** StartupXLOG(void)
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (StandbyMode)
{
/*
* The last valid checkpoint record required for a streaming
--- 6013,6019 ----
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (standby_mode)
{
/*
* The last valid checkpoint record required for a streaming
***************
*** 6280,6286 **** StartupXLOG(void)
/*
* Check whether we need to force recovery from WAL. If it appears to
! * have been a clean shutdown and we did not have a recovery.conf file,
* then assume no recovery needed.
*/
if (XLByteLT(checkPoint.redo, RecPtr))
--- 6082,6088 ----
/*
* Check whether we need to force recovery from WAL. If it appears to
! * have been a clean shutdown and we did not have a recovery.ready file,
* then assume no recovery needed.
*/
if (XLByteLT(checkPoint.redo, RecPtr))
***************
*** 6294,6300 **** StartupXLOG(void)
InRecovery = true;
else if (InArchiveRecovery)
{
! /* force recovery due to presence of recovery.conf */
InRecovery = true;
}
--- 6096,6102 ----
InRecovery = true;
else if (InArchiveRecovery)
{
! /* force recovery due to presence of recovery.ready */
InRecovery = true;
}
***************
*** 6564,6570 **** StartupXLOG(void)
* Pause only if users can connect to send a resume
* message
*/
! if (recoveryPauseAtTarget && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
--- 6366,6372 ----
* Pause only if users can connect to send a resume
* message
*/
! if (pause_at_recovery_target && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
***************
*** 6663,6669 **** StartupXLOG(void)
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (StandbyMode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
--- 6465,6471 ----
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (standby_mode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
***************
*** 6671,6677 **** StartupXLOG(void)
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! StandbyMode = false;
/*
* Re-fetch the last valid or last applied record, so we can identify the
--- 6473,6479 ----
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! SetConfigOption("standby_mode", "false", PGC_POSTMASTER, PGC_S_OVERRIDE);
/*
* Re-fetch the last valid or last applied record, so we can identify the
***************
*** 6864,6871 **** StartupXLOG(void)
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recoveryEndCommand)
! ExecuteRecoveryCommand(recoveryEndCommand,
"recovery_end_command",
true);
}
--- 6666,6673 ----
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recovery_end_command[0])
! ExecuteRecoveryCommand(recovery_end_command,
"recovery_end_command",
true);
}
***************
*** 8188,8195 **** CreateRestartPoint(int flags)
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (XLogCtl->archiveCleanupCommand[0])
! ExecuteRecoveryCommand(XLogCtl->archiveCleanupCommand,
"archive_cleanup_command",
false);
--- 7990,7997 ----
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (archive_cleanup_command[0])
! ExecuteRecoveryCommand(archive_cleanup_command,
"archive_cleanup_command",
false);
***************
*** 10020,10025 **** HandleStartupProcInterrupts(void)
--- 9822,9830 ----
{
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /* Check for compulsory parameter */
+ CheckRestoreCommandSet();
}
/*
***************
*** 10144,10150 **** XLogPageRead(XLogRecPtr *RecPtr, int emode, bool fetching_ckpt,
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (StandbyMode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
--- 9949,9955 ----
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (standby_mode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
***************
*** 10166,10172 **** retry:
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (StandbyMode)
{
/*
* In standby mode, wait for the requested record to become
--- 9971,9977 ----
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (standby_mode)
{
/*
* In standby mode, wait for the requested record to become
***************
*** 10330,10340 **** retry:
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (PrimaryConnInfo)
{
RequestXLogStreaming(
! fetching_ckpt ? RedoStartLSN : *RecPtr,
! PrimaryConnInfo);
continue;
}
}
--- 10135,10144 ----
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (primary_conninfo[0])
{
RequestXLogStreaming(
! fetching_ckpt ? RedoStartLSN : *RecPtr);
continue;
}
}
***************
*** 10477,10483 **** next_record_is_invalid:
readSource = 0;
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return false;
--- 10281,10287 ----
readSource = 0;
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return false;
***************
*** 10549,10563 **** CheckForStandbyTrigger(void)
return true;
}
! if (TriggerFile == NULL)
return false;
! if (stat(TriggerFile, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", TriggerFile)));
ShutdownWalRcv();
! unlink(TriggerFile);
triggered = true;
return true;
}
--- 10353,10367 ----
return true;
}
! if (!trigger_file[0])
return false;
! if (stat(trigger_file, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", trigger_file)));
ShutdownWalRcv();
! unlink(trigger_file);
triggered = true;
return true;
}
*** a/src/backend/commands/extension.c
--- b/src/backend/commands/extension.c
***************
*** 9,15 ****
* dependent objects can be associated with it. An extension is created by
* populating the pg_extension catalog from a "control" file.
* The extension control file is parsed with the same parser we use for
! * postgresql.conf and recovery.conf. An extension also has an installation
* script file, containing SQL commands to create the extension's objects.
*
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
--- 9,15 ----
* dependent objects can be associated with it. An extension is created by
* populating the pg_extension catalog from a "control" file.
* The extension control file is parsed with the same parser we use for
! * postgresql.conf. An extension also has an installation
* script file, containing SQL commands to create the extension's objects.
*
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
*** a/src/backend/replication/walreceiver.c
--- b/src/backend/replication/walreceiver.c
***************
*** 168,174 **** DisableWalRcvImmediateExit(void)
void
WalReceiverMain(void)
{
- char conninfo[MAXCONNINFO];
XLogRecPtr startpoint;
/* use volatile pointer to prevent code rearrangement */
--- 168,173 ----
***************
*** 216,222 **** WalReceiverMain(void)
walrcv->walRcvState = WALRCV_RUNNING;
/* Fetch information required to start streaming */
- strlcpy(conninfo, (char *) walrcv->conninfo, MAXCONNINFO);
startpoint = walrcv->receiveStart;
SpinLockRelease(&walrcv->mutex);
--- 215,220 ----
***************
*** 272,278 **** WalReceiverMain(void)
/* Establish the connection to the primary for XLOG streaming */
EnableWalRcvImmediateExit();
! walrcv_connect(conninfo, startpoint);
DisableWalRcvImmediateExit();
/* Loop until end-of-streaming or error */
--- 270,276 ----
/* Establish the connection to the primary for XLOG streaming */
EnableWalRcvImmediateExit();
! walrcv_connect(primary_conninfo, startpoint);
DisableWalRcvImmediateExit();
/* Loop until end-of-streaming or error */
***************
*** 302,309 **** WalReceiverMain(void)
--- 300,320 ----
if (got_SIGHUP)
{
+ char *conninfo = pstrdup(primary_conninfo);
+
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /*
+ * If primary_conninfo has been changed while walreceiver is running,
+ * shut down walreceiver so that new walreceiver is started and
+ * it initiates replication with new primary_conninfo.
+ */
+ if (strcmp(conninfo, primary_conninfo) != 0)
+ ereport(FATAL,
+ (errcode(ERRCODE_ADMIN_SHUTDOWN),
+ errmsg("terminating walreceiver process because primary_conninfo was changed")));
+ pfree(conninfo);
}
/* Wait a while for data to arrive */
*** a/src/backend/replication/walreceiverfuncs.c
--- b/src/backend/replication/walreceiverfuncs.c
***************
*** 166,176 **** ShutdownWalRcv(void)
/*
* Request postmaster to start walreceiver.
*
! * recptr indicates the position where streaming should begin, and conninfo
! * is a libpq connection string to use.
*/
void
! RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo)
{
/* use volatile pointer to prevent code rearrangement */
volatile WalRcvData *walrcv = WalRcv;
--- 166,175 ----
/*
* Request postmaster to start walreceiver.
*
! * recptr indicates the position where streaming should begin.
*/
void
! RequestXLogStreaming(XLogRecPtr recptr)
{
/* use volatile pointer to prevent code rearrangement */
volatile WalRcvData *walrcv = WalRcv;
***************
*** 190,199 **** RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo)
/* It better be stopped before we try to restart it */
Assert(walrcv->walRcvState == WALRCV_STOPPED);
- if (conninfo != NULL)
- strlcpy((char *) walrcv->conninfo, conninfo, MAXCONNINFO);
- else
- walrcv->conninfo[0] = '\0';
walrcv->walRcvState = WALRCV_STARTING;
walrcv->startTime = now;
--- 189,194 ----
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 30,35 ****
--- 30,36 ----
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
+ #include "access/xlog_internal.h"
#include "catalog/namespace.h"
#include "commands/async.h"
#include "commands/prepare.h"
***************
*** 206,211 **** static bool check_application_name(char **newval, void **extra, GucSource source
--- 207,220 ----
static void assign_application_name(const char *newval, void *extra);
static const char *show_unix_socket_permissions(void);
static const char *show_log_file_mode(void);
+ static bool check_recovery_target_xid(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_xid(const char *newval, void *extra);
+ static bool check_recovery_target_name(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_name(const char *newval, void *extra);
+ static bool check_recovery_target_time(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_time(const char *newval, void *extra);
+ static bool check_recovery_target_timeline(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_timeline(const char *newval, void *extra);
static char *config_enum_get_options(struct config_enum * record,
const char *prefix, const char *suffix,
***************
*** 477,482 **** static int wal_block_size;
--- 486,493 ----
static int wal_segment_size;
static bool integer_datetimes;
static int effective_io_concurrency;
+ static char *recovery_target_xid_string;
+ static char *recovery_target_time_string;
/* should be static, but commands/variable.c needs to get at this */
char *role_string;
***************
*** 556,561 **** const char *const config_group_names[] =
--- 567,576 ----
gettext_noop("Write-Ahead Log / Checkpoints"),
/* WAL_ARCHIVING */
gettext_noop("Write-Ahead Log / Archiving"),
+ /* WAL_ARCHIVE_RECOVERY */
+ gettext_noop("Write-Ahead Log / Archive Recovery"),
+ /* WAL_RECOVERY_TARGET */
+ gettext_noop("Write-Ahead Log / Recovery Target"),
/* REPLICATION */
gettext_noop("Replication"),
/* REPLICATION_SENDING */
***************
*** 1366,1371 **** static struct config_bool ConfigureNamesBool[] =
--- 1381,1416 ----
},
{
+ {"recovery_target_inclusive", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether to include or exclude transaction with recovery target."),
+ NULL
+ },
+ &recovery_target_inclusive,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"pause_at_recovery_target", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether recovery should pause when the recovery target is reached."),
+ NULL
+ },
+ &pause_at_recovery_target,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"standby_mode", PGC_POSTMASTER, REPLICATION_STANDBY,
+ gettext_noop("Sets whether to start the server as a standby."),
+ NULL
+ },
+ &standby_mode,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"hot_standby", PGC_POSTMASTER, REPLICATION_STANDBY,
gettext_noop("Allows connections and queries during recovery."),
NULL
***************
*** 2522,2527 **** static struct config_string ConfigureNamesString[] =
--- 2567,2662 ----
},
{
+ {"restore_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will retrieve an archived WAL file."),
+ NULL
+ },
+ &restore_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"archive_cleanup_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed at every restartpoint."),
+ NULL
+ },
+ &archive_cleanup_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_end_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed once only at the end of recovery."),
+ NULL
+ },
+ &recovery_end_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_target_xid", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the transaction ID up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_xid_string,
+ "",
+ check_recovery_target_xid, assign_recovery_target_xid, NULL
+ },
+
+ {
+ {"recovery_target_name", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the named restore point."),
+ NULL
+ },
+ &recovery_target_name,
+ "",
+ check_recovery_target_name, assign_recovery_target_name, NULL
+ },
+
+ {
+ {"recovery_target_time", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the time stamp up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_time_string,
+ "",
+ check_recovery_target_time, assign_recovery_target_time, NULL
+ },
+
+ {
+ {"recovery_target_timeline", PGC_POSTMASTER, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets recoverying into a particular timeline."),
+ NULL
+ },
+ &recovery_target_timeline_string,
+ "",
+ check_recovery_target_timeline, assign_recovery_target_timeline, NULL
+ },
+
+ {
+ {"primary_conninfo", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the connection string to be used to connect with the primary."),
+ NULL
+ },
+ &primary_conninfo,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"trigger_file", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the trigger file whose presence ends recovery in the standby."),
+ NULL
+ },
+ &trigger_file,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
{"client_encoding", PGC_USERSET, CLIENT_CONN_LOCALE,
gettext_noop("Sets the client's character set encoding."),
NULL,
***************
*** 8691,8694 **** show_log_file_mode(void)
--- 8826,8978 ----
return buf;
}
+ static bool
+ check_recovery_target_xid(char **newval, void **extra, GucSource source)
+ {
+ TransactionId xid;
+ TransactionId *myextra;
+
+ errno = 0;
+ xid = (TransactionId) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_xid is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+
+ myextra = (TransactionId *) guc_malloc(ERROR, sizeof(TransactionId));
+ *myextra = xid;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_xid(const char *newval, void *extra)
+ {
+ recovery_target_xid = *((TransactionId *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_name(char **newval, void **extra, GucSource source)
+ {
+ if (strlen(*newval) >= MAXFNAMELEN)
+ {
+ GUC_check_errdetail("\"recovery_target_name\" is too long (maximum %d characters)",
+ MAXFNAMELEN - 1);
+ return false;
+ }
+ return true;
+ }
+
+ static void
+ assign_recovery_target_name(const char *newval, void *extra)
+ {
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (newval[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_time(char **newval, void **extra, GucSource source)
+ {
+ TimestampTz time;
+ TimestampTz *myextra;
+ MemoryContext oldcontext = CurrentMemoryContext;
+
+ PG_TRY();
+ {
+ time = (strcmp(*newval, "") == 0) ?
+ 0 :
+ DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
+ CStringGetDatum(*newval),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ }
+ PG_CATCH();
+ {
+ ErrorData *edata;
+
+ /* Save error info */
+ MemoryContextSwitchTo(oldcontext);
+ edata = CopyErrorData();
+ FlushErrorState();
+
+ /* Pass the error message */
+ GUC_check_errdetail("%s", edata->message);
+ FreeErrorData(edata);
+ return false;
+ }
+ PG_END_TRY();
+
+ myextra = (TimestampTz *) guc_malloc(ERROR, sizeof(TimestampTz));
+ *myextra = time;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_time(const char *newval, void *extra)
+ {
+ recovery_target_time = *((TimestampTz *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_timeline(char **newval, void **extra, GucSource source)
+ {
+ TimeLineID tli = 0;
+ TimeLineID *myextra;
+
+ if (strcmp(*newval, "latest") == 0)
+ tli = 0;
+ else
+ {
+ errno = 0;
+ tli = (TimeLineID) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_timeline is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+ }
+
+ myextra = (TimeLineID *) guc_malloc(ERROR, sizeof(TimeLineID));
+ *myextra = tli;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_timeline(const char *newval, void *extra)
+ {
+ recovery_target_timeline = *((TimeLineID *) extra);
+ }
+
#include "guc-file.c"
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 192,197 ****
--- 192,214 ----
#archive_timeout = 0 # force a logfile segment switch after this
# number of seconds; 0 disables
+ # - Archive Recovery -
+ #restore_command = '' # command to use to restore an archived logfile segment
+ # placeholders: %p = path of file to restore
+ # %f = file name only
+ # e.g. 'cp /mnt/server/archivedir/%f %p'
+ #archive_cleanup_command = '' # command to execute at every restartpoint
+ #recovery_end_command = '' # command to execute at completion of recovery
+
+ # - Recovery Target -
+ #recovery_target_xid = ''
+ #recovery_target_name = ''
+ #recovery_target_time = ''
+ #recovery_target_inclusive = on
+ #pause_at_recovery_target = on
+ #recovery_target_timeline = '' # timeline ID or 'latest'
+ # (change requires restart)
+
#------------------------------------------------------------------------------
# REPLICATION
***************
*** 219,224 ****
--- 236,245 ----
# These settings are ignored on a master server
+ #standby_mode = off # "on" starts the server as a standby
+ # (change requires restart)
+ #primary_conninfo = '' # connection string to connect to the master
+ #trigger_file = '' # trigger file to promote the standby
#hot_standby = off # "on" allows queries during recovery
# (change requires restart)
#max_standby_archive_delay = 30s # max delay before canceling queries
*** a/src/bin/pg_ctl/pg_ctl.c
--- b/src/bin/pg_ctl/pg_ctl.c
***************
*** 883,889 **** do_stop(void)
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.conf is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
--- 883,889 ----
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.ready is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
***************
*** 971,977 **** do_restart(void)
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.conf is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
--- 971,977 ----
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.ready is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
***************
*** 1082,1088 **** do_promote(void)
exit(1);
}
! /* If recovery.conf doesn't exist, the server is not in standby mode */
if (stat(recovery_file, &statbuf) != 0)
{
write_stderr(_("%s: cannot promote server; "
--- 1082,1088 ----
exit(1);
}
! /* If recovery.ready doesn't exist, the server is not in standby mode */
if (stat(recovery_file, &statbuf) != 0)
{
write_stderr(_("%s: cannot promote server; "
***************
*** 2163,2169 **** main(int argc, char **argv)
snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
! snprintf(recovery_file, MAXPGPATH, "%s/recovery.conf", pg_data);
snprintf(promote_file, MAXPGPATH, "%s/promote", pg_data);
}
--- 2163,2169 ----
snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
! snprintf(recovery_file, MAXPGPATH, "%s/recovery.ready", pg_data);
snprintf(promote_file, MAXPGPATH, "%s/promote", pg_data);
}
*** a/src/include/access/xlog.h
--- b/src/include/access/xlog.h
***************
*** 198,203 **** extern bool XLogArchiveMode;
--- 198,217 ----
extern char *XLogArchiveCommand;
extern bool EnableHotStandby;
extern bool log_checkpoints;
+ extern char *restore_command;
+ extern char *archive_cleanup_command;
+ extern char *recovery_end_command;
+ extern bool standby_mode;
+ extern char *primary_conninfo;
+ extern char *trigger_file;
+ extern RecoveryTargetType recovery_target;
+ extern TransactionId recovery_target_xid;
+ extern TimestampTz recovery_target_time;
+ extern char *recovery_target_name;
+ extern bool recovery_target_inclusive;
+ extern bool pause_at_recovery_target;
+ extern char *recovery_target_timeline_string;
+ extern TimeLineID recovery_target_timeline;
/* WAL levels */
typedef enum WalLevel
*** a/src/include/replication/walreceiver.h
--- b/src/include/replication/walreceiver.h
***************
*** 110,116 **** extern Size WalRcvShmemSize(void);
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
! extern void RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
#endif /* _WALRECEIVER_H */
--- 110,116 ----
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
! extern void RequestXLogStreaming(XLogRecPtr recptr);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
#endif /* _WALRECEIVER_H */
*** a/src/include/utils/guc_tables.h
--- b/src/include/utils/guc_tables.h
***************
*** 68,73 **** enum config_group
--- 68,75 ----
WAL_SETTINGS,
WAL_CHECKPOINTS,
WAL_ARCHIVING,
+ WAL_ARCHIVE_RECOVERY,
+ WAL_RECOVERY_TARGET,
REPLICATION,
REPLICATION_SENDING,
REPLICATION_MASTER,
Fujii Masao <masao.fujii@gmail.com> writes:
It seems to need a bit more time until we've reached a consensus about
the treatment of recovery.conf. How about committing the core patch
first, and addressing the recovery.conf issue as a different patch later?
The attached patch provides a core part of the feature, i.e., it moves
every recovery parameters from recovery.conf to postgresql.conf.
Even if you create recovery.conf, the server doesn't read it automatically.
The patch renames recovery.conf to recovery.ready, so if you want to
enter archive recovery or standby mode, you need to create
recovery.ready file in the cluster data directory. Since recovery.ready is
just a signal file, its contents have no effect.
This seems like it's already predetermining the outcome of the argument
about recovery.conf. Mind you, I'm not unhappy with this choice, but
it's hardly implementing only behavior that's not being debated.
If we're satisfied with not treating recovery-only parameters different
from run-of-the-mill GUCs, this is fine.
regards, tom lane
On Thu, Sep 15, 2011 at 11:37 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
This seems like it's already predetermining the outcome of the argument
about recovery.conf. Mind you, I'm not unhappy with this choice, but
it's hardly implementing only behavior that's not being debated.If we're satisfied with not treating recovery-only parameters different
from run-of-the-mill GUCs, this is fine.
Okay, we need to reach a consensus about the treatment of
recovery.conf.
We have three choices. Which do you like the best?
#1
Use empty recovery.ready file to enter arhicve recovery. recovery.conf
is not read automatically. All recovery parameters are expected to be
specified in postgresql.conf. If you must specify them in recovery.conf,
you need to add "include 'recovery.conf'" into postgresql.conf. But note
that that recovery.conf will not be renamed to recovery.done at the
end of recovery. This is what the patch I've posted does. This is
simplest approach, but might confuse people who use the tools which
depend on recovery.conf.
#2
Use empty recovery.ready file to enter archive recovery. recovery.conf
is read automatically. You can specify recovery parameters in
recovery.conf without adding "include 'recovery.conf'" into
postgresql.conf. But note that that recovery.conf will not be renamed
to recovery.done at the end of recovery. If we adopt this, we might
need to implement what Dimitri suggested. I guess this is not so difficult
thing.
http://archives.postgresql.org/pgsql-hackers/2011-09/msg00745.php
#3
Use recovery.conf file to enter archive recovery. recovery.conf is read
automatically, and will be renamed to recovery.done at the end of
recovery. This is least confusing approach for the existing users, but
I guess we need to add lots of code (e.g., as Peter suggested, we might
need to invent new context setting like PGC_RECOVERY) to address
the problem I pointed before.
http://archives.postgresql.org/pgsql-hackers/2011-09/msg00482.php
If we want to use recovery.conf as a temporary configuration file for
recovery (i.e., configuration file disappears after use), we must choose
#3. If we can live with that recovery.conf is treated as a permanent
configuration file but don't want to add "include 'recovery.conf'" into
postgresql.conf, #2 is best. If we don't use recovery.conf or mind
editing postgresql.conf to use recovery.conf, #1 is best. Which behavior
are you expecting?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Fujii Masao <masao.fujii@gmail.com> writes:
We have three choices. Which do you like the best?
I'm in favor of defining a separate, content-free trigger file to enable
archive recovery. Not sure about the name "recovery.ready", though ---
that makes it look like one of the WAL archive transfer trigger files,
which does not seem like a great analogy. The pg_standby documentation
suggests names like "foo.trigger" for failover triggers, which is a bit
better analogy because something external to the database creates the
file. What about "recovery.trigger"?
As far as the other issues go, I think there is actually a prerequisite
discussion to be had here, which is whether we are turning the recovery
parameters into plain old GUCs or not. If they are plain old GUCs, then
they will presumably still have their values when we are *not* doing
recovery. That leads to a couple of questions:
* will seeing these values present in pg_settings confuse anybody?
* can the values be changed when not in recovery, if so what happens,
and again will that confuse anybody?
* is there any security hazard from ordinary users being able to see
what settings had been used?
If these settings are to be plain old GUCs, then I think we should just
stick them in postgresql.conf and forget about recovery.conf (although
of course someone could "include recovery.conf" if he were bent on
storing them separately). On the other hand, if we think they are *not*
plain old GUCs, maybe they shouldn't be in postgresql.conf. But the
source file isn't the first thing to worry about in that case.
regards, tom lane
On fre, 2011-09-16 at 01:32 -0400, Tom Lane wrote:
As far as the other issues go, I think there is actually a
prerequisite
discussion to be had here, which is whether we are turning the
recovery
parameters into plain old GUCs or not. If they are plain old GUCs,
then
they will presumably still have their values when we are *not* doing
recovery. That leads to a couple of questions:
* will seeing these values present in pg_settings confuse anybody?
How so? We add or change the available parameters all the time.
* can the values be changed when not in recovery, if so what happens,
and again will that confuse anybody?
Should be similar to archive_command and archive_mode. You can still
see and change archive_command when archive_mode is off.
* is there any security hazard from ordinary users being able to see
what settings had been used?
Again, not much different from the archive_* settings. They are, after
all, almost the same in the opposite direction.
On fre, 2011-09-16 at 11:54 +0900, Fujii Masao wrote:
#1
Use empty recovery.ready file to enter arhicve recovery. recovery.conf
is not read automatically. All recovery parameters are expected to be
specified in postgresql.conf. If you must specify them in recovery.conf,
you need to add "include 'recovery.conf'" into postgresql.conf. But note
that that recovery.conf will not be renamed to recovery.done at the
end of recovery. This is what the patch I've posted does. This is
simplest approach, but might confuse people who use the tools which
depend on recovery.conf.
A small variant to this: When you are actually doing recovery from a
backup, having a recovery trigger and a recovery done file is obviously
quite helpful and necessary for safety. But when you're setting up a
replication slave, it adds extra complexity for the user. The
approximately goal ought to be to be able to do
pg_basebackup -h master -D there
postgres -D there --standby-mode=on --primary-conninfo=master
without the need to touch any obscure recovery trigger files.
So perhaps recovery.{trigger,ready} should not be needed if, say,
standby_mode=on. But then what impact should the presence of
recovery.done have?
On 15-09-2011 23:54, Fujii Masao wrote:
#1
Use empty recovery.ready file to enter arhicve recovery. recovery.conf
is not read automatically. All recovery parameters are expected to be
specified in postgresql.conf. If you must specify them in recovery.conf,
you need to add "include 'recovery.conf'" into postgresql.conf. But note
that that recovery.conf will not be renamed to recovery.done at the
end of recovery. This is what the patch I've posted does. This is
simplest approach, but might confuse people who use the tools which
depend on recovery.conf.
more or less +1. We don't need two config files.; just one: postgresql.conf.
Just turn all recovery.conf parameters to GUCs. As already said, the
recovery.conf settings are not different from archive settings, we just need a
way to trigger the recovery. And that trigger could be pulled by a GUC
(standby_mode) or a file (say recovery -> recovery.done). Also, recovery.done
could be filled with recovery information just for DBA record. standby_mode
does not create any file, it just trigger the recovery (as it will be used
mainly for replication purposes).
--
Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/
PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
I'm in favor of defining a separate, content-free trigger file to
enable
archive recovery. Not sure about the name "recovery.ready", though
---
that makes it look like one of the WAL archive transfer trigger
files,
which does not seem like a great analogy. The pg_standby
documentation
suggests names like "foo.trigger" for failover triggers, which is a
bit
better analogy because something external to the database creates the
file. What about "recovery.trigger"?
Do we want a trigger file to enable recovery, or one to *disable* recovery? Or both?
Also, I might point out that we're really confusing our users by talking about "recovery" all the time, if they're just using streaming replication. Just sayin'
* will seeing these values present in pg_settings confuse anybody?
No. pg_settings already has a couple dozen "developer" parameters which nobody not on this mailing list understands. Adding the recovery parameters to it wouldn't confuse anyone further, and would have the advantage of making the recovery parameters available by monitoring query on a hot standby.
For that matter, I'd suggest that we add a read-only setting called in_recovery.
* can the values be changed when not in recovery, if so what happens,
and again will that confuse anybody?
Yes, and no.
* is there any security hazard from ordinary users being able to see
what settings had been used?
primary_conninfo could be a problem, since it's possible to set a password there.
--Josh
On Fri, Sep 16, 2011 at 3:22 PM, Joshua Berkus <josh@agliodbs.com> wrote:
No. pg_settings already has a couple dozen "developer" parameters which nobody not on this mailing list understands. Adding the recovery parameters to it wouldn't confuse anyone further, and would have the advantage of making the recovery parameters available by monitoring query on a hot standby.
+1.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Sat, Sep 17, 2011 at 4:22 AM, Joshua Berkus <josh@agliodbs.com> wrote:
that makes it look like one of the WAL archive transfer trigger
files,
which does not seem like a great analogy. The pg_standby
documentation
suggests names like "foo.trigger" for failover triggers, which is a
bit
better analogy because something external to the database creates the
file. What about "recovery.trigger"?
I'm OK with that name.
Do we want a trigger file to enable recovery, or one to *disable* recovery? Or both?
ISTM that only supporting a trigger file to enable recovery is less confusing.
* will seeing these values present in pg_settings confuse anybody?
No. pg_settings already has a couple dozen "developer" parameters which nobody not on this mailing list understands. Adding the recovery parameters to it wouldn't confuse anyone further, and would have the advantage of making the recovery parameters available by monitoring query on a hot standby.
+1
* is there any security hazard from ordinary users being able to see
what settings had been used?primary_conninfo could be a problem, since it's possible to set a password there.
True. I agree that primary_conninfo should be restricted to superuser.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Fri, Sep 16, 2011 at 9:25 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On fre, 2011-09-16 at 11:54 +0900, Fujii Masao wrote:
#1
Use empty recovery.ready file to enter arhicve recovery. recovery.conf
is not read automatically. All recovery parameters are expected to be
specified in postgresql.conf. If you must specify them in recovery.conf,
you need to add "include 'recovery.conf'" into postgresql.conf. But note
that that recovery.conf will not be renamed to recovery.done at the
end of recovery. This is what the patch I've posted does. This is
simplest approach, but might confuse people who use the tools which
depend on recovery.conf.A small variant to this: When you are actually doing recovery from a
backup, having a recovery trigger and a recovery done file is obviously
quite helpful and necessary for safety. But when you're setting up a
replication slave, it adds extra complexity for the user. The
approximately goal ought to be to be able to dopg_basebackup -h master -D there
postgres -D there --standby-mode=on --primary-conninfo=masterwithout the need to touch any obscure recovery trigger files.
So perhaps recovery.{trigger,ready} should not be needed if, say,
standby_mode=on.
Or what about making pg_basebackup automatically create a
recovery trigger file?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Fri, Sep 16, 2011 at 3:54 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, Sep 15, 2011 at 11:37 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
This seems like it's already predetermining the outcome of the argument
about recovery.conf. Mind you, I'm not unhappy with this choice, but
it's hardly implementing only behavior that's not being debated.If we're satisfied with not treating recovery-only parameters different
from run-of-the-mill GUCs, this is fine.Okay, we need to reach a consensus about the treatment of
recovery.conf.We have three choices.
What we should focus on is these requirements
1. Don't break software that relies on the existing behaviour
2. Allow parameters to be reloaded at SIGHUP
3. Allow recovery parameters to be handled same way as other GUCs
4. We need to retain recovery.conf/.done style behaviour to mark end
of archive recovery
Making an automatic include makes (2) and (3) work OK, without
breaking (1). I haven't seen another solution that doesnt break (1).
Rename of .conf to .done allows us to maintain working code for
existing solutions (there are *many* out there...)
We should say that the automatic include only works during recovery,
so if someone puts recovery.conf back then we ignore it.
If we treat recovery,conf as being read *first* then any parameter
mentioned twice (i.e. mentioned again in postgresql.conf) will
override the setting in recovery.conf and we have no issues.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Fri, Sep 16, 2011 at 2:46 PM, Euler Taveira de Oliveira
<euler@timbira.com> wrote:
On 15-09-2011 23:54, Fujii Masao wrote:
#1
Use empty recovery.ready file to enter arhicve recovery. recovery.conf
is not read automatically. All recovery parameters are expected to be
specified in postgresql.conf. If you must specify them in recovery.conf,
you need to add "include 'recovery.conf'" into postgresql.conf. But note
that that recovery.conf will not be renamed to recovery.done at the
end of recovery. This is what the patch I've posted does. This is
simplest approach, but might confuse people who use the tools which
depend on recovery.conf.more or less +1. We don't need two config files.; just one: postgresql.conf.
Just turn all recovery.conf parameters to GUCs. As already said, the
recovery.conf settings are not different from archive settings, we just need
a way to trigger the recovery. And that trigger could be pulled by a GUC
(standby_mode) or a file (say recovery -> recovery.done). Also,
recovery.done could be filled with recovery information just for DBA record.
standby_mode does not create any file, it just trigger the recovery (as it
will be used mainly for replication purposes).
I sympathise with this view, to an extent.
If people want to put all parameters in one file, they can do so. So +1 to that.
Should they be forced to adopt that new capability by us deliberately
breaking their existing setups? No. So -1 to that.
If we do an automatic include of recovery.conf first, then follow by
reading postgresql,conf then we will preserve the old as well as
allowing the new.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Fri, Sep 16, 2011 at 1:13 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On fre, 2011-09-16 at 01:32 -0400, Tom Lane wrote:
As far as the other issues go, I think there is actually a
prerequisite
discussion to be had here, which is whether we are turning the
recovery
parameters into plain old GUCs or not. If they are plain old GUCs,
then
they will presumably still have their values when we are *not* doing
recovery. That leads to a couple of questions:
* will seeing these values present in pg_settings confuse anybody?How so? We add or change the available parameters all the time.
* can the values be changed when not in recovery, if so what happens,
and again will that confuse anybody?Should be similar to archive_command and archive_mode. You can still
see and change archive_command when archive_mode is off.
I do think special handling would be useful here, of some kind. We
could reset them as well, if we wished, but that is probably more
trouble than is worth.
Perhaps we need a new SCOPE attribute on pg_settings to show whether
the parameter applies in recovery, in normal or both.
* is there any security hazard from ordinary users being able to see
what settings had been used?Again, not much different from the archive_* settings. They are, after
all, almost the same in the opposite direction.
There is a potential security hole if people hardcode passwords into
primary_conninfo. As long as we document not to do that, we're OK.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon Riggs <simon@2ndQuadrant.com> writes:
I sympathise with this view, to an extent.
If people want to put all parameters in one file, they can do so. So +1 to that.
Should they be forced to adopt that new capability by us deliberately
breaking their existing setups? No. So -1 to that.
If we do an automatic include of recovery.conf first, then follow by
reading postgresql,conf then we will preserve the old as well as
allowing the new.
I don't buy this argument at all. I don't believe that recovery.conf is
part of anyone's automated processes at all, let alone to an extent that
they won't be able to cope with a change to rationalize the file layout.
And most especially I don't buy that someone who does want to keep using
it couldn't cope with adding an "include" to postgresql.conf manually.
If we're going to move these parameters into postgresql.conf, we should
just do that and remove all mention of recovery.conf. Anything else
will generate much more confusion than benefit.
regards, tom lane
I don't buy this argument at all. I don't believe that recovery.conf is
part of anyone's automated processes at all, let alone to an extent that
they won't be able to cope with a change to rationalize the file layout.
And most especially I don't buy that someone who does want to keep using
it couldn't cope with adding an "include" to postgresql.conf manually.
Speaking as someone who has a lot of client admin scripts written around
recovery.conf, we will be *thrilled* to update those scripts in order to
simplify the configuration file situation. Having a third conf file (or
fourth, or fifth, depending on which auth features you're using) already
adds unwarranted complexity to automation scripts, and each config file
we get rid of makes those scripts easier to maintain and troubleshoot.
For that matter, I'd love to consolidate pg_hba and pg_ident into a
single file. Any chance?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
All,
First, if we're going to change behavior, I assert that we should stop
calling stuff "recovery" and either call it "replica" or "standby". Our
use of the word "recovery" confuses users; it is historical in nature
and requires an understanding of PostgreSQL internals to know why it's
called that. It's also inconsistent with our use of the word "standby"
everywhere else.
Second, I haven't seen a response to this:
Do we want a trigger file to enable recovery, or one to *disable*
recovery? Or both?
I'll go further and say that we only want one trigger file by default,
one which either enables or disables recovery. I'll further suggest
that we:
a) have a standby.on file which puts the server in replica/recovery mode
if it's detected on startup, and
b) that we poll for the standby.on file going away as a trigger to stop
recovery and bring up the server in master mode, and
c) that pg_basebackup automatically create a standby.on file.
Perhaps we need a new SCOPE attribute on pg_settings to show whether
the parameter applies in recovery, in normal or both.
An overhaul of the category tree would also do that. I've been putting
off an overhaul/cleanup of categories for pg_settings, maybe it's about
time.
There is a potential security hole if people hardcode passwords into
primary_conninfo. As long as we document not to do that, we're OK.
Yeah, I'd almost be inclined to actively prohibit this, but that would
draw user complaints. We'll have to be satisfied with a doc plus a comment.
Speaking of which, .pgpass could be better documented as an option for
handling this sort of thing. Will take a stab, eventually.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Tue, Sep 20, 2011 at 1:01 PM, Josh Berkus <josh@agliodbs.com> wrote:
I'll go further and say that we only want one trigger file by default,
one which either enables or disables recovery. I'll further suggest
that we:a) have a standby.on file which puts the server in replica/recovery mode
if it's detected on startup, and
b) that we poll for the standby.on file going away as a trigger to stop
recovery and bring up the server in master mode, and
c) that pg_basebackup automatically create a standby.on file.
It seems a bit confusing to me to have a file that takes effect only
at startup when created but anytime when removed.
I think one of the insufficiently-lauded 9.1 features is Fujii Masao's
"pg_ctl promote". Now THAT is a good interface. Unlike
trigger_file, it doesn't require any advance preparation, or monkeying
with files on disk. You just tell it to promote, and it does. Sweet.
Now it turns out that it uses a file to make that happen behind the
scenes, but who cares? From the user's perspective It Just Works.
I like the idea of some kind of sentinel file that tells the server to
start up in recovery mode. But instead of having the user remove it
to cause a promotion, I think the server should remove it when it does
promote. That's more like what we've done in the past, and it ties in
very nicely with what pg_ctl promote already does.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 9/20/11 10:09 AM, Robert Haas wrote:
I like the idea of some kind of sentinel file that tells the server to
start up in recovery mode. But instead of having the user remove it
to cause a promotion, I think the server should remove it when it does
promote. That's more like what we've done in the past, and it ties in
very nicely with what pg_ctl promote already does.
Yes, I agree that that is a superior approach.
And then we could keep the trigger_file configuration parameter for
backwards compatibility, with intent to depreciate it after a couple
more Postgres versions.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
Josh Berkus <josh@agliodbs.com> writes:
First, if we're going to change behavior, I assert that we should stop
calling stuff "recovery" and either call it "replica" or "standby". Our
use of the word "recovery" confuses users; it is historical in nature
and requires an understanding of PostgreSQL internals to know why it's
called that. It's also inconsistent with our use of the word "standby"
everywhere else.
Are we all talking about the same thing? In my mind recovery.conf is
for configuring a point-in-time archive recovery run. It's got nothing
to do with either replication or standbys. Perhaps part of our problem
here is overloading that case with standby behavior.
Second, I haven't seen a response to this:
Do we want a trigger file to enable recovery, or one to *disable*
recovery? Or both?
As far as the PITR scenario is concerned, only the former can possibly
make any sense; the latter would be downright dangerous.
There is a potential security hole if people hardcode passwords into
primary_conninfo. As long as we document not to do that, we're OK.
Yeah, I'd almost be inclined to actively prohibit this, but that would
draw user complaints. We'll have to be satisfied with a doc plus a comment.
I think that marking the GUC as "only readable by superuser" is a
sufficient fix.
regards, tom lane
On Tue, Sep 20, 2011 at 1:30 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Josh Berkus <josh@agliodbs.com> writes:
First, if we're going to change behavior, I assert that we should stop
calling stuff "recovery" and either call it "replica" or "standby". Our
use of the word "recovery" confuses users; it is historical in nature
and requires an understanding of PostgreSQL internals to know why it's
called that. It's also inconsistent with our use of the word "standby"
everywhere else.Are we all talking about the same thing? In my mind recovery.conf is
for configuring a point-in-time archive recovery run. It's got nothing
to do with either replication or standbys.
Huh? How else can you create a standby? I do it by creating a
recovery.conf file that says:
standby_mode=on
primary_conninfo='whatever'
I wasn't aware that there is another method.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
On Tue, Sep 20, 2011 at 1:30 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Are we all talking about the same thing? �In my mind recovery.conf is
for configuring a point-in-time archive recovery run. �It's got nothing
to do with either replication or standbys.
Huh? How else can you create a standby? I do it by creating a
recovery.conf file that says:
standby_mode=on
The point I'm trying to make is that it seems like this discussion is
getting driven entirely by the standby case, without remembering that
recovery.conf was originally designed for, and is still used in,
a significantly different use-case. Maybe we had better take two
steps back and think about the implications for the archive-recovery
case.
regards, tom lane
The point I'm trying to make is that it seems like this discussion is
getting driven entirely by the standby case, without remembering that
recovery.conf was originally designed for, and is still used in,
a significantly different use-case. Maybe we had better take two
steps back and think about the implications for the archive-recovery
case.
I think we should take that into consideration, sure. But it should not
be in the driver's seat for things like nomenclature. Far more people
use replication than use PITR.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Tue, Sep 20, 2011 at 3:10 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Tue, Sep 20, 2011 at 1:30 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Are we all talking about the same thing? In my mind recovery.conf is
for configuring a point-in-time archive recovery run. It's got nothing
to do with either replication or standbys.Huh? How else can you create a standby? I do it by creating a
recovery.conf file that says:
standby_mode=onThe point I'm trying to make is that it seems like this discussion is
getting driven entirely by the standby case, without remembering that
recovery.conf was originally designed for, and is still used in,
a significantly different use-case. Maybe we had better take two
steps back and think about the implications for the archive-recovery
case.
I'm not sure there really are any implications that are worth getting
excited about. The problem is really one of naming; I'm reminded of
our recent discussions of the use of the term "relation". The problem
is that the word "recovery" encompasses a number of very different
scenarios. First, we have crash recovery. Second, we have archive
recovery (which, confusingly enough, no longer requires an archive).
Archive recovery can be further subdivided by where the xlog files are
coming from (archive, streaming replication, both, or neither) and how
long we plan to stay in recovery (forever, if acting as a standby;
until end of WAL, if promoting a standby or recovering a hot backup;
or until the recovery target is reached, if performing a point in time
recovery). I would guess that most of those are not what the typical
user thinks of as "recovery", but what else are we gonna call it?
Josh is arguing that we ought to use the term "replication", but it
seems to me that's just as misleading - maybe moreso, since "recovery"
is sufficiently a term of art to make you at least think about reading
the manual, whereas you know (or think you know) what replication is.
For now, I think we're best off not changing the terminology, and
confining the remit of this patch to (a) turning all of the existing
recovery.conf parameters into GUCs and (b) replacing recovery.conf
with a sentinel file a sentinel file (name TBB) to indicate that the
server is to start in recovery mode. The naming isn't great but the
more we change at once the less chance of reaching agreement. It
seems like we have pretty broad agreement on the basics here, so let's
start with that.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Robert,
Josh is arguing that we ought to use the term "replication", but it
Actually, no. I'm arguing that we should use the term "standby", since
that term is consistent with how we refer to replica servers throughout
the docs, and the term "recovery" is not.
seems to me that's just as misleading - maybe moreso, since "recovery"
is sufficiently a term of art to make you at least think about reading
the manual, whereas you know (or think you know) what replication is.
Nope. What it means is that users see stuff relating to "recovery" and
say "oh, that's not right, the replication stuff must be somewhere else".
I've taught a half-dozen classes on PostgreSQL binary replication now,
and the "recovery" nomenclature *always* confuses students.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Wed, Sep 21, 2011 at 12:55 PM, Josh Berkus <josh@agliodbs.com> wrote:
Josh is arguing that we ought to use the term "replication", but it
Actually, no. I'm arguing that we should use the term "standby", since
that term is consistent with how we refer to replica servers throughout
the docs, and the term "recovery" is not.seems to me that's just as misleading - maybe moreso, since "recovery"
is sufficiently a term of art to make you at least think about reading
the manual, whereas you know (or think you know) what replication is.Nope. What it means is that users see stuff relating to "recovery" and
say "oh, that's not right, the replication stuff must be somewhere else".I've taught a half-dozen classes on PostgreSQL binary replication now,
and the "recovery" nomenclature *always* confuses students.
Yeah, I get it. But I think standby would confuse them, too, just in
a different set of situations.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Yeah, I get it. But I think standby would confuse them, too, just in
a different set of situations.
Other than PITR, what situations?
PITR is used by a minority of our users. Binary replication, if not
already used by a majority, will be in the future (it's certainly the
majority of my professional clients). Further, PITR is usually
something which is either handled by vendor backup management software,
or by professional DBAs, whereas replication is used by developers with
little or no DBA support.
Why should we make terminology obscure for the majority usecase to make
it clear for the minority one? Especially since the majority use-case
has almost all the newbies?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Wed, Sep 21, 2011 at 1:03 PM, Josh Berkus <josh@agliodbs.com> wrote:
Yeah, I get it. But I think standby would confuse them, too, just in
a different set of situations.Other than PITR, what situations?
Hot backup?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 9/21/11 10:07 AM, Robert Haas wrote:
On Wed, Sep 21, 2011 at 1:03 PM, Josh Berkus <josh@agliodbs.com> wrote:
Yeah, I get it. But I think standby would confuse them, too, just in
a different set of situations.Other than PITR, what situations?
Hot backup?
Hot backup == PITR. You're just not bothering to accumulate WAL logs.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Wed, Sep 21, 2011 at 1:08 PM, Josh Berkus <josh@agliodbs.com> wrote:
On 9/21/11 10:07 AM, Robert Haas wrote:
On Wed, Sep 21, 2011 at 1:03 PM, Josh Berkus <josh@agliodbs.com> wrote:
Yeah, I get it. But I think standby would confuse them, too, just in
a different set of situations.Other than PITR, what situations?
Hot backup?
Hot backup == PITR. You're just not bothering to accumulate WAL logs.
Well, I don't think of it that way, but YMMV, of course.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Sep 21, 2011 at 1:13 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Sep 21, 2011 at 1:08 PM, Josh Berkus <josh@agliodbs.com> wrote:
On 9/21/11 10:07 AM, Robert Haas wrote:
On Wed, Sep 21, 2011 at 1:03 PM, Josh Berkus <josh@agliodbs.com> wrote:
Yeah, I get it. But I think standby would confuse them, too, just in
a different set of situations.Other than PITR, what situations?
Hot backup?
Hot backup == PITR. You're just not bothering to accumulate WAL logs.
Well, I don't think of it that way, but YMMV, of course.
I think that the major differentiating factor is the "intended action
when caught up", and the definition of caught up, and trying to use a
single term for both of them is going to always cause confusion.
So I tend to think of the use cases by their "continuation". A
"slave" is intended to "continually keep trying to get more" once it's
retrieved and applied all the changes it can. It can be hot, or cold,
streaming, or archive, etc... And "recovery" is intended to stop
recovering and become "normal" once it's finished retrieving and
applying all changes it can. Again, it has multiple ways to retrive
it's wal too.
And I think Tom touched on this point in the
"recovery.conf/recovery.done" thread a bit too. Maybe we need to
really start talking about the different "when done do ..."
distinctions, and and using that distinction to help our nomenclature.
Both recovery/slave (both hot or cold) use the same retrieve/apply
machinery (and thus configuration options). But because of the
different "caught up action", are different features.
a.
--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
On Wed, Sep 21, 2011 at 1:34 PM, Aidan Van Dyk <aidan@highrise.ca> wrote:
And I think Tom touched on this point in the
"recovery.conf/recovery.done" thread a bit too.
Doh! That's this thread....
/me slinks away, ashamed for not even taking a close look at the to/cc list...
--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
On tis, 2011-09-20 at 16:38 -0400, Robert Haas wrote:
For now, I think we're best off not changing the terminology, and
confining the remit of this patch to (a) turning all of the existing
recovery.conf parameters into GUCs and (b) replacing recovery.conf
with a sentinel file a sentinel file (name TBB) to indicate that the
server is to start in recovery mode. The naming isn't great but the
more we change at once the less chance of reaching agreement. It
seems like we have pretty broad agreement on the basics here, so let's
start with that.
The only thing that's slightly bogus about that is that if you were
doing an archive recovery, you'd have to edit the main postgresql.conf
with one-shot parameters for that particular recovery (and then delete
them again, or leave them in place, confusing the next guy). But
perhaps that's worth the overall simplification.
On Tue, Sep 20, 2011 at 5:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Simon Riggs <simon@2ndQuadrant.com> writes:
I sympathise with this view, to an extent.
If people want to put all parameters in one file, they can do so. So +1 to that.
Should they be forced to adopt that new capability by us deliberately
breaking their existing setups? No. So -1 to that.If we do an automatic include of recovery.conf first, then follow by
reading postgresql,conf then we will preserve the old as well as
allowing the new.I don't buy this argument at all. I don't believe that recovery.conf is
part of anyone's automated processes at all, let alone to an extent that
they won't be able to cope with a change to rationalize the file layout.
There are many. Tools I can name include pgpool, 2warm, PITRtools, but
there are also various tools from Sun, an IBM reseller I have
forgotten the name of, OmniTI and various other backup software
providers. Those are just the ones I can recall quickly. We've
encouraged people to write software on top and they have done so.
Breaking backwards compatibility isn't something we do elsewhere, when
its easy to keep it.
I don't object to new functionality (and agreed to it upthread), just
don't break the old way when there is no need.
And most especially I don't buy that someone who does want to keep using
it couldn't cope with adding an "include" to postgresql.conf manually.
This just creates a barrier to upgrade. Most people using PostgreSQL
use multiple releases, so its a pain to have to maintain multiple
versions of scripts to have things work properly. Even people that
don't mind changing won't be able to do it fully. That creates
confusion, which leads to mistakes.
These things relate to backup and recovery. Any changes to them give
risk of data loss. Cosmetic considerations are secondary.
There is no command to safely confirm that "include recovery.conf" is
added to postgresql.conf, so leaving it optional makes it unclear as
to whether the old ways will work or not.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 09/20/2011 09:23 AM, Tom Lane wrote:
Simon Riggs<simon@2ndQuadrant.com> writes:
I sympathise with this view, to an extent.
If we do an automatic include of recovery.conf first, then follow by
reading postgresql,conf then we will preserve the old as well as
allowing the new.I don't buy this argument at all. I don't believe that recovery.conf is
part of anyone's automated processes at all, let alone to an extent that
they won't be able to cope with a change to rationalize the file layout.
And most especially I don't buy that someone who does want to keep using
it couldn't cope with adding an "include" to postgresql.conf manually.
As Simon has already appropriately posted.... You would be incorrect.
Joshua D. Drake
--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579
Simon,
There are many. Tools I can name include pgpool, 2warm, PITRtools, but
there are also various tools from Sun, an IBM reseller I have
forgotten the name of, OmniTI and various other backup software
providers. Those are just the ones I can recall quickly. We've
encouraged people to write software on top and they have done so.
Actually, just to correct this list:
* there are no tools from Sun
* pgPool2 does not deal with recovery.conf
* there are additional ones: WAL-E, etc., which may or may not need to
be edited.
Breaking backwards compatibility isn't something we do elsewhere, when
its easy to keep it.
FWIW, I've already found that I have to modify all my backup scripts for
9.0 from 8.4, and again for 9.1 from 9.0. So we do break backwards
compatibility, frequently.
I don't object to new functionality (and agreed to it upthread), just
don't break the old way when there is no need.
I'm happy to make upgrades easier, but I want a path which eventually
ends in recovery.conf going away. It's a bad API, confuses our users,
and is difficult to support and maintain. If you think it's easier on
our users to do that in stages over several versions rather than in one
fell swoop, then let's plan it that way.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Fri, Sep 23, 2011 at 12:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
I'm happy to make upgrades easier, but I want a path which eventually
ends in recovery.conf going away. It's a bad API, confuses our users,
and is difficult to support and maintain.
I agree.
GUC = Grand Unified Configuration, but recovery.conf is an example of
where it's not so unified after all. We've already done a non-trivial
amount of work to allow recovery.conf values to be specified without
quotes, a random incompatibility with GUCs that resulted from having
different parsing code for each file. If that were the last issue,
then maybe it wouldn't be worth worrying about, but it's not. For
example, it would be nice to have reload behavior on SIGHUP. And we
keep talking about having an ALTER SYSTEM SET guc = value or SET
PERMANENT guc = value command, and I think ALTER SYSTEM SET
recovery_target_time = '...' would be pretty sweet. I don't want us
to have to implement such things separately for postgresql.conf and
recovery.conf.
Now, it's true that Simon's proposal (of having recovery.conf
automatically included) if it exists doesn't necessarily preclude
those things. But it seems to me that it is adding a lot of
complexity to core for a pretty minimal benefit to end-users, and that
the semantics are not going to end up being very clean. For example,
now you potentially have the situation where recovery.conf has
work_mem=128MB and postgresql.conf has work_mem=4MB, and now when you
end recovery you've got to make sure that everyone picks up the new
setting. Now, in some sense you could say that's a feature addition,
and I'm not going to deny that it might be useful to some people, but
I think it's also going to require a fairly substantial convolution of
the GUC machinery, and it's going to discourage people from moving
away from recovery.conf. And like Josh, I think that ought to be the
long-term goal, for the reasons he states.
I don't want to go willy-nilly breaking third-party tools that work
with PostgreSQL, but in this case I think that the reason there are so
many tools in the first place is because what we're providing in core
is not very good. If we are unwilling to improve it for fear of
breaking compatibility with the tools, then we are stuck.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Sep23, 2011, at 17:46 , Peter Eisentraut wrote:
On tis, 2011-09-20 at 16:38 -0400, Robert Haas wrote:
For now, I think we're best off not changing the terminology, and
confining the remit of this patch to (a) turning all of the existing
recovery.conf parameters into GUCs and (b) replacing recovery.conf
with a sentinel file a sentinel file (name TBB) to indicate that the
server is to start in recovery mode. The naming isn't great but the
more we change at once the less chance of reaching agreement. It
seems like we have pretty broad agreement on the basics here, so let's
start with that.The only thing that's slightly bogus about that is that if you were
doing an archive recovery, you'd have to edit the main postgresql.conf
with one-shot parameters for that particular recovery (and then delete
them again, or leave them in place, confusing the next guy). But
perhaps that's worth the overall simplification.
OTOH, if they're GUCs, you can specify them on the postmaster's
command line. We could even get crazy and patch pg_ctl to allow
<untar base backup>
pg_ctl recover -D <dir> --target_xid=<the xid that broke stuff> --target_inclusive=false
pg_ctl start -D <dir>
best regards,
Florian Pflug
Josh,
There are many. Tools I can name include pgpool, 2warm, PITRtools, but
there are also various tools from Sun, an IBM reseller I have
forgotten the name of, OmniTI and various other backup software
providers. Those are just the ones I can recall quickly. We've
encouraged people to write software on top and they have done so.Actually, just to correct this list:
* there are no tools from Sun
* pgPool2 does not deal with recovery.conf
I'm not sure what you mean by "not deal with" but part of pgpool-II's
functionality assumes that we can easily generate recovery.conf. If
reconf.conf is integrated into postgresql.conf, we need to edit
postgresql.conf, which is a little bit harder than generating
recovery.conf, I think.
Show quoted text
* there are additional ones: WAL-E, etc., which may or may not need to
be edited.Breaking backwards compatibility isn't something we do elsewhere, when
its easy to keep it.FWIW, I've already found that I have to modify all my backup scripts for
9.0 from 8.4, and again for 9.1 from 9.0. So we do break backwards
compatibility, frequently.I don't object to new functionality (and agreed to it upthread), just
don't break the old way when there is no need.I'm happy to make upgrades easier, but I want a path which eventually
ends in recovery.conf going away. It's a bad API, confuses our users,
and is difficult to support and maintain. If you think it's easier on
our users to do that in stages over several versions rather than in one
fell swoop, then let's plan it that way.--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I'm not sure what you mean by "not deal with" but part of pgpool-II's
functionality assumes that we can easily generate recovery.conf. If
reconf.conf is integrated into postgresql.conf, we need to edit
postgresql.conf, which is a little bit harder than generating
recovery.conf, I think.
Oh? Clearly I've been abusing pgPool2 then. Where's the code that
generates that?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Fri, Sep 23, 2011 at 6:17 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Sep 23, 2011 at 12:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
I'm happy to make upgrades easier, but I want a path which eventually
ends in recovery.conf going away. It's a bad API, confuses our users,
and is difficult to support and maintain.I agree.
GUC = Grand Unified Configuration, but recovery.conf is an example of
where it's not so unified after all. We've already done a non-trivial
amount of work to allow recovery.conf values to be specified without
quotes, a random incompatibility with GUCs that resulted from having
different parsing code for each file. If that were the last issue,
then maybe it wouldn't be worth worrying about, but it's not. For
example, it would be nice to have reload behavior on SIGHUP.
...
I don't want us
to have to implement such things separately for postgresql.conf and
recovery.conf.
It was always my plan to do exactly the above, and there are code
comments that say that from 2004. The time to replace it is now and I
welcome that day and have already agreed to it.
We all want every word quoted above and nothing there is under debate.
And we
keep talking about having an ALTER SYSTEM SET guc = value or SET
PERMANENT guc = value command, and I think ALTER SYSTEM SET
recovery_target_time = '...' would be pretty sweet. I don't want us
to have to implement such things separately for postgresql.conf and
recovery.conf.
There is a reason why it doesn't work that way which you overlook.
Please start a separate thread if you wish to discuss that.
Now, it's true that Simon's proposal (of having recovery.conf
automatically included) if it exists doesn't necessarily preclude
those things. But it seems to me that it is adding a lot of
complexity to core for a pretty minimal benefit to end-users, and that
the semantics are not going to end up being very clean. For example,
now you potentially have the situation where recovery.conf has
work_mem=128MB and postgresql.conf has work_mem=4MB, and now when you
end recovery you've got to make sure that everyone picks up the new
setting. Now, in some sense you could say that's a feature addition,
and I'm not going to deny that it might be useful to some people, but
I think it's also going to require a fairly substantial convolution of
the GUC machinery, and it's going to discourage people from moving
away from recovery.conf. And like Josh, I think that ought to be the
long-term goal, for the reasons he states.
The semantics are clear: recovery.conf is read first, then
postgresql.conf. It's easy to implement (1 line of code) and easy to
understand.
So we can support the old and the new very, very easily and clearly.
"Complexity" - no definitely not. "Minimal benefit for end users" -
backwards compatibility isn't minimal benefit. It's a major issue.
If you put things in two places, yes that causes problems. You can
already add the same parameter twice and cause exactly the same
problems.
I don't want to go willy-nilly breaking third-party tools that work
with PostgreSQL, but in this case I think that the reason there are so
many tools in the first place is because what we're providing in core
is not very good. If we are unwilling to improve it for fear of
breaking compatibility with the tools, then we are stuck.
No, there are many tools because there are many requirements. A
simple, open API has allowed our technology to be widely used. That
was by design not by chance.
Nobody is unwilling to improve it. The debate is about people being
unwilling to provide a simple and easy to understand backwards
compatibility feature, which breaks things for no reason and does not
interfere with the proposed new features.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Fri, Sep 23, 2011 at 5:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
Simon,
There are many. Tools I can name include pgpool, 2warm, PITRtools, but
there are also various tools from Sun, an IBM reseller I have
forgotten the name of, OmniTI and various other backup software
providers. Those are just the ones I can recall quickly. We've
encouraged people to write software on top and they have done so.Actually, just to correct this list:
* there are no tools from Sun
Just for the record, I sat through a 1 hour presentation at the
PostgreSQL UK conference from a Sun employee describing the product
and its very clear use of PostgreSQL facilities. Josh definitely
wasn't at that presentation, many others here were.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
I'm not sure what you mean by "not deal with" but part of pgpool-II's
functionality assumes that we can easily generate recovery.conf. If
reconf.conf is integrated into postgresql.conf, we need to edit
postgresql.conf, which is a little bit harder than generating
recovery.conf, I think.Oh? Clearly I've been abusing pgPool2 then. Where's the code that
generates that?
pgpool-II itself does not generate the file but scripts for pgpool-II
are generating the file as stated in documentation comimg with
pgpool-II.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp
On Sat, Sep 24, 2011 at 9:02 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
The semantics are clear: recovery.conf is read first, then
postgresql.conf. It's easy to implement (1 line of code) and easy to
understand.
Eh, well, if you can implement it in one line of code, consider my
objection withdrawn. I can't see how that would be possible, though.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Fri, Sep 23, 2011 at 6:55 PM, Tatsuo Ishii <ishii@postgresql.org> wrote:
There are many. Tools I can name include pgpool, 2warm, PITRtools, but
there are also various tools from Sun, an IBM reseller I have
forgotten the name of, OmniTI and various other backup software
providers. Those are just the ones I can recall quickly. We've
encouraged people to write software on top and they have done so.Actually, just to correct this list:
* there are no tools from Sun
* pgPool2 does not deal with recovery.confI'm not sure what you mean by "not deal with" but part of pgpool-II's
functionality assumes that we can easily generate recovery.conf. If
reconf.conf is integrated into postgresql.conf, we need to edit
postgresql.conf, which is a little bit harder than generating
recovery.conf, I think.
Since we haven't yet come up with a reasonable way of machine-editing
postgresql.conf, this seems like a fairly serious objection to getting
rid of recovery.conf. I wonder if there's a way we can work around
that...
*thinks a little*
What if we modified pg_ctl to allow passing configuration parameters
through to postmaster, so you could do something like this:
pg_ctl start -c work_mem=8MB -c recovery_target_time='...'
Would that meet pgpool's needs?
(Sadly pg_ctl -c means something else right now, so we'd probably have
to pick another option name, but you get the idea.)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Simon Riggs <simon@2ndQuadrant.com> writes:
... The time to replace it is now and I
welcome that day and have already agreed to it.
Okay, so you do agree that eventually we want to be rid of
recovery.conf? I think everyone else agrees on that. But if we are
going to remove recovery.conf eventually, what is the benefit of
postponing doing so? The pain is going to be inflicted sooner or later,
and the longer we wait, the more third-party code there is likely to be
that expects it to exist. If optionally reading it helped provide a
smoother transition, then maybe there would be some point. But AFAICS
having a temporary third behavior will just make things even more
complicated, not less so, for third-party code that needs to cope with
multiple versions.
The semantics are clear: recovery.conf is read first, then
postgresql.conf. It's easy to implement (1 line of code) and easy to
understand.
It's not clear to me why the override order should be that way; I'd have
expected the other way myself. So this isn't as open-and-shut as you
think.
regards, tom lane
Robert Haas <robertmhaas@gmail.com> writes:
On Fri, Sep 23, 2011 at 6:55 PM, Tatsuo Ishii <ishii@postgresql.org> wrote:
I'm not sure what you mean by "not deal with" but part of pgpool-II's
functionality assumes that we can easily generate recovery.conf. If
reconf.conf is integrated into postgresql.conf, we need to edit
postgresql.conf, which is a little bit harder than generating
recovery.conf, I think.
Since we haven't yet come up with a reasonable way of machine-editing
postgresql.conf, this seems like a fairly serious objection to getting
rid of recovery.conf.
I don't exactly buy this argument. If postgresql.conf is hard to
machine-edit, why is recovery.conf any easier?
What if we modified pg_ctl to allow passing configuration parameters
through to postmaster,
You mean like pg_ctl -o?
regards, tom lane
On Sep 24, 2011, at 1:04 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I don't exactly buy this argument. If postgresql.conf is hard to
machine-edit, why is recovery.conf any easier?
Because you generally just write a brand-new file, without worrying about preserving existing settings. You aren't really editing at all, just writing.
What if we modified pg_ctl to allow passing configuration parameters
through to postmaster,You mean like pg_ctl -o?
Oh, cool. Yes, like that.
...Robert
On Sat, Sep 24, 2011 at 6:01 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Simon Riggs <simon@2ndQuadrant.com> writes:
... The time to replace it is now and I
welcome that day and have already agreed to it.Okay, so you do agree that eventually we want to be rid of
recovery.conf? I think everyone else agrees on that. But if we are
going to remove recovery.conf eventually, what is the benefit of
postponing doing so?
I am happy that we don't recommend the use of recovery.conf in the
future, and that the handling of the contents of recovery.conf should
be identical to the handling of postgresql.conf. The latter point has
always been the plan from day one.
(I should not have used "it" in my sentence, since doing so always
leads to confusion)
My joyous rush into agreeing to removal has since been replaced with
the cold reality that we must support backwards compatibility.
Emphasise "must".
The semantics are clear: recovery.conf is read first, then
postgresql.conf. It's easy to implement (1 line of code) and easy to
understand.It's not clear to me why the override order should be that way; I'd have
expected the other way myself. So this isn't as open-and-shut as you
think.
I agree with you that recovery.conf as an override makes more sense,
though am happy with either way.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Since we haven't yet come up with a reasonable way of machine-editing
postgresql.conf, this seems like a fairly serious objection to
getting
rid of recovery.conf. I wonder if there's a way we can work around
that...
Well, we *did* actually come up with a reasonable way, but it died under an avalanche of bikeshedding and "we-must-do-everything-the-way-we-always-have-done". I refer, of course, to the "configuration directory" patch, which was a fine solution, and would indeed take care of the recovery.conf issues as well had we implemented it. We can *still* implement it, for 9.2.
pg_ctl start -c work_mem=8MB -c recovery_target_time='...'
This wouldn't survive a restart, and isn't compatible with init scripts.
On Sat, Sep 24, 2011 at 3:49 PM, Joshua Berkus <josh@agliodbs.com> wrote:
Since we haven't yet come up with a reasonable way of machine-editing
postgresql.conf, this seems like a fairly serious objection to
getting
rid of recovery.conf. I wonder if there's a way we can work around
that...Well, we *did* actually come up with a reasonable way, but it died under an avalanche of bikeshedding and "we-must-do-everything-the-way-we-always-have-done". I refer, of course, to the "configuration directory" patch, which was a fine solution, and would indeed take care of the recovery.conf issues as well had we implemented it. We can *still* implement it, for 9.2.
Well, I find that a fairly ugly solution to the problem, but I agree
that it is solvable, if we could get a critical mass on any some
particular solution.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Simon Riggs <simon@2ndQuadrant.com> writes:
On Sat, Sep 24, 2011 at 6:01 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Okay, so you do agree that eventually we want to be rid of
recovery.conf? �I think everyone else agrees on that. �But if we are
going to remove recovery.conf eventually, what is the benefit of
postponing doing so?
My joyous rush into agreeing to removal has since been replaced with
the cold reality that we must support backwards compatibility.
Emphasise "must".
[ shrug... ] I do not agree with your conclusion. We have to break
some eggs to make this omelet. The reason why we have a mess here is
that the recovery.conf mechanism, which was designed with only the
one-shot archive-recovery case in mind, has been abused beyond its
capacity. If we don't break with past practice we are not going to be
able to fix it. And it's not like we don't break configuration file
contents in most releases anyway, so I really fail to see why this one
has suddenly become sacrosanct.
regards, tom lane
Robert Haas <robertmhaas@gmail.com> writes:
On Sep 24, 2011, at 1:04 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I don't exactly buy this argument. If postgresql.conf is hard to
machine-edit, why is recovery.conf any easier?
Because you generally just write a brand-new file, without worrying
about preserving existing settings. You aren't really editing at all,
just writing.
If that's all the requirement is, it's trivial to implement.
1. Write your-random-configuration-settings into recovery.conf (or any
other file name you choose ... something named after your tool would be
a better idea).
2. Temporarily append "include recovery.conf" to the end of
postgresql.conf. Restart server.
3. When done, remove "include recovery.conf" from the end of
postgresql.conf.
The hard cases involve merging user-supplied and tool-supplied settings,
but "let's overwrite recovery.conf in toto" never would have been able
to handle such cases either.
regards, tom lane
Folks,
What happens currently if we have an \include in postgresql.conf for a file which doesn't exist? Is it ignored, or do we error out?
If it could just be ignored, maybe with a note in the logs, then we could be a lot more flexible.
--Josh Berkus
Joshua Berkus <josh@agliodbs.com> writes:
What happens currently if we have an \include in postgresql.conf for a file which doesn't exist? Is it ignored, or do we error out?
It's an error, and I think changing that would be a really bad idea.
If it could just be ignored, maybe with a note in the logs, then we could be a lot more flexible.
There might be a use case for a separate directive include_if_exists,
or some such name. But I think the user should have to tell us very
clearly that it's okay for the file to not be found.
regards, tom lane
There might be a use case for a separate directive include_if_exists,
or some such name. But I think the user should have to tell us very
clearly that it's okay for the file to not be found.
Better to go back to include_directory, then.
--Josh Berkus
On 09/25/2011 02:39 PM, Joshua Berkus wrote:
There might be a use case for a separate directive include_if_exists,
or some such name. But I think the user should have to tell us very
clearly that it's okay for the file to not be found.Better to go back to include_directory, then.
I rather like Tom's suggestion of include_if_exists.
There might be a case for include_directory, but I think that needs to
be made separately.
cheers
andrew
I rather like Tom's suggestion of include_if_exists.
include_if_exists certainly solves the recovery.conf/recovery.done problem. We can even phase it out, like this:
9.2: include_if_exists = 'recovery.conf' in the default postgresql.conf file.
9.3: include_if_exists = 'recovery.conf' commented out by default
9.4: renaming recovery.conf to recovery.done by core PG code removed.
This gives users/vendors 3 years to update their scripts to remove dependence on recovery.conf. I'm afraid that I agree with Simon that there's already a whole buncha 3rd-party code out there to support the current system.
--Josh Berkus
Joshua Berkus <josh@agliodbs.com> writes:
include_if_exists certainly solves the recovery.conf/recovery.done problem. We can even phase it out, like this:
9.2: include_if_exists = 'recovery.conf' in the default postgresql.conf file.
9.3: include_if_exists = 'recovery.conf' commented out by default
9.4: renaming recovery.conf to recovery.done by core PG code removed.
This gives users/vendors 3 years to update their scripts to remove dependence on recovery.conf. I'm afraid that I agree with Simon that there's already a whole buncha 3rd-party code out there to support the current system.
If there is indeed code out there that depends on the current system,
why do you think that waiting several releases to change it will make
things better? I think it's more likely that we'd just have *more* code
that needs to be changed, and no reduction in the pain level when the
transition does finally happen.
In any case, I thought we'd agreed that the use of that file as a flag
should go away now, so I quite fail to understand your comment about 9.4.
regards, tom lane
On lör, 2011-09-24 at 13:04 -0400, Tom Lane wrote:
What if we modified pg_ctl to allow passing configuration parameters
through to postmaster,You mean like pg_ctl -o?
Note that pg_ctl -o saves the parameters it uses and reapplies them
after a restart. So this is not really the way to apply parameter
settings only once for recovery. (Or at least it has the potential to
be a very confusing way.)
On lör, 2011-09-24 at 14:02 +0100, Simon Riggs wrote:
The semantics are clear: recovery.conf is read first, then
postgresql.conf. It's easy to implement (1 line of code) and easy to
understand.
What's clear about that? My intuition would have been that
recovery.conf is read second.
On sön, 2011-09-25 at 12:58 -0400, Tom Lane wrote:
And it's not like we don't break configuration file
contents in most releases anyway, so I really fail to see why this one
has suddenly become sacrosanct.
Well, there is a slight difference. Changes in postgresql.conf
parameter names and settings are adjusted automatically for me by my
package upgrade script. If we, say, changed the names of recovery.conf
parameters, I'd have to get a new version of my $SuperReplicationTool.
That tool could presumably look at PG_VERSION and put a recovery.conf
with the right spellings in the right place.
But if we completely change the way the replication configuration
interacts, it's not clear that a smooth upgrade is possible without
significant effort. That said, I don't see why it wouldn't be possible,
but let's design with upgradability in mind, instead of claiming that we
have never supported upgrades of this kind anyway.
On Tue, Sep 20, 2011 at 3:38 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
What about "recovery.trigger"?
I'm OK with that name.
<snip>
* is there any security hazard from ordinary users being able to see
what settings had been used?primary_conninfo could be a problem, since it's possible to set a password there.
True. I agree that primary_conninfo should be restricted to superuser.
Though there is still ongonig discussion, since there is no objection about
the above two changes, I revised the patch that way. And I fixed the minor
bug handling the default value of recovery_target_timeline wrongly.
Attached is the revised version of the patch.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Attachments:
unite_recoveryconf_postgresqlconf_v2.patchtext/x-patch; charset=US-ASCII; name=unite_recoveryconf_postgresqlconf_v2.patchDownload
*** a/contrib/pg_archivecleanup/pg_archivecleanup.c
--- b/contrib/pg_archivecleanup/pg_archivecleanup.c
***************
*** 208,214 **** usage(void)
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "For use as archive_cleanup_command in recovery.conf when standby_mode = on:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
"e.g.\n"
" archive_cleanup_command = 'pg_archivecleanup /mnt/server/archiverdir %%r'\n");
--- 208,214 ----
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "For use as archive_cleanup_command in postgresql.conf when standby_mode = on:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
"e.g.\n"
" archive_cleanup_command = 'pg_archivecleanup /mnt/server/archiverdir %%r'\n");
*** a/contrib/pg_standby/pg_standby.c
--- b/contrib/pg_standby/pg_standby.c
***************
*** 531,537 **** usage(void)
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "Main intended use as restore_command in recovery.conf:\n"
" restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
"e.g.\n"
" restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
--- 531,537 ----
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "Main intended use as restore_command in postgresql.conf:\n"
" restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
"e.g.\n"
" restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
*** a/doc/src/sgml/backup.sgml
--- b/doc/src/sgml/backup.sgml
***************
*** 995,1002 **** SELECT pg_stop_backup();
</listitem>
<listitem>
<para>
! Create a recovery command file <filename>recovery.conf</> in the cluster
! data directory (see <xref linkend="recovery-config">). You might
also want to temporarily modify <filename>pg_hba.conf</> to prevent
ordinary users from connecting until you are sure the recovery was successful.
</para>
--- 995,1009 ----
</listitem>
<listitem>
<para>
! Set up recovery parameters in <filename>postgresql.conf</> (see
! <xref linkend="runtime-config-wal-archive-recovery"> and
! <xref linkend="runtime-config-wal-recovery-target">).
! </para>
! </listitem>
! <listitem>
! <para>
! Create a recovery trigger file <filename>recovery.trigger</>
! in the cluster data directory. You might
also want to temporarily modify <filename>pg_hba.conf</> to prevent
ordinary users from connecting until you are sure the recovery was successful.
</para>
***************
*** 1008,1014 **** SELECT pg_stop_backup();
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
! <filename>recovery.conf</> to <filename>recovery.done</> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
</para>
--- 1015,1021 ----
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
! <filename>recovery.trigger</> to <filename>recovery.done</> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
</para>
***************
*** 1024,1035 **** SELECT pg_stop_backup();
</para>
<para>
! The key part of all this is to set up a recovery configuration file that
! describes how you want to recover and how far the recovery should
! run. You can use <filename>recovery.conf.sample</> (normally
! located in the installation's <filename>share/</> directory) as a
! prototype. The one thing that you absolutely must specify in
! <filename>recovery.conf</> is the <varname>restore_command</>,
which tells <productname>PostgreSQL</> how to retrieve archived
WAL file segments. Like the <varname>archive_command</>, this is
a shell command string. It can contain <literal>%f</>, which is
--- 1031,1041 ----
</para>
<para>
! The key part of all this is to set up recovery parameters that
! specify how you want to recover and how far the recovery should
! run. The one thing that you absolutely must specify in
! <filename>postgresql.conf</> to recover from the backup is
! the <varname>restore_command</>,
which tells <productname>PostgreSQL</> how to retrieve archived
WAL file segments. Like the <varname>archive_command</>, this is
a shell command string. It can contain <literal>%f</>, which is
***************
*** 1085,1091 **** restore_command = 'cp /mnt/server/archivedir/%f %p'
<para>
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
! required stopping point in <filename>recovery.conf</>. You can specify
the stop point, known as the <quote>recovery target</>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
--- 1091,1097 ----
<para>
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
! required stopping point in <filename>postgresql.conf</>. You can specify
the stop point, known as the <quote>recovery target</>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
***************
*** 1182,1189 **** restore_command = 'cp /mnt/server/archivedir/%f %p'
The default behavior of recovery is to recover along the same timeline
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
! was itself generated after a recovery attempt), you need to specify the
! target timeline ID in <filename>recovery.conf</>. You cannot recover into
timelines that branched off earlier than the base backup.
</para>
</sect2>
--- 1188,1196 ----
The default behavior of recovery is to recover along the same timeline
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
! was itself generated after a recovery attempt), you need to set
! <xref linkend="guc-recovery-target-timeline"> to the
! target timeline ID in <filename>postgresql.conf</>. You cannot recover into
timelines that branched off earlier than the base backup.
</para>
</sect2>
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 1961,1966 **** SET ENABLE_SEQSCAN TO OFF;
--- 1961,2238 ----
</variablelist>
</sect2>
+ <sect2 id="runtime-config-wal-archive-recovery">
+ <title>Archive Recovery</title>
+
+ <variablelist>
+ <varlistentry id="guc-restore-command" xreflabel="restore_command">
+ <term><varname>restore_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>restore_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command to execute to retrieve an archived segment of
+ the WAL file series. This parameter is required for archive recovery,
+ but optional for streaming replication.
+ Any <literal>%f</> in the string is
+ replaced by the name of the file to retrieve from the archive,
+ and any <literal>%p</> is replaced by the copy destination path name
+ on the server.
+ (The path name is relative to the current working directory,
+ i.e., the cluster's data directory.)
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point. That is the earliest file that must be kept
+ to allow a restore to be restartable, so this information can be used
+ to truncate the archive to just the minimum required to support
+ restarting from the current restore. <literal>%r</> is typically only
+ used by warm-standby configurations
+ (see <xref linkend="warm-standby">).
+ Write <literal>%%</> to embed an actual <literal>%</> character.
+ </para>
+ <para>
+ It is important for the command to return a zero exit status
+ only if it succeeds. The command <emphasis>will</> be asked for file
+ names that are not present in the archive; it must return nonzero
+ when so asked. Examples:
+ <programlisting>
+ restore_command = 'cp /mnt/server/archivedir/%f "%p"'
+ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
+ </programlisting>
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-archive-cleanup-command" xreflabel="archive_cleanup_command">
+ <term><varname>archive_cleanup_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>archive_cleanup_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command that will be executed at every restartpoint.
+ The purpose of <varname>archive_cleanup_command</> is to
+ provide a mechanism for cleaning up old archived WAL files that
+ are no longer needed by the standby server.
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point.
+ That is the earliest file that must be <emphasis>kept</> to allow a
+ restore to be restartable, and so all files earlier than <literal>%r</>
+ may be safely removed.
+ This information can be used to truncate the archive to just the
+ minimum required to support restart from the current restore.
+ The <xref linkend="pgarchivecleanup"> module
+ is often used in <varname>archive_cleanup_command</> for
+ single-standby configurations, for example:
+ <programlisting>archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'</programlisting>
+ Note however that if multiple standby servers are restoring from the
+ same archive directory, you will need to ensure that you do not delete
+ WAL files until they are no longer needed by any of the servers.
+ <varname>archive_cleanup_command</> would typically be used in a
+ warm-standby configuration (see <xref linkend="warm-standby">).
+ Write <literal>%%</> to embed an actual <literal>%</> character in the
+ command.
+ </para>
+ <para>
+ If the command returns a non-zero exit status then a WARNING log
+ message will be written.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-end-command" xreflabel="recovery_end_command">
+ <term><varname>recovery_end_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_end_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command that will be executed once only
+ at the end of recovery. This parameter is optional. The purpose of the
+ <varname>recovery_end_command</> is to provide a mechanism for cleanup
+ following replication or recovery.
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point, like in <varname>archive_cleanup_command</>.
+ </para>
+ <para>
+ If the command returns a non-zero exit status then a WARNING log
+ message will be written and the database will proceed to start up
+ anyway. An exception is that if the command was terminated by a
+ signal, the database will not proceed with startup.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="runtime-config-wal-recovery-target">
+ <title>Recovery Target</title>
+
+ <variablelist>
+ <varlistentry id="guc-recovery-target-name" xreflabel="recovery_target_name">
+ <term><varname>recovery_target_name</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_name</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the named restore point, created with
+ <function>pg_create_restore_point()</> to which recovery will proceed.
+ At most one of <varname>recovery_target_name</>,
+ <varname>recovery_target_time</> or
+ <varname>recovery_target_xid</> can be specified. The default
+ value is an empty string, which will recover to the end of the WAL log.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-time" xreflabel="recovery_target_time">
+ <term><varname>recovery_target_time</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_time</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the time stamp up to which recovery will proceed.
+ This parameter must be specified in the date/time format
+ (see <xref linkend="datatype-datetime-input"> for details).
+ At most one of <varname>recovery_target_time</>,
+ <varname>recovery_target_name</> or
+ <varname>recovery_target_xid</> can be specified.
+ The default value is an empty string, which will recover to
+ the end of the WAL log. The precise stopping point is also
+ influenced by <varname>recovery_target_inclusive</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-xid" xreflabel="recovery_target_xid">
+ <term><varname>recovery_target_xid</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_xid</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the transaction ID up to which recovery will proceed.
+ Keep in mind that while transaction IDs are assigned sequentially
+ at transaction start, transactions can complete in a different
+ numeric order. The transactions that will be recovered are
+ those that committed before (and optionally including)
+ the specified one. At most one of <varname>recovery_target_xid</>,
+ <varname>recovery_target_name</> or
+ <varname>recovery_target_time</> can be specified.
+ The default value is an empty string, which will recover to the end of
+ the WAL log. The precise stopping point is also influenced by
+ <varname>recovery_target_inclusive</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-inclusive" xreflabel="recovery_target_inclusive">
+ <term><varname>recovery_target_inclusive</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_inclusive</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether we stop just after the specified recovery target
+ (<literal>on</>), or just before the recovery target (<literal>off</>).
+ Applies to both <varname>recovery_target_time</>
+ and <varname>recovery_target_xid</>, whichever one is
+ specified for this recovery. This indicates whether transactions
+ having exactly the target commit time or ID, respectively, will
+ be included in the recovery. Default is <literal>on</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-timeline" xreflabel="recovery_target_timeline">
+ <term><varname>recovery_target_timeline</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_timeline</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies recovering into a particular timeline. The default value is
+ an empty string, which will recover along the same timeline that was
+ current when the base backup was taken. Setting this to
+ <literal>latest</> recovers to the latest timeline found in the archive,
+ which is useful in a standby server. Other than that you only need to
+ set this parameter in complex re-recovery situations, where you need
+ to return to a state that itself was reached after a point-in-time
+ recovery. See <xref linkend="backup-timelines"> for discussion.
+ </para>
+ <para>
+ This parameter can only be set at server start. It only has effect
+ during archive recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-pause-at-recovery-target" xreflabel="pause_at_recovery_target">
+ <term><varname>pause_at_recovery_target</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>pause_at_recovery_target</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether recovery should pause when the recovery target
+ is reached. The default is <literal>on</>.
+ This is intended to allow queries to be executed against the
+ database to check if this recovery target is the most desirable
+ point for recovery. The paused state can be resumed by using
+ <function>pg_xlog_replay_resume()</> (See
+ <xref linkend="functions-recovery-control-table">), which then
+ causes recovery to end. If this recovery target is not the
+ desired stopping point, then shutdown the server, change the
+ recovery target settings to a later target and restart to
+ continue recovery.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode if recovery target is set.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
</sect1>
<sect1 id="runtime-config-replication">
***************
*** 2182,2187 **** SET ENABLE_SEQSCAN TO OFF;
--- 2454,2546 ----
<variablelist>
+ <varlistentry id="guc-standby-mode" xreflabel="standby_mode">
+ <term><varname>standby_mode</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>standby_mode</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether to start the <productname>PostgreSQL</> server as
+ a standby when recovery trigger file <filename>recovery.trigger</> exists.
+ The default value is <literal>off</>.
+ If this parameter is <literal>on</>, the server will not
+ stop recovery when the end of archived WAL is reached,
+ but will keep trying to continue recovery by fetching new WAL segments
+ using <varname>restore_command</> and/or by connecting to
+ the primary server as specified by the <varname>primary_conninfo</>
+ setting.
+ </para>
+ <para>
+ This parameter can only be set at server start. It only has effect
+ if recovery trigger file <filename>recovery.trigger</> exists.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-primary-conninfo" xreflabel="primary_conninfo">
+ <term><varname>primary_conninfo</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>primary_conninfo</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies a connection string to be used for the standby server
+ to connect with the primary. This string is in the format
+ accepted by the libpq <function>PQconnectdb</function> function,
+ described in <xref linkend="libpq-connect">. If any option is
+ unspecified in this string, then the corresponding environment
+ variable (see <xref linkend="libpq-envars">) is checked. If the
+ environment variable is not set either, then defaults are used.
+ If this parameter is an empty string (the default), no attempt is
+ made to connect to the master.
+ </para>
+ <para>
+ The connection string should specify the host name (or address)
+ of the primary server, as well as the port number if it is not
+ the same as the standby server's default.
+ Also specify a user name corresponding to a role that has the
+ <literal>REPLICATION</> and <literal>LOGIN</> privileges on the
+ primary (see
+ <xref linkend="streaming-replication-authentication">).
+ A password needs to be provided too, if the primary demands password
+ authentication. It can be provided in the
+ <varname>primary_conninfo</varname> string, or in a separate
+ <filename>~/.pgpass</> file on the standby server (use
+ <literal>replication</> as the database name).
+ Do not specify a database name in the
+ <varname>primary_conninfo</varname> string.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect in standby mode.
+ </para>
+ <para>
+ If this parameter is changed while replication is in progress,
+ the standby terminates replication, and then tries to restart
+ replication with new setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-trigger-file" xreflabel="trigger_file">
+ <term><varname>trigger_file</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>trigger_file</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies a trigger file whose presence ends recovery in the
+ standby. Even if this value is not set, you can still promote
+ the standby using <command>pg_ctl promote</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-hot-standby" xreflabel="hot_standby">
<term><varname>hot_standby</varname> (<type>boolean</type>)</term>
<indexterm>
*** a/doc/src/sgml/filelist.sgml
--- b/doc/src/sgml/filelist.sgml
***************
*** 42,48 ****
<!ENTITY manage-ag SYSTEM "manage-ag.sgml">
<!ENTITY monitoring SYSTEM "monitoring.sgml">
<!ENTITY regress SYSTEM "regress.sgml">
- <!ENTITY recovery-config SYSTEM "recovery-config.sgml">
<!ENTITY runtime SYSTEM "runtime.sgml">
<!ENTITY config SYSTEM "config.sgml">
<!ENTITY user-manag SYSTEM "user-manag.sgml">
--- 42,47 ----
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***************
*** 14134,14140 **** postgres=# select pg_start_backup('label_goes_here');
<function>pg_create_restore_point</> creates a named transaction log
record that can be used as recovery target, and returns the corresponding
transaction log location. The given name can then be used with
! <xref linkend="recovery-target-name"> to specify the point up to which
recovery will proceed. Avoid creating multiple restore points with the
same name, since recovery will stop at the first one whose name matches
the recovery target.
--- 14134,14140 ----
<function>pg_create_restore_point</> creates a named transaction log
record that can be used as recovery target, and returns the corresponding
transaction log location. The given name can then be used with
! <xref linkend="guc-recovery-target-name"> to specify the point up to which
recovery will proceed. Avoid creating multiple restore points with the
same name, since recovery will stop at the first one whose name matches
the recovery target.
*** a/doc/src/sgml/high-availability.sgml
--- b/doc/src/sgml/high-availability.sgml
***************
*** 591,597 **** protocol to make nodes agree on a serializable transactional order.
<para>
In standby mode, the server continuously applies WAL received from the
master server. The standby server can read WAL from a WAL archive
! (see <xref linkend="restore-command">) or directly from the master
over a TCP connection (streaming replication). The standby server will
also attempt to restore any WAL found in the standby cluster's
<filename>pg_xlog</> directory. That typically happens after a server
--- 591,597 ----
<para>
In standby mode, the server continuously applies WAL received from the
master server. The standby server can read WAL from a WAL archive
! (see <xref linkend="guc-restore-command">) or directly from the master
over a TCP connection (streaming replication). The standby server will
also attempt to restore any WAL found in the standby cluster's
<filename>pg_xlog</> directory. That typically happens after a server
***************
*** 658,665 **** protocol to make nodes agree on a serializable transactional order.
<para>
To set up the standby server, restore the base backup taken from primary
server (see <xref linkend="backup-pitr-recovery">). Create a recovery
! command file <filename>recovery.conf</> in the standby's cluster data
! directory, and turn on <varname>standby_mode</>. Set
<varname>restore_command</> to a simple command to copy files from
the WAL archive. If you plan to have multiple standby servers for high
availability purposes, set <varname>recovery_target_timeline</> to
--- 658,665 ----
<para>
To set up the standby server, restore the base backup taken from primary
server (see <xref linkend="backup-pitr-recovery">). Create a recovery
! trigger file <filename>recovery.trigger</> in the standby's cluster data
! directory. Turn on <varname>standby_mode</> and set
<varname>restore_command</> to a simple command to copy files from
the WAL archive. If you plan to have multiple standby servers for high
availability purposes, set <varname>recovery_target_timeline</> to
***************
*** 695,701 **** protocol to make nodes agree on a serializable transactional order.
<para>
If you're using a WAL archive, its size can be minimized using the <xref
! linkend="archive-cleanup-command"> parameter to remove files that are no
longer required by the standby server.
The <application>pg_archivecleanup</> utility is designed specifically to
be used with <varname>archive_cleanup_command</> in typical single-standby
--- 695,701 ----
<para>
If you're using a WAL archive, its size can be minimized using the <xref
! linkend="guc-archive-cleanup-command"> parameter to remove files that are no
longer required by the standby server.
The <application>pg_archivecleanup</> utility is designed specifically to
be used with <varname>archive_cleanup_command</> in typical single-standby
***************
*** 706,712 **** protocol to make nodes agree on a serializable transactional order.
</para>
<para>
! A simple example of a <filename>recovery.conf</> is:
<programlisting>
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
--- 706,712 ----
</para>
<para>
! A simple example of standby settings is:
<programlisting>
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
***************
*** 762,769 **** archive_cleanup_command = 'pg_archivecleanup /path/to/archive %r'
To use streaming replication, set up a file-based log-shipping standby
server as described in <xref linkend="warm-standby">. The step that
turns a file-based log-shipping standby into streaming replication
! standby is setting <varname>primary_conninfo</> setting in the
! <filename>recovery.conf</> file to point to the primary server. Set
<xref linkend="guc-listen-addresses"> and authentication options
(see <filename>pg_hba.conf</>) on the primary so that the standby server
can connect to the <literal>replication</> pseudo-database on the primary
--- 762,769 ----
To use streaming replication, set up a file-based log-shipping standby
server as described in <xref linkend="warm-standby">. The step that
turns a file-based log-shipping standby into streaming replication
! standby is setting <varname>primary_conninfo</> to
! point to the primary server. Set
<xref linkend="guc-listen-addresses"> and authentication options
(see <filename>pg_hba.conf</>) on the primary so that the standby server
can connect to the <literal>replication</> pseudo-database on the primary
***************
*** 832,846 **** host replication foo 192.168.1.100/32 md5
</para>
<para>
The host name and port number of the primary, connection user name,
! and password are specified in the <filename>recovery.conf</> file.
The password can also be set in the <filename>~/.pgpass</> file on the
standby (specify <literal>replication</> in the <replaceable>database</>
field).
For example, if the primary is running on host IP <literal>192.168.1.50</>,
port <literal>5432</literal>, the account name for replication is
<literal>foo</>, and the password is <literal>foopass</>, the administrator
! can add the following line to the <filename>recovery.conf</> file on the
! standby:
<programlisting>
# The standby connects to the primary that is running on host 192.168.1.50
--- 832,845 ----
</para>
<para>
The host name and port number of the primary, connection user name,
! and password are specified in <varname>primary_conninfo</>.
The password can also be set in the <filename>~/.pgpass</> file on the
standby (specify <literal>replication</> in the <replaceable>database</>
field).
For example, if the primary is running on host IP <literal>192.168.1.50</>,
port <literal>5432</literal>, the account name for replication is
<literal>foo</>, and the password is <literal>foopass</>, the administrator
! can set <varname>primary_conninfo</> on the standby like this:
<programlisting>
# The standby connects to the primary that is running on host 192.168.1.50
***************
*** 1204,1211 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<para>
To trigger failover of a log-shipping standby server,
run <command>pg_ctl promote</> or create a trigger
! file with the file name and path specified by the <varname>trigger_file</>
! setting in <filename>recovery.conf</>. If you're planning to use
<command>pg_ctl promote</> to fail over, <varname>trigger_file</> is
not required. If you're setting up the reporting servers that are
only used to offload read-only queries from the primary, not for high
--- 1203,1210 ----
<para>
To trigger failover of a log-shipping standby server,
run <command>pg_ctl promote</> or create a trigger
! file with the file name and path specified by the <varname>trigger_file</>.
! If you're planning to use
<command>pg_ctl promote</> to fail over, <varname>trigger_file</> is
not required. If you're setting up the reporting servers that are
only used to offload read-only queries from the primary, not for high
***************
*** 1250,1257 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
The magic that makes the two loosely coupled servers work together is
simply a <varname>restore_command</> used on the standby that,
when asked for the next WAL file, waits for it to become available from
! the primary. The <varname>restore_command</> is specified in the
! <filename>recovery.conf</> file on the standby server. Normal recovery
processing would request a file from the WAL archive, reporting failure
if the file was unavailable. For standby processing it is normal for
the next WAL file to be unavailable, so the standby must wait for
--- 1249,1255 ----
The magic that makes the two loosely coupled servers work together is
simply a <varname>restore_command</> used on the standby that,
when asked for the next WAL file, waits for it to become available from
! the primary. Normal recovery
processing would request a file from the WAL archive, reporting failure
if the file was unavailable. For standby processing it is normal for
the next WAL file to be unavailable, so the standby must wait for
***************
*** 1338,1345 **** if (!triggered)
</listitem>
<listitem>
<para>
Begin recovery on the standby server from the local WAL
! archive, using a <filename>recovery.conf</> that specifies a
<varname>restore_command</> that waits as described
previously (see <xref linkend="backup-pitr-recovery">).
</para>
--- 1336,1349 ----
</listitem>
<listitem>
<para>
+ Create a recovery trigger file <filename>recovery.trigger</>
+ in the standby's cluster data directory.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
Begin recovery on the standby server from the local WAL
! archive, specifying a
<varname>restore_command</> that waits as described
previously (see <xref linkend="backup-pitr-recovery">).
</para>
***************
*** 1830,1838 **** if (!triggered)
<title>Administrator's Overview</title>
<para>
! If <varname>hot_standby</> is turned <literal>on</> in
! <filename>postgresql.conf</> and there is a <filename>recovery.conf</>
! file present, the server will run in Hot Standby mode.
However, it may take some time for Hot Standby connections to be allowed,
because the server will not accept connections until it has completed
sufficient recovery to provide a consistent state against which queries
--- 1834,1842 ----
<title>Administrator's Overview</title>
<para>
! If <varname>hot_standby</> is turned <literal>on</>
! and there is a recovery trigger file
! <filename>recovery.trigger</> present, the server will run in Hot Standby mode.
However, it may take some time for Hot Standby connections to be allowed,
because the server will not accept connections until it has completed
sufficient recovery to provide a consistent state against which queries
*** a/doc/src/sgml/pgarchivecleanup.sgml
--- b/doc/src/sgml/pgarchivecleanup.sgml
***************
*** 37,44 ****
<para>
To configure a standby
! server to use <application>pg_archivecleanup</>, put this into its
! <filename>recovery.conf</filename> configuration file:
<programlisting>
archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
</programlisting>
--- 37,44 ----
<para>
To configure a standby
! server to use <application>pg_archivecleanup</>, specify
! <xref linkend="guc-archive-cleanup-command"> like this:
<programlisting>
archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
</programlisting>
***************
*** 46,52 **** archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
files should be removed.
</para>
<para>
! When used within <xref linkend="archive-cleanup-command">, all WAL files
logically preceding the value of the <literal>%r</> argument will be removed
from <replaceable>archivelocation</>. This minimizes the number of files
that need to be retained, while preserving crash-restart capability. Use of
--- 46,52 ----
files should be removed.
</para>
<para>
! When used within <varname>archive_cleanup_command</>, all WAL files
logically preceding the value of the <literal>%r</> argument will be removed
from <replaceable>archivelocation</>. This minimizes the number of files
that need to be retained, while preserving crash-restart capability. Use of
*** a/doc/src/sgml/pgstandby.sgml
--- b/doc/src/sgml/pgstandby.sgml
***************
*** 48,55 ****
<para>
To configure a standby
! server to use <application>pg_standby</>, put this into its
! <filename>recovery.conf</filename> configuration file:
<programlisting>
restore_command = 'pg_standby <replaceable>archiveDir</> %f %p %r'
</programlisting>
--- 48,55 ----
<para>
To configure a standby
! server to use <application>pg_standby</>, specify
! <xref linkend="guc-restore-command"> like this:
<programlisting>
restore_command = 'pg_standby <replaceable>archiveDir</> %f %p %r'
</programlisting>
*** a/doc/src/sgml/postgres.sgml
--- b/doc/src/sgml/postgres.sgml
***************
*** 155,161 ****
&maintenance;
&backup;
&high-availability;
- &recovery-config;
&monitoring;
&diskusage;
&wal;
--- 155,160 ----
*** a/doc/src/sgml/recovery-config.sgml
--- /dev/null
***************
*** 1,363 ****
- <!-- doc/src/sgml/recovery-config.sgml -->
-
- <chapter id="recovery-config">
- <title>Recovery Configuration</title>
-
- <indexterm>
- <primary>configuration</primary>
- <secondary>of recovery</secondary>
- <tertiary>of a standby server</tertiary>
- </indexterm>
-
- <para>
- This chapter describes the settings available in the
- <filename>recovery.conf</><indexterm><primary>recovery.conf</></>
- file. They apply only for the duration of the
- recovery. They must be reset for any subsequent recovery you wish to
- perform. They cannot be changed once recovery has begun.
- </para>
-
- <para>
- Settings in <filename>recovery.conf</> are specified in the format
- <literal>name = 'value'</>. One parameter is specified per line.
- Hash marks (<literal>#</literal>) designate the rest of the
- line as a comment. To embed a single quote in a parameter
- value, write two quotes (<literal>''</>).
- </para>
-
- <para>
- A sample file, <filename>share/recovery.conf.sample</>,
- is provided in the installation's <filename>share/</> directory.
- </para>
-
- <sect1 id="archive-recovery-settings">
-
- <title>Archive Recovery Settings</title>
- <variablelist>
-
- <varlistentry id="restore-command" xreflabel="restore_command">
- <term><varname>restore_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>restore_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- The shell command to execute to retrieve an archived segment of
- the WAL file series. This parameter is required for archive recovery,
- but optional for streaming replication.
- Any <literal>%f</> in the string is
- replaced by the name of the file to retrieve from the archive,
- and any <literal>%p</> is replaced by the copy destination path name
- on the server.
- (The path name is relative to the current working directory,
- i.e., the cluster's data directory.)
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point. That is the earliest file that must be kept
- to allow a restore to be restartable, so this information can be used
- to truncate the archive to just the minimum required to support
- restarting from the current restore. <literal>%r</> is typically only
- used by warm-standby configurations
- (see <xref linkend="warm-standby">).
- Write <literal>%%</> to embed an actual <literal>%</> character.
- </para>
-
- <para>
- It is important for the command to return a zero exit status
- only if it succeeds. The command <emphasis>will</> be asked for file
- names that are not present in the archive; it must return nonzero
- when so asked. Examples:
- <programlisting>
- restore_command = 'cp /mnt/server/archivedir/%f "%p"'
- restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
- </programlisting>
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="archive-cleanup-command" xreflabel="archive_cleanup_command">
- <term><varname>archive_cleanup_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>archive_cleanup_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This optional parameter specifies a shell command that will be executed
- at every restartpoint. The purpose of
- <varname>archive_cleanup_command</> is to provide a mechanism for
- cleaning up old archived WAL files that are no longer needed by the
- standby server.
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point.
- That is the earliest file that must be <emphasis>kept</> to allow a
- restore to be restartable, and so all files earlier than <literal>%r</>
- may be safely removed.
- This information can be used to truncate the archive to just the
- minimum required to support restart from the current restore.
- The <xref linkend="pgarchivecleanup"> module
- is often used in <varname>archive_cleanup_command</> for
- single-standby configurations, for example:
- <programlisting>archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'</programlisting>
- Note however that if multiple standby servers are restoring from the
- same archive directory, you will need to ensure that you do not delete
- WAL files until they are no longer needed by any of the servers.
- <varname>archive_cleanup_command</> would typically be used in a
- warm-standby configuration (see <xref linkend="warm-standby">).
- Write <literal>%%</> to embed an actual <literal>%</> character in the
- command.
- </para>
- <para>
- If the command returns a non-zero exit status then a WARNING log
- message will be written.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-end-command" xreflabel="recovery_end_command">
- <term><varname>recovery_end_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>recovery_end_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies a shell command that will be executed once only
- at the end of recovery. This parameter is optional. The purpose of the
- <varname>recovery_end_command</> is to provide a mechanism for cleanup
- following replication or recovery.
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point, like in <xref linkend="archive-cleanup-command">.
- </para>
- <para>
- If the command returns a non-zero exit status then a WARNING log
- message will be written and the database will proceed to start up
- anyway. An exception is that if the command was terminated by a
- signal, the database will not proceed with startup.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
-
- </sect1>
-
- <sect1 id="recovery-target-settings">
-
- <title>Recovery Target Settings</title>
- <variablelist>
-
- <varlistentry id="recovery-target-name" xreflabel="recovery_target_name">
- <term><varname>recovery_target_name</varname>
- (<type>string</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_name</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the named restore point, created with
- <function>pg_create_restore_point()</> to which recovery will proceed.
- At most one of <varname>recovery_target_name</>,
- <xref linkend="recovery-target-time"> or
- <xref linkend="recovery-target-xid"> can be specified. The default is to
- recover to the end of the WAL log.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-time" xreflabel="recovery_target_time">
- <term><varname>recovery_target_time</varname>
- (<type>timestamp</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_time</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the time stamp up to which recovery
- will proceed.
- At most one of <varname>recovery_target_time</>,
- <xref linkend="recovery-target-name"> or
- <xref linkend="recovery-target-xid"> can be specified.
- The default is to recover to the end of the WAL log.
- The precise stopping point is also influenced by
- <xref linkend="recovery-target-inclusive">.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-xid" xreflabel="recovery_target_xid">
- <term><varname>recovery_target_xid</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>recovery_target_xid</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the transaction ID up to which recovery
- will proceed. Keep in mind
- that while transaction IDs are assigned sequentially at transaction
- start, transactions can complete in a different numeric order.
- The transactions that will be recovered are those that committed
- before (and optionally including) the specified one.
- At most one of <varname>recovery_target_xid</>,
- <xref linkend="recovery-target-name"> or
- <xref linkend="recovery-target-time"> can be specified.
- The default is to recover to the end of the WAL log.
- The precise stopping point is also influenced by
- <xref linkend="recovery-target-inclusive">.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-inclusive"
- xreflabel="recovery_target_inclusive">
- <term><varname>recovery_target_inclusive</varname>
- (<type>boolean</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_inclusive</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether we stop just after the specified recovery target
- (<literal>true</literal>), or just before the recovery target
- (<literal>false</literal>).
- Applies to both <xref linkend="recovery-target-time">
- and <xref linkend="recovery-target-xid">, whichever one is
- specified for this recovery. This indicates whether transactions
- having exactly the target commit time or ID, respectively, will
- be included in the recovery. Default is <literal>true</>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-timeline"
- xreflabel="recovery_target_timeline">
- <term><varname>recovery_target_timeline</varname>
- (<type>string</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_timeline</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies recovering into a particular timeline. The default is
- to recover along the same timeline that was current when the
- base backup was taken. Setting this to <literal>latest</> recovers
- to the latest timeline found in the archive, which is useful in
- a standby server. Other than that you only need to set this parameter
- in complex re-recovery situations, where you need to return to
- a state that itself was reached after a point-in-time recovery.
- See <xref linkend="backup-timelines"> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="pause-at-recovery-target"
- xreflabel="pause_at_recovery_target">
- <term><varname>pause_at_recovery_target</varname>
- (<type>boolean</type>)
- </term>
- <indexterm>
- <primary><varname>pause_at_recovery_target</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether recovery should pause when the recovery target
- is reached. The default is true.
- This is intended to allow queries to be executed against the
- database to check if this recovery target is the most desirable
- point for recovery. The paused state can be resumed by using
- <function>pg_xlog_replay_resume()</> (See
- <xref linkend="functions-recovery-control-table">), which then
- causes recovery to end. If this recovery target is not the
- desired stopping point, then shutdown the server, change the
- recovery target settings to a later target and restart to
- continue recovery.
- </para>
- <para>
- This setting has no effect if <xref linkend="guc-hot-standby"> is not
- enabled, or if no recovery target is set.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect1>
-
- <sect1 id="standby-settings">
-
- <title>Standby Server Settings</title>
- <variablelist>
-
- <varlistentry id="standby-mode" xreflabel="standby_mode">
- <term><varname>standby_mode</varname> (<type>boolean</type>)</term>
- <indexterm>
- <primary><varname>standby_mode</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether to start the <productname>PostgreSQL</> server as
- a standby. If this parameter is <literal>on</>, the server will
- not stop recovery when the end of archived WAL is reached, but
- will keep trying to continue recovery by fetching new WAL segments
- using <varname>restore_command</>
- and/or by connecting to the primary server as specified by the
- <varname>primary_conninfo</> setting.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry id="primary-conninfo" xreflabel="primary_conninfo">
- <term><varname>primary_conninfo</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>primary_conninfo</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies a connection string to be used for the standby server
- to connect with the primary. This string is in the format
- accepted by the libpq <function>PQconnectdb</function> function,
- described in <xref linkend="libpq-connect">. If any option is
- unspecified in this string, then the corresponding environment
- variable (see <xref linkend="libpq-envars">) is checked. If the
- environment variable is not set either, then
- defaults are used.
- </para>
- <para>
- The connection string should specify the host name (or address)
- of the primary server, as well as the port number if it is not
- the same as the standby server's default.
- Also specify a user name corresponding to a role that has the
- <literal>REPLICATION</> and <literal>LOGIN</> privileges on the
- primary (see
- <xref linkend="streaming-replication-authentication">).
- A password needs to be provided too, if the primary demands password
- authentication. It can be provided in the
- <varname>primary_conninfo</varname> string, or in a separate
- <filename>~/.pgpass</> file on the standby server (use
- <literal>replication</> as the database name).
- Do not specify a database name in the
- <varname>primary_conninfo</varname> string.
- </para>
- <para>
- This setting has no effect if <varname>standby_mode</> is <literal>off</>.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry id="trigger-file" xreflabel="trigger_file">
- <term><varname>trigger_file</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>trigger_file</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies a trigger file whose presence ends recovery in the
- standby. Even if this value is not set, you can still promote
- the standby using <command>pg_ctl promote</>.
- This setting has no effect if <varname>standby_mode</> is <literal>off</>.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect1>
-
- </chapter>
--- 0 ----
*** a/doc/src/sgml/release-9.1.sgml
--- b/doc/src/sgml/release-9.1.sgml
***************
*** 1078,1084 ****
<listitem>
<para>
Add <filename>recovery.conf</> setting <link
! linkend="pause-at-recovery-target"><varname>pause_at_recovery_target</></link>
to pause recovery at target (Simon Riggs)
</para>
--- 1078,1084 ----
<listitem>
<para>
Add <filename>recovery.conf</> setting <link
! linkend="guc-pause-at-recovery-target"><varname>pause_at_recovery_target</></link>
to pause recovery at target (Simon Riggs)
</para>
***************
*** 1098,1104 ****
<para>
These named restore points can be specified as recovery
targets using the new <filename>recovery.conf</> setting
! <link linkend="recovery-target-name"><varname>recovery_target_name</></link>.
</para>
</listitem>
--- 1098,1104 ----
<para>
These named restore points can be specified as recovery
targets using the new <filename>recovery.conf</> setting
! <link linkend="guc-recovery-target-name"><varname>recovery_target_name</></link>.
</para>
</listitem>
***************
*** 1130,1137 ****
<listitem>
<para>
! Allow <link
! linkend="recovery-config"><filename>recovery.conf</></link>
to use the same quoting behavior as <filename>postgresql.conf</>
(Dimitri Fontaine)
</para>
--- 1130,1136 ----
<listitem>
<para>
! Allow <filename>recovery.conf</>
to use the same quoting behavior as <filename>postgresql.conf</>
(Dimitri Fontaine)
</para>
*** a/doc/src/sgml/release.sgml
--- b/doc/src/sgml/release.sgml
***************
*** 5,12 **** Typical markup:
&<> use & escapes
PostgreSQL <productname>
! postgresql.conf, pg_hba.conf,
! recovery.conf <filename>
[A-Z][A-Z_ ]+[A-Z_] <command>, <literal>, <envar>
[A-Za-z_][A-Za-z0-9_]+() <function>
-[-A-Za-z_]+ <option>
--- 5,12 ----
&<> use & escapes
PostgreSQL <productname>
! postgresql.conf, pg_hba.conf
! recovery.trigger <filename>
[A-Z][A-Z_ ]+[A-Z_] <command>, <literal>, <envar>
[A-Za-z_][A-Za-z0-9_]+() <function>
-[-A-Za-z_]+ <option>
*** a/src/backend/access/transam/recovery.conf.sample
--- b/src/backend/access/transam/recovery.conf.sample
***************
*** 2,24 ****
# PostgreSQL recovery config file
# -------------------------------
#
! # Edit this file to provide the parameters that PostgreSQL needs to
! # perform an archive recovery of a database, or to act as a replication
! # standby.
#
! # If "recovery.conf" is present in the PostgreSQL data directory, it is
! # read on postmaster startup. After successful recovery, it is renamed
! # to "recovery.done" to ensure that we do not accidentally re-enter
! # archive recovery or standby mode.
#
! # This file consists of lines of the form:
! #
! # name = value
! #
! # Comments are introduced with '#'.
! #
! # The complete list of option names and allowed values can be found
! # in the PostgreSQL documentation.
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
--- 2,15 ----
# PostgreSQL recovery config file
# -------------------------------
#
! # PostgreSQL 9.2 or later, recovery.conf is no longer used. Instead,
! # specify all recovery parameters in postgresql.conf and create
! # recovery.trigger to enter archive recovery or standby mode.
#
! # If you must use recovery.conf, specify "include directives" in
! # postgresql.conf like this:
#
! # include 'recovery.conf'
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 63,69 ****
/* File path names (all relative to $PGDATA) */
! #define RECOVERY_COMMAND_FILE "recovery.conf"
#define RECOVERY_COMMAND_DONE "recovery.done"
#define PROMOTE_SIGNAL_FILE "promote"
--- 63,69 ----
/* File path names (all relative to $PGDATA) */
! #define RECOVERY_COMMAND_READY "recovery.trigger"
#define RECOVERY_COMMAND_DONE "recovery.done"
#define PROMOTE_SIGNAL_FILE "promote"
***************
*** 80,85 **** bool fullPageWrites = true;
--- 80,99 ----
bool log_checkpoints = false;
int sync_method = DEFAULT_SYNC_METHOD;
int wal_level = WAL_LEVEL_MINIMAL;
+ char *restore_command = NULL;
+ char *archive_cleanup_command = NULL;
+ char *recovery_end_command = NULL;
+ bool standby_mode = false;
+ char *primary_conninfo = NULL;
+ char *trigger_file = NULL;
+ RecoveryTargetType recovery_target = RECOVERY_TARGET_UNSET;
+ TransactionId recovery_target_xid = InvalidTransactionId;
+ TimestampTz recovery_target_time = 0;
+ char *recovery_target_name = NULL;
+ bool recovery_target_inclusive = true;
+ bool pause_at_recovery_target = true;
+ char *recovery_target_timeline_string = NULL;
+ TimeLineID recovery_target_timeline = 0;
#ifdef WAL_DEBUG
bool XLOG_DEBUG = false;
***************
*** 186,208 **** static bool InArchiveRecovery = false;
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* options taken from recovery.conf for archive recovery */
! static char *recoveryRestoreCommand = NULL;
! static char *recoveryEndCommand = NULL;
! static char *archiveCleanupCommand = NULL;
! static RecoveryTargetType recoveryTarget = RECOVERY_TARGET_UNSET;
! static bool recoveryTargetInclusive = true;
! static bool recoveryPauseAtTarget = true;
! static TransactionId recoveryTargetXid;
! static TimestampTz recoveryTargetTime;
! static char *recoveryTargetName;
!
! /* options taken from recovery.conf for XLOG streaming */
! static bool StandbyMode = false;
! static char *PrimaryConnInfo = NULL;
! static char *TriggerFile = NULL;
!
! /* if recoveryStopsHere returns true, it saves actual stop xid/time/name here */
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
--- 200,207 ----
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* if recoveryStopsHere returns true, it saves actual stop type/xid/time/name here */
! static RecoveryTargetType recoveryStopTarget;
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
***************
*** 408,419 **** typedef struct XLogCtlData
TimeLineID RecoveryTargetTLI;
/*
- * archiveCleanupCommand is read from recovery.conf but needs to be in
- * shared memory so that the bgwriter process can access it.
- */
- char archiveCleanupCommand[MAXPGPATH];
-
- /*
* SharedRecoveryInProgress indicates if we're still in crash or archive
* recovery. Protected by info_lck.
*/
--- 407,412 ----
***************
*** 602,608 **** static void XLogArchiveNotifySeg(uint32 log, uint32 seg);
static bool XLogArchiveCheckDone(const char *xlog);
static bool XLogArchiveIsBusy(const char *xlog);
static void XLogArchiveCleanup(const char *xlog);
! static void readRecoveryCommandFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
--- 595,601 ----
static bool XLogArchiveCheckDone(const char *xlog);
static bool XLogArchiveIsBusy(const char *xlog);
static void XLogArchiveCleanup(const char *xlog);
! static void CheckRecoveryReadyFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
***************
*** 612,617 **** static void SetRecoveryPause(bool recoveryPause);
--- 605,611 ----
static void SetLatestXTime(TimestampTz xtime);
static TimestampTz GetLatestXTime(void);
static void CheckRequiredParameterValues(void);
+ static void CheckRestoreCommandSet(void);
static void XLogReportParameters(void);
static void LocalSetXLogInsertAllowed(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
***************
*** 2932,2938 **** RestoreArchivedFile(char *path, const char *xlogfname,
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (recoveryRestoreCommand == NULL)
goto not_available;
/*
--- 2926,2932 ----
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (!restore_command[0])
goto not_available;
/*
***************
*** 2952,2958 **** RestoreArchivedFile(char *path, const char *xlogfname,
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the recoveryRestoreCommand may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
--- 2946,2952 ----
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the restore_command may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
***************
*** 3016,3022 **** RestoreArchivedFile(char *path, const char *xlogfname,
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = recoveryRestoreCommand; *sp; sp++)
{
if (*sp == '%')
{
--- 3010,3016 ----
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = restore_command; *sp; sp++)
{
if (*sp == '%')
{
***************
*** 3107,3113 **** RestoreArchivedFile(char *path, const char *xlogfname,
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (StandbyMode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
--- 3101,3107 ----
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (standby_mode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
***************
*** 3283,3289 **** ExecuteRecoveryCommand(char *command, char *commandName, bool failOnSignal)
ereport((signaled && failOnSignal) ? FATAL : WARNING,
/*------
! translator: First %s represents a recovery.conf parameter name like
"recovery_end_command", and the 2nd is the value of that parameter. */
(errmsg("%s \"%s\": return code %d", commandName,
command, rc)));
--- 3277,3283 ----
ereport((signaled && failOnSignal) ? FATAL : WARNING,
/*------
! translator: First %s represents a recovery parameter name like
"recovery_end_command", and the 2nd is the value of that parameter. */
(errmsg("%s \"%s\": return code %d", commandName,
command, rc)));
***************
*** 4064,4070 **** next_record_is_invalid:
}
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return NULL;
--- 4058,4064 ----
}
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return NULL;
***************
*** 4523,4529 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
--- 4517,4523 ----
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryStopTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4531,4537 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
--- 4525,4531 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryStopTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4539,4545 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
--- 4533,4539 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryStopTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
***************
*** 5268,5498 **** str_time(pg_time_t tnow)
}
/*
! * See if there is a recovery command file (recovery.conf), and if so
! * read in parameters for archive recovery and XLOG streaming.
! *
! * The file is parsed using the main configuration parser.
*/
static void
! readRecoveryCommandFile(void)
{
! FILE *fd;
! TimeLineID rtli = 0;
! bool rtliGiven = false;
! ConfigVariable *item,
! *head = NULL,
! *tail = NULL;
!
! fd = AllocateFile(RECOVERY_COMMAND_FILE, "r");
! if (fd == NULL)
! {
! if (errno == ENOENT)
! return; /* not there, so no archive recovery */
! ereport(FATAL,
! (errcode_for_file_access(),
! errmsg("could not open recovery command file \"%s\": %m",
! RECOVERY_COMMAND_FILE)));
! }
!
! /*
! * Since we're asking ParseConfigFp() to error out at FATAL, there's no
! * need to check the return value.
! */
! ParseConfigFp(fd, RECOVERY_COMMAND_FILE, 0, FATAL, &head, &tail);
!
! for (item = head; item; item = item->next)
! {
! if (strcmp(item->name, "restore_command") == 0)
! {
! recoveryRestoreCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("restore_command = '%s'",
! recoveryRestoreCommand)));
! }
! else if (strcmp(item->name, "recovery_end_command") == 0)
! {
! recoveryEndCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("recovery_end_command = '%s'",
! recoveryEndCommand)));
! }
! else if (strcmp(item->name, "archive_cleanup_command") == 0)
! {
! archiveCleanupCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("archive_cleanup_command = '%s'",
! archiveCleanupCommand)));
! }
! else if (strcmp(item->name, "pause_at_recovery_target") == 0)
! {
! if (!parse_bool(item->value, &recoveryPauseAtTarget))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value", "pause_at_recovery_target")));
! ereport(DEBUG2,
! (errmsg_internal("pause_at_recovery_target = '%s'",
! item->value)));
! }
! else if (strcmp(item->name, "recovery_target_timeline") == 0)
! {
! rtliGiven = true;
! if (strcmp(item->value, "latest") == 0)
! rtli = 0;
! else
! {
! errno = 0;
! rtli = (TimeLineID) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_timeline is not a valid number: \"%s\"",
! item->value)));
! }
! if (rtli)
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = %u", rtli)));
! else
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = latest")));
! }
! else if (strcmp(item->name, "recovery_target_xid") == 0)
! {
! errno = 0;
! recoveryTargetXid = (TransactionId) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_xid is not a valid number: \"%s\"",
! item->value)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_xid = %u",
! recoveryTargetXid)));
! recoveryTarget = RECOVERY_TARGET_XID;
! }
! else if (strcmp(item->name, "recovery_target_time") == 0)
! {
! /*
! * if recovery_target_xid or recovery_target_name specified, then
! * this overrides recovery_target_time
! */
! if (recoveryTarget == RECOVERY_TARGET_XID ||
! recoveryTarget == RECOVERY_TARGET_NAME)
! continue;
! recoveryTarget = RECOVERY_TARGET_TIME;
!
! /*
! * Convert the time string given by the user to TimestampTz form.
! */
! recoveryTargetTime =
! DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
! CStringGetDatum(item->value),
! ObjectIdGetDatum(InvalidOid),
! Int32GetDatum(-1)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_time = '%s'",
! timestamptz_to_str(recoveryTargetTime))));
! }
! else if (strcmp(item->name, "recovery_target_name") == 0)
! {
! /*
! * if recovery_target_xid specified, then this overrides
! * recovery_target_name
! */
! if (recoveryTarget == RECOVERY_TARGET_XID)
! continue;
! recoveryTarget = RECOVERY_TARGET_NAME;
!
! recoveryTargetName = pstrdup(item->value);
! if (strlen(recoveryTargetName) >= MAXFNAMELEN)
! ereport(FATAL,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("recovery_target_name is too long (maximum %d characters)",
! MAXFNAMELEN - 1)));
!
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_name = '%s'",
! recoveryTargetName)));
! }
! else if (strcmp(item->name, "recovery_target_inclusive") == 0)
! {
! /*
! * does nothing if a recovery_target is not also set
! */
! if (!parse_bool(item->value, &recoveryTargetInclusive))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "recovery_target_inclusive")));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_inclusive = %s",
! item->value)));
! }
! else if (strcmp(item->name, "standby_mode") == 0)
! {
! if (!parse_bool(item->value, &StandbyMode))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "standby_mode")));
! ereport(DEBUG2,
! (errmsg_internal("standby_mode = '%s'", item->value)));
! }
! else if (strcmp(item->name, "primary_conninfo") == 0)
! {
! PrimaryConnInfo = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("primary_conninfo = '%s'",
! PrimaryConnInfo)));
! }
! else if (strcmp(item->name, "trigger_file") == 0)
! {
! TriggerFile = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("trigger_file = '%s'",
! TriggerFile)));
! }
! else
! ereport(FATAL,
! (errmsg("unrecognized recovery parameter \"%s\"",
! item->name)));
! }
! /*
! * Check for compulsory parameters
! */
! if (StandbyMode)
! {
! if (PrimaryConnInfo == NULL && recoveryRestoreCommand == NULL)
! ereport(WARNING,
! (errmsg("recovery command file \"%s\" specified neither primary_conninfo nor restore_command",
! RECOVERY_COMMAND_FILE),
! errhint("The database server will regularly poll the pg_xlog subdirectory to check for files placed there.")));
! }
! else
! {
! if (recoveryRestoreCommand == NULL)
! ereport(FATAL,
! (errmsg("recovery command file \"%s\" must specify restore_command when standby mode is not enabled",
! RECOVERY_COMMAND_FILE)));
! }
/* Enable fetching from archive recovery area */
InArchiveRecovery = true;
/*
* If user specified recovery_target_timeline, validate it or compute the
* "latest" value. We can't do this until after we've gotten the restore
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (rtliGiven)
{
! if (rtli)
{
/* Timeline 1 does not have a history file, all else should */
! if (rtli != 1 && !existsTimeLineHistory(rtli))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! rtli)));
! recoveryTargetTLI = rtli;
recoveryTargetIsLatest = false;
}
else
--- 5262,5305 ----
}
/*
! * Check to see if there is a recovery trigger file (recovery.trigger), and if so
! * validate recovery parameters and determine recovery target timeline
*/
static void
! CheckRecoveryReadyFile(void)
{
! struct stat stat_buf;
! if (stat(RECOVERY_COMMAND_READY, &stat_buf) != 0)
! return; /* not there, so no archive recovery */
/* Enable fetching from archive recovery area */
InArchiveRecovery = true;
+ /* Check for compulsory parameters */
+ if (standby_mode && !restore_command[0] && !primary_conninfo[0])
+ ereport(WARNING,
+ (errmsg("neither primary_conninfo nor restore_command is specified"),
+ errhint("The database server will regularly poll the pg_xlog subdirectory to "
+ "check for files placed there until either of them is set in postgresql.conf.")));
+ CheckRestoreCommandSet();
+
/*
* If user specified recovery_target_timeline, validate it or compute the
* "latest" value. We can't do this until after we've gotten the restore
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (strcmp(recovery_target_timeline_string, "") != 0)
{
! if (recovery_target_timeline)
{
/* Timeline 1 does not have a history file, all else should */
! if (recovery_target_timeline != 1 && !existsTimeLineHistory(recovery_target_timeline))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! recovery_target_timeline)));
! recoveryTargetTLI = recovery_target_timeline;
recoveryTargetIsLatest = false;
}
else
***************
*** 5502,5510 **** readRecoveryCommandFile(void)
recoveryTargetIsLatest = true;
}
}
-
- FreeConfigVariables(head);
- FreeFile(fd);
}
/*
--- 5309,5314 ----
***************
*** 5580,5590 **** exitArchiveRecovery(TimeLineID endTLI, uint32 endLogId, uint32 endLogSeg)
* re-enter archive recovery mode in a subsequent crash.
*/
unlink(RECOVERY_COMMAND_DONE);
! if (rename(RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE) != 0)
ereport(FATAL,
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
! RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
--- 5384,5394 ----
* re-enter archive recovery mode in a subsequent crash.
*/
unlink(RECOVERY_COMMAND_DONE);
! if (rename(RECOVERY_COMMAND_READY, RECOVERY_COMMAND_DONE) != 0)
ereport(FATAL,
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
! RECOVERY_COMMAND_READY, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
***************
*** 5647,5653 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
/* Do we have a PITR target at all? */
! if (recoveryTarget == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
--- 5451,5457 ----
return false;
/* Do we have a PITR target at all? */
! if (recovery_target == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
***************
*** 5658,5664 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
}
! if (recoveryTarget == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
--- 5462,5468 ----
return false;
}
! if (recovery_target == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
***************
*** 5669,5688 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recoveryTargetXid);
if (stopsHere)
! *includeThis = recoveryTargetInclusive;
}
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recoveryTargetName) == 0);
/*
! * Ignore recoveryTargetInclusive because this is not a transaction
* record
*/
*includeThis = false;
--- 5473,5492 ----
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recovery_target_xid);
if (stopsHere)
! *includeThis = recovery_target_inclusive;
}
! else if (recovery_target == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recovery_target_name) == 0);
/*
! * Ignore recovery_target_inclusive because this is not a transaction
* record
*/
*includeThis = false;
***************
*** 5694,5709 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recoveryTargetInclusive)
! stopsHere = (recordXtime > recoveryTargetTime);
else
! stopsHere = (recordXtime >= recoveryTargetTime);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
--- 5498,5514 ----
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recovery_target_inclusive)
! stopsHere = (recordXtime > recovery_target_time);
else
! stopsHere = (recordXtime >= recovery_target_time);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
+ recoveryStopTarget = recovery_target;
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
***************
*** 6004,6009 **** CheckRequiredParameterValues(void)
--- 5809,5827 ----
}
/*
+ * Check to see if restore_command must be set for archive recovery
+ * when standby mode is not enabled
+ */
+ static void
+ CheckRestoreCommandSet(void)
+ {
+ if (InArchiveRecovery && !standby_mode && !restore_command[0])
+ ereport(FATAL,
+ (errmsg("restore_command must be specified for archive recovery "
+ "when standby mode is not enabled")));
+ }
+
+ /*
* This must be called ONCE during postmaster or standalone-backend startup
*/
void
***************
*** 6097,6106 **** StartupXLOG(void)
recoveryTargetTLI = ControlFile->checkPointCopy.ThisTimeLineID;
/*
! * Check for recovery control file, and if so set up state for offline
* recovery
*/
! readRecoveryCommandFile();
/* Now we can determine the list of expected TLIs */
expectedTLIs = readTimeLineHistory(recoveryTargetTLI);
--- 5915,5924 ----
recoveryTargetTLI = ControlFile->checkPointCopy.ThisTimeLineID;
/*
! * Check for recovery trigger file, and if so set up state for offline
* recovery
*/
! CheckRecoveryReadyFile();
/* Now we can determine the list of expected TLIs */
expectedTLIs = readTimeLineHistory(recoveryTargetTLI);
***************
*** 6118,6149 **** StartupXLOG(void)
ControlFile->checkPointCopy.ThisTimeLineID)));
/*
! * Save the selected recovery target timeline ID and
! * archive_cleanup_command in shared memory so that other processes can
! * see them
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
- strncpy(XLogCtl->archiveCleanupCommand,
- archiveCleanupCommand ? archiveCleanupCommand : "",
- sizeof(XLogCtl->archiveCleanupCommand));
if (InArchiveRecovery)
{
! if (StandbyMode)
ereport(LOG,
(errmsg("entering standby mode")));
- else if (recoveryTarget == RECOVERY_TARGET_XID)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to XID %u",
- recoveryTargetXid)));
- else if (recoveryTarget == RECOVERY_TARGET_TIME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to %s",
- timestamptz_to_str(recoveryTargetTime))));
- else if (recoveryTarget == RECOVERY_TARGET_NAME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to \"%s\"",
- recoveryTargetName)));
else
ereport(LOG,
(errmsg("starting archive recovery")));
--- 5936,5951 ----
ControlFile->checkPointCopy.ThisTimeLineID)));
/*
! * Save the selected recovery target timeline ID in shared memory
! * so that other processes can see it
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
if (InArchiveRecovery)
{
! if (standby_mode)
ereport(LOG,
(errmsg("entering standby mode")));
else
ereport(LOG,
(errmsg("starting archive recovery")));
***************
*** 6153,6159 **** StartupXLOG(void)
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (StandbyMode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
--- 5955,5961 ----
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (standby_mode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
***************
*** 6211,6217 **** StartupXLOG(void)
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (StandbyMode)
{
/*
* The last valid checkpoint record required for a streaming
--- 6013,6019 ----
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (standby_mode)
{
/*
* The last valid checkpoint record required for a streaming
***************
*** 6280,6286 **** StartupXLOG(void)
/*
* Check whether we need to force recovery from WAL. If it appears to
! * have been a clean shutdown and we did not have a recovery.conf file,
* then assume no recovery needed.
*/
if (XLByteLT(checkPoint.redo, RecPtr))
--- 6082,6088 ----
/*
* Check whether we need to force recovery from WAL. If it appears to
! * have been a clean shutdown and we did not have a recovery.trigger file,
* then assume no recovery needed.
*/
if (XLByteLT(checkPoint.redo, RecPtr))
***************
*** 6294,6300 **** StartupXLOG(void)
InRecovery = true;
else if (InArchiveRecovery)
{
! /* force recovery due to presence of recovery.conf */
InRecovery = true;
}
--- 6096,6102 ----
InRecovery = true;
else if (InArchiveRecovery)
{
! /* force recovery due to presence of recovery.trigger */
InRecovery = true;
}
***************
*** 6564,6570 **** StartupXLOG(void)
* Pause only if users can connect to send a resume
* message
*/
! if (recoveryPauseAtTarget && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
--- 6366,6372 ----
* Pause only if users can connect to send a resume
* message
*/
! if (pause_at_recovery_target && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
***************
*** 6663,6669 **** StartupXLOG(void)
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (StandbyMode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
--- 6465,6471 ----
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (standby_mode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
***************
*** 6671,6677 **** StartupXLOG(void)
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! StandbyMode = false;
/*
* Re-fetch the last valid or last applied record, so we can identify the
--- 6473,6479 ----
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! SetConfigOption("standby_mode", "false", PGC_POSTMASTER, PGC_S_OVERRIDE);
/*
* Re-fetch the last valid or last applied record, so we can identify the
***************
*** 6864,6871 **** StartupXLOG(void)
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recoveryEndCommand)
! ExecuteRecoveryCommand(recoveryEndCommand,
"recovery_end_command",
true);
}
--- 6666,6673 ----
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recovery_end_command[0])
! ExecuteRecoveryCommand(recovery_end_command,
"recovery_end_command",
true);
}
***************
*** 8188,8195 **** CreateRestartPoint(int flags)
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (XLogCtl->archiveCleanupCommand[0])
! ExecuteRecoveryCommand(XLogCtl->archiveCleanupCommand,
"archive_cleanup_command",
false);
--- 7990,7997 ----
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (archive_cleanup_command[0])
! ExecuteRecoveryCommand(archive_cleanup_command,
"archive_cleanup_command",
false);
***************
*** 10020,10025 **** HandleStartupProcInterrupts(void)
--- 9822,9830 ----
{
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /* Check for compulsory parameter */
+ CheckRestoreCommandSet();
}
/*
***************
*** 10144,10150 **** XLogPageRead(XLogRecPtr *RecPtr, int emode, bool fetching_ckpt,
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (StandbyMode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
--- 9949,9955 ----
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (standby_mode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
***************
*** 10166,10172 **** retry:
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (StandbyMode)
{
/*
* In standby mode, wait for the requested record to become
--- 9971,9977 ----
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (standby_mode)
{
/*
* In standby mode, wait for the requested record to become
***************
*** 10330,10340 **** retry:
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (PrimaryConnInfo)
{
RequestXLogStreaming(
! fetching_ckpt ? RedoStartLSN : *RecPtr,
! PrimaryConnInfo);
continue;
}
}
--- 10135,10144 ----
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (primary_conninfo[0])
{
RequestXLogStreaming(
! fetching_ckpt ? RedoStartLSN : *RecPtr);
continue;
}
}
***************
*** 10477,10483 **** next_record_is_invalid:
readSource = 0;
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return false;
--- 10281,10287 ----
readSource = 0;
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return false;
***************
*** 10549,10563 **** CheckForStandbyTrigger(void)
return true;
}
! if (TriggerFile == NULL)
return false;
! if (stat(TriggerFile, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", TriggerFile)));
ShutdownWalRcv();
! unlink(TriggerFile);
triggered = true;
return true;
}
--- 10353,10367 ----
return true;
}
! if (!trigger_file[0])
return false;
! if (stat(trigger_file, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", trigger_file)));
ShutdownWalRcv();
! unlink(trigger_file);
triggered = true;
return true;
}
*** a/src/backend/commands/extension.c
--- b/src/backend/commands/extension.c
***************
*** 9,15 ****
* dependent objects can be associated with it. An extension is created by
* populating the pg_extension catalog from a "control" file.
* The extension control file is parsed with the same parser we use for
! * postgresql.conf and recovery.conf. An extension also has an installation
* script file, containing SQL commands to create the extension's objects.
*
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
--- 9,15 ----
* dependent objects can be associated with it. An extension is created by
* populating the pg_extension catalog from a "control" file.
* The extension control file is parsed with the same parser we use for
! * postgresql.conf. An extension also has an installation
* script file, containing SQL commands to create the extension's objects.
*
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
*** a/src/backend/replication/walreceiver.c
--- b/src/backend/replication/walreceiver.c
***************
*** 168,174 **** DisableWalRcvImmediateExit(void)
void
WalReceiverMain(void)
{
- char conninfo[MAXCONNINFO];
XLogRecPtr startpoint;
/* use volatile pointer to prevent code rearrangement */
--- 168,173 ----
***************
*** 216,222 **** WalReceiverMain(void)
walrcv->walRcvState = WALRCV_RUNNING;
/* Fetch information required to start streaming */
- strlcpy(conninfo, (char *) walrcv->conninfo, MAXCONNINFO);
startpoint = walrcv->receiveStart;
SpinLockRelease(&walrcv->mutex);
--- 215,220 ----
***************
*** 272,278 **** WalReceiverMain(void)
/* Establish the connection to the primary for XLOG streaming */
EnableWalRcvImmediateExit();
! walrcv_connect(conninfo, startpoint);
DisableWalRcvImmediateExit();
/* Loop until end-of-streaming or error */
--- 270,276 ----
/* Establish the connection to the primary for XLOG streaming */
EnableWalRcvImmediateExit();
! walrcv_connect(primary_conninfo, startpoint);
DisableWalRcvImmediateExit();
/* Loop until end-of-streaming or error */
***************
*** 302,309 **** WalReceiverMain(void)
--- 300,320 ----
if (got_SIGHUP)
{
+ char *conninfo = pstrdup(primary_conninfo);
+
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /*
+ * If primary_conninfo has been changed while walreceiver is running,
+ * shut down walreceiver so that new walreceiver is started and
+ * it initiates replication with new primary_conninfo.
+ */
+ if (strcmp(conninfo, primary_conninfo) != 0)
+ ereport(FATAL,
+ (errcode(ERRCODE_ADMIN_SHUTDOWN),
+ errmsg("terminating walreceiver process because primary_conninfo was changed")));
+ pfree(conninfo);
}
/* Wait a while for data to arrive */
*** a/src/backend/replication/walreceiverfuncs.c
--- b/src/backend/replication/walreceiverfuncs.c
***************
*** 166,176 **** ShutdownWalRcv(void)
/*
* Request postmaster to start walreceiver.
*
! * recptr indicates the position where streaming should begin, and conninfo
! * is a libpq connection string to use.
*/
void
! RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo)
{
/* use volatile pointer to prevent code rearrangement */
volatile WalRcvData *walrcv = WalRcv;
--- 166,175 ----
/*
* Request postmaster to start walreceiver.
*
! * recptr indicates the position where streaming should begin.
*/
void
! RequestXLogStreaming(XLogRecPtr recptr)
{
/* use volatile pointer to prevent code rearrangement */
volatile WalRcvData *walrcv = WalRcv;
***************
*** 190,199 **** RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo)
/* It better be stopped before we try to restart it */
Assert(walrcv->walRcvState == WALRCV_STOPPED);
- if (conninfo != NULL)
- strlcpy((char *) walrcv->conninfo, conninfo, MAXCONNINFO);
- else
- walrcv->conninfo[0] = '\0';
walrcv->walRcvState = WALRCV_STARTING;
walrcv->startTime = now;
--- 189,194 ----
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 30,35 ****
--- 30,36 ----
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
+ #include "access/xlog_internal.h"
#include "catalog/namespace.h"
#include "commands/async.h"
#include "commands/prepare.h"
***************
*** 206,211 **** static bool check_application_name(char **newval, void **extra, GucSource source
--- 207,220 ----
static void assign_application_name(const char *newval, void *extra);
static const char *show_unix_socket_permissions(void);
static const char *show_log_file_mode(void);
+ static bool check_recovery_target_xid(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_xid(const char *newval, void *extra);
+ static bool check_recovery_target_name(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_name(const char *newval, void *extra);
+ static bool check_recovery_target_time(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_time(const char *newval, void *extra);
+ static bool check_recovery_target_timeline(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_timeline(const char *newval, void *extra);
static char *config_enum_get_options(struct config_enum * record,
const char *prefix, const char *suffix,
***************
*** 477,482 **** static int wal_block_size;
--- 486,493 ----
static int wal_segment_size;
static bool integer_datetimes;
static int effective_io_concurrency;
+ static char *recovery_target_xid_string;
+ static char *recovery_target_time_string;
/* should be static, but commands/variable.c needs to get at this */
char *role_string;
***************
*** 556,561 **** const char *const config_group_names[] =
--- 567,576 ----
gettext_noop("Write-Ahead Log / Checkpoints"),
/* WAL_ARCHIVING */
gettext_noop("Write-Ahead Log / Archiving"),
+ /* WAL_ARCHIVE_RECOVERY */
+ gettext_noop("Write-Ahead Log / Archive Recovery"),
+ /* WAL_RECOVERY_TARGET */
+ gettext_noop("Write-Ahead Log / Recovery Target"),
/* REPLICATION */
gettext_noop("Replication"),
/* REPLICATION_SENDING */
***************
*** 1366,1371 **** static struct config_bool ConfigureNamesBool[] =
--- 1381,1416 ----
},
{
+ {"recovery_target_inclusive", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether to include or exclude transaction with recovery target."),
+ NULL
+ },
+ &recovery_target_inclusive,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"pause_at_recovery_target", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether recovery should pause when the recovery target is reached."),
+ NULL
+ },
+ &pause_at_recovery_target,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"standby_mode", PGC_POSTMASTER, REPLICATION_STANDBY,
+ gettext_noop("Sets whether to start the server as a standby."),
+ NULL
+ },
+ &standby_mode,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"hot_standby", PGC_POSTMASTER, REPLICATION_STANDBY,
gettext_noop("Allows connections and queries during recovery."),
NULL
***************
*** 2522,2527 **** static struct config_string ConfigureNamesString[] =
--- 2567,2663 ----
},
{
+ {"restore_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will retrieve an archived WAL file."),
+ NULL
+ },
+ &restore_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"archive_cleanup_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed at every restartpoint."),
+ NULL
+ },
+ &archive_cleanup_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_end_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed once only at the end of recovery."),
+ NULL
+ },
+ &recovery_end_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_target_xid", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the transaction ID up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_xid_string,
+ "",
+ check_recovery_target_xid, assign_recovery_target_xid, NULL
+ },
+
+ {
+ {"recovery_target_name", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the named restore point."),
+ NULL
+ },
+ &recovery_target_name,
+ "",
+ check_recovery_target_name, assign_recovery_target_name, NULL
+ },
+
+ {
+ {"recovery_target_time", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the time stamp up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_time_string,
+ "",
+ check_recovery_target_time, assign_recovery_target_time, NULL
+ },
+
+ {
+ {"recovery_target_timeline", PGC_POSTMASTER, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets recoverying into a particular timeline."),
+ NULL
+ },
+ &recovery_target_timeline_string,
+ "",
+ check_recovery_target_timeline, assign_recovery_target_timeline, NULL
+ },
+
+ {
+ {"primary_conninfo", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the connection string to be used to connect with the primary."),
+ NULL,
+ GUC_SUPERUSER_ONLY
+ },
+ &primary_conninfo,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"trigger_file", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the trigger file whose presence ends recovery in the standby."),
+ NULL
+ },
+ &trigger_file,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
{"client_encoding", PGC_USERSET, CLIENT_CONN_LOCALE,
gettext_noop("Sets the client's character set encoding."),
NULL,
***************
*** 8691,8694 **** show_log_file_mode(void)
--- 8827,8984 ----
return buf;
}
+ static bool
+ check_recovery_target_xid(char **newval, void **extra, GucSource source)
+ {
+ TransactionId xid;
+ TransactionId *myextra;
+
+ if (strcmp(*newval, "") == 0)
+ xid = InvalidTransactionId;
+ else
+ {
+ errno = 0;
+ xid = (TransactionId) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_xid is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+ }
+
+ myextra = (TransactionId *) guc_malloc(ERROR, sizeof(TransactionId));
+ *myextra = xid;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_xid(const char *newval, void *extra)
+ {
+ recovery_target_xid = *((TransactionId *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_name(char **newval, void **extra, GucSource source)
+ {
+ if (strlen(*newval) >= MAXFNAMELEN)
+ {
+ GUC_check_errdetail("\"recovery_target_name\" is too long (maximum %d characters)",
+ MAXFNAMELEN - 1);
+ return false;
+ }
+ return true;
+ }
+
+ static void
+ assign_recovery_target_name(const char *newval, void *extra)
+ {
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (newval[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_time(char **newval, void **extra, GucSource source)
+ {
+ TimestampTz time;
+ TimestampTz *myextra;
+ MemoryContext oldcontext = CurrentMemoryContext;
+
+ PG_TRY();
+ {
+ time = (strcmp(*newval, "") == 0) ?
+ 0 :
+ DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
+ CStringGetDatum(*newval),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ }
+ PG_CATCH();
+ {
+ ErrorData *edata;
+
+ /* Save error info */
+ MemoryContextSwitchTo(oldcontext);
+ edata = CopyErrorData();
+ FlushErrorState();
+
+ /* Pass the error message */
+ GUC_check_errdetail("%s", edata->message);
+ FreeErrorData(edata);
+ return false;
+ }
+ PG_END_TRY();
+
+ myextra = (TimestampTz *) guc_malloc(ERROR, sizeof(TimestampTz));
+ *myextra = time;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_time(const char *newval, void *extra)
+ {
+ recovery_target_time = *((TimestampTz *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_timeline(char **newval, void **extra, GucSource source)
+ {
+ TimeLineID tli = 0;
+ TimeLineID *myextra;
+
+ if (strcmp(*newval, "") == 0 || strcmp(*newval, "latest") == 0)
+ tli = 0;
+ else
+ {
+ errno = 0;
+ tli = (TimeLineID) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_timeline is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+ }
+
+ myextra = (TimeLineID *) guc_malloc(ERROR, sizeof(TimeLineID));
+ *myextra = tli;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_timeline(const char *newval, void *extra)
+ {
+ recovery_target_timeline = *((TimeLineID *) extra);
+ }
+
#include "guc-file.c"
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 192,197 ****
--- 192,214 ----
#archive_timeout = 0 # force a logfile segment switch after this
# number of seconds; 0 disables
+ # - Archive Recovery -
+ #restore_command = '' # command to use to restore an archived logfile segment
+ # placeholders: %p = path of file to restore
+ # %f = file name only
+ # e.g. 'cp /mnt/server/archivedir/%f %p'
+ #archive_cleanup_command = '' # command to execute at every restartpoint
+ #recovery_end_command = '' # command to execute at completion of recovery
+
+ # - Recovery Target -
+ #recovery_target_xid = ''
+ #recovery_target_name = ''
+ #recovery_target_time = ''
+ #recovery_target_inclusive = on
+ #pause_at_recovery_target = on
+ #recovery_target_timeline = '' # timeline ID or 'latest'
+ # (change requires restart)
+
#------------------------------------------------------------------------------
# REPLICATION
***************
*** 219,224 ****
--- 236,245 ----
# These settings are ignored on a master server
+ #standby_mode = off # "on" starts the server as a standby
+ # (change requires restart)
+ #primary_conninfo = '' # connection string to connect to the master
+ #trigger_file = '' # trigger file to promote the standby
#hot_standby = off # "on" allows queries during recovery
# (change requires restart)
#max_standby_archive_delay = 30s # max delay before canceling queries
*** a/src/bin/pg_ctl/pg_ctl.c
--- b/src/bin/pg_ctl/pg_ctl.c
***************
*** 883,889 **** do_stop(void)
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.conf is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
--- 883,889 ----
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.trigger is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
***************
*** 971,977 **** do_restart(void)
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.conf is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
--- 971,977 ----
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.trigger is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
***************
*** 1082,1088 **** do_promote(void)
exit(1);
}
! /* If recovery.conf doesn't exist, the server is not in standby mode */
if (stat(recovery_file, &statbuf) != 0)
{
write_stderr(_("%s: cannot promote server; "
--- 1082,1088 ----
exit(1);
}
! /* If recovery.trigger doesn't exist, the server is not in standby mode */
if (stat(recovery_file, &statbuf) != 0)
{
write_stderr(_("%s: cannot promote server; "
***************
*** 2163,2169 **** main(int argc, char **argv)
snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
! snprintf(recovery_file, MAXPGPATH, "%s/recovery.conf", pg_data);
snprintf(promote_file, MAXPGPATH, "%s/promote", pg_data);
}
--- 2163,2169 ----
snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
! snprintf(recovery_file, MAXPGPATH, "%s/recovery.trigger", pg_data);
snprintf(promote_file, MAXPGPATH, "%s/promote", pg_data);
}
*** a/src/include/access/xlog.h
--- b/src/include/access/xlog.h
***************
*** 198,203 **** extern bool XLogArchiveMode;
--- 198,217 ----
extern char *XLogArchiveCommand;
extern bool EnableHotStandby;
extern bool log_checkpoints;
+ extern char *restore_command;
+ extern char *archive_cleanup_command;
+ extern char *recovery_end_command;
+ extern bool standby_mode;
+ extern char *primary_conninfo;
+ extern char *trigger_file;
+ extern RecoveryTargetType recovery_target;
+ extern TransactionId recovery_target_xid;
+ extern TimestampTz recovery_target_time;
+ extern char *recovery_target_name;
+ extern bool recovery_target_inclusive;
+ extern bool pause_at_recovery_target;
+ extern char *recovery_target_timeline_string;
+ extern TimeLineID recovery_target_timeline;
/* WAL levels */
typedef enum WalLevel
*** a/src/include/replication/walreceiver.h
--- b/src/include/replication/walreceiver.h
***************
*** 110,116 **** extern Size WalRcvShmemSize(void);
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
! extern void RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
#endif /* _WALRECEIVER_H */
--- 110,116 ----
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
! extern void RequestXLogStreaming(XLogRecPtr recptr);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
#endif /* _WALRECEIVER_H */
*** a/src/include/utils/guc_tables.h
--- b/src/include/utils/guc_tables.h
***************
*** 68,73 **** enum config_group
--- 68,75 ----
WAL_SETTINGS,
WAL_CHECKPOINTS,
WAL_ARCHIVING,
+ WAL_ARCHIVE_RECOVERY,
+ WAL_RECOVERY_TARGET,
REPLICATION,
REPLICATION_SENDING,
REPLICATION_MASTER,
On Mon, Sep 26, 2011 at 7:45 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On sön, 2011-09-25 at 12:58 -0400, Tom Lane wrote:
And it's not like we don't break configuration file
contents in most releases anyway, so I really fail to see why this one
has suddenly become sacrosanct.Well, there is a slight difference. Changes in postgresql.conf
parameter names and settings are adjusted automatically for me by my
package upgrade script. If we, say, changed the names of recovery.conf
parameters, I'd have to get a new version of my $SuperReplicationTool.
That tool could presumably look at PG_VERSION and put a recovery.conf
with the right spellings in the right place.But if we completely change the way the replication configuration
interacts, it's not clear that a smooth upgrade is possible without
significant effort. That said, I don't see why it wouldn't be possible,
but let's design with upgradability in mind, instead of claiming that we
have never supported upgrades of this kind anyway.
Currently recovery.conf has two roles:
#1. recovery.conf is used as a trigger file to enable archive recovery.
At the end of recovery, recovery.conf is renamed to recovery.done.
#2. recovery.conf is used as a configuration file for recovery parameters.
Which role do you think we should support in 9.2 because of the backward
compatibility? Both? Unless I misunderstand the discussion so far, Tom and
Robert (and I) agree to get rid of both. Simon seems to agree to remove
only the former role, but not the latter. How about you? If you agree to
remove the former, too, let's focus on the discussion about whether the
latter role should be supported in 9.2.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Tue, Sep 27, 2011 at 1:33 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
Though there is still ongonig discussion, since there is no objection about
the above two changes, I revised the patch that way. And I fixed the minor
bug handling the default value of recovery_target_timeline wrongly.
Attached is the revised version of the patch.
This patch no longer applies as it conflicts with the following commit:
commit d56b3afc0376afe491065d9eca6440b3cc7b1346
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Oct 2 16:50:04 2011 -0400
Restructure error handling in reading of postgresql.conf.
Cheers,
Jeff
On Tue, Sep 27, 2011 at 10:34 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Mon, Sep 26, 2011 at 7:45 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On sön, 2011-09-25 at 12:58 -0400, Tom Lane wrote:
And it's not like we don't break configuration file
contents in most releases anyway, so I really fail to see why this one
has suddenly become sacrosanct.Well, there is a slight difference. Changes in postgresql.conf
parameter names and settings are adjusted automatically for me by my
package upgrade script. If we, say, changed the names of recovery.conf
parameters, I'd have to get a new version of my $SuperReplicationTool.
That tool could presumably look at PG_VERSION and put a recovery.conf
with the right spellings in the right place.But if we completely change the way the replication configuration
interacts, it's not clear that a smooth upgrade is possible without
significant effort. That said, I don't see why it wouldn't be possible,
but let's design with upgradability in mind, instead of claiming that we
have never supported upgrades of this kind anyway.Currently recovery.conf has two roles:
#1. recovery.conf is used as a trigger file to enable archive recovery.
At the end of recovery, recovery.conf is renamed to recovery.done.#2. recovery.conf is used as a configuration file for recovery parameters.
Which role do you think we should support in 9.2 because of the backward
compatibility? Both? Unless I misunderstand the discussion so far, Tom and
Robert (and I) agree to get rid of both. Simon seems to agree to remove
only the former role, but not the latter. How about you? If you agree to
remove the former, too, let's focus on the discussion about whether the
latter role should be supported in 9.2.
Tatsuo/Josh/Robert also discussed how recovery.conf can be used to
provide parameters solely for recovery. That is difficult to do
without causing all downstream tools to make major changes in the ways
they supply parameters.
Keeping our APIs relatively stable is important to downstream tools. I
have no objection to a brave new world, as long as you don't chuck out
the one that works right now. Breaking APIs needs a good reason and
I've not seen one discussed anywhere. No problem with immediately
deprecating the old API and declare is planned to be removed in
release 10.0.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon,
Tatsuo/Josh/Robert also discussed how recovery.conf can be used to
provide parameters solely for recovery. That is difficult to do
without causing all downstream tools to make major changes in the ways
they supply parameters.
Actually, this case is easily solved by an "include recovery.conf"
parameter. So it's a non-issue.
Keeping our APIs relatively stable is important to downstream tools. I
have no objection to a brave new world, as long as you don't chuck out
the one that works right now. Breaking APIs needs a good reason and
I've not seen one discussed anywhere. No problem with immediately
deprecating the old API and declare is planned to be removed in
release 10.0.
So after debugging some of our failover scripts, here's the real-world
problems I'm trying to solve. These design flaws are issues which cause
automated failover or failback to abort, leading to unexpected downtime,
so they are not just issues of neatness:
1. Recovery.conf being both a configuration file AND a trigger to
initiate recovery mode, preventing us from separating configuration
management from failover.
2. The inability to read recovery.conf parameters via SQL on a hot
standby, forcing us to parse the file to find out its settings, or guess.
(1) is a quite serious issue; it effectively makes recovery.conf
impossible to integrate with puppet and other configuration management
frameworks. I also don't see a way to fix it without breaking backwards
compatibility.
BTW, I'm not criticizing the original design for this. We simply didn't
know better until lots of people were using these tools in production.
But it's time to fix them, and the longer we wait, the more painful it
will be.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On 10/10/11 10:52 AM, Josh Berkus wrote:
So after debugging some of our failover scripts, here's the real-world
problems I'm trying to solve. These design flaws are issues which cause
automated failover or failback to abort, leading to unexpected downtime,
so they are not just issues of neatness:
That's "automated failover or *manual* failback". I never, ever
recommend automated failback. Just FYI.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Mon, Oct 10, 2011 at 6:52 PM, Josh Berkus <josh@agliodbs.com> wrote:
Tatsuo/Josh/Robert also discussed how recovery.conf can be used to
provide parameters solely for recovery. That is difficult to do
without causing all downstream tools to make major changes in the ways
they supply parameters.Actually, this case is easily solved by an "include recovery.conf"
parameter. So it's a non-issue.
That is what I've suggested and yes, doing that is straightforward.
If you mean "do that in a program" if we had a problem with adding
parameters, we also have a problem adding an include.
We should avoid breaking programs which we have no reason to break.
Stability is good, change without purpose is not.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Sun, Oct 9, 2011 at 9:15 AM, Jeff Janes <jeff.janes@gmail.com> wrote:
On Tue, Sep 27, 2011 at 1:33 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
Though there is still ongonig discussion, since there is no objection about
the above two changes, I revised the patch that way. And I fixed the minor
bug handling the default value of recovery_target_timeline wrongly.
Attached is the revised version of the patch.This patch no longer applies as it conflicts with the following commit:
Thanks! Attached patch is the updated version.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Attachments:
unite_recoveryconf_postgresqlconf_v3.patchtext/x-patch; charset=US-ASCII; name=unite_recoveryconf_postgresqlconf_v3.patchDownload
*** a/contrib/pg_archivecleanup/pg_archivecleanup.c
--- b/contrib/pg_archivecleanup/pg_archivecleanup.c
***************
*** 208,214 **** usage(void)
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "For use as archive_cleanup_command in recovery.conf when standby_mode = on:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
"e.g.\n"
" archive_cleanup_command = 'pg_archivecleanup /mnt/server/archiverdir %%r'\n");
--- 208,214 ----
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "For use as archive_cleanup_command in postgresql.conf when standby_mode = on:\n"
" archive_cleanup_command = 'pg_archivecleanup [OPTION]... ARCHIVELOCATION %%r'\n"
"e.g.\n"
" archive_cleanup_command = 'pg_archivecleanup /mnt/server/archiverdir %%r'\n");
*** a/contrib/pg_standby/pg_standby.c
--- b/contrib/pg_standby/pg_standby.c
***************
*** 531,537 **** usage(void)
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "Main intended use as restore_command in recovery.conf:\n"
" restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
"e.g.\n"
" restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
--- 531,537 ----
printf(" --help show this help, then exit\n");
printf(" --version output version information, then exit\n");
printf("\n"
! "Main intended use as restore_command in postgresql.conf:\n"
" restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
"e.g.\n"
" restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
*** a/doc/src/sgml/backup.sgml
--- b/doc/src/sgml/backup.sgml
***************
*** 995,1002 **** SELECT pg_stop_backup();
</listitem>
<listitem>
<para>
! Create a recovery command file <filename>recovery.conf</> in the cluster
! data directory (see <xref linkend="recovery-config">). You might
also want to temporarily modify <filename>pg_hba.conf</> to prevent
ordinary users from connecting until you are sure the recovery was successful.
</para>
--- 995,1009 ----
</listitem>
<listitem>
<para>
! Set up recovery parameters in <filename>postgresql.conf</> (see
! <xref linkend="runtime-config-wal-archive-recovery"> and
! <xref linkend="runtime-config-wal-recovery-target">).
! </para>
! </listitem>
! <listitem>
! <para>
! Create a recovery trigger file <filename>recovery.trigger</>
! in the cluster data directory. You might
also want to temporarily modify <filename>pg_hba.conf</> to prevent
ordinary users from connecting until you are sure the recovery was successful.
</para>
***************
*** 1008,1014 **** SELECT pg_stop_backup();
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
! <filename>recovery.conf</> to <filename>recovery.done</> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
</para>
--- 1015,1021 ----
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
! <filename>recovery.trigger</> to <filename>recovery.done</> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
</para>
***************
*** 1024,1035 **** SELECT pg_stop_backup();
</para>
<para>
! The key part of all this is to set up a recovery configuration file that
! describes how you want to recover and how far the recovery should
! run. You can use <filename>recovery.conf.sample</> (normally
! located in the installation's <filename>share/</> directory) as a
! prototype. The one thing that you absolutely must specify in
! <filename>recovery.conf</> is the <varname>restore_command</>,
which tells <productname>PostgreSQL</> how to retrieve archived
WAL file segments. Like the <varname>archive_command</>, this is
a shell command string. It can contain <literal>%f</>, which is
--- 1031,1041 ----
</para>
<para>
! The key part of all this is to set up recovery parameters that
! specify how you want to recover and how far the recovery should
! run. The one thing that you absolutely must specify in
! <filename>postgresql.conf</> to recover from the backup is
! the <varname>restore_command</>,
which tells <productname>PostgreSQL</> how to retrieve archived
WAL file segments. Like the <varname>archive_command</>, this is
a shell command string. It can contain <literal>%f</>, which is
***************
*** 1085,1091 **** restore_command = 'cp /mnt/server/archivedir/%f %p'
<para>
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
! required stopping point in <filename>recovery.conf</>. You can specify
the stop point, known as the <quote>recovery target</>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
--- 1091,1097 ----
<para>
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
! required stopping point in <filename>postgresql.conf</>. You can specify
the stop point, known as the <quote>recovery target</>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
***************
*** 1182,1189 **** restore_command = 'cp /mnt/server/archivedir/%f %p'
The default behavior of recovery is to recover along the same timeline
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
! was itself generated after a recovery attempt), you need to specify the
! target timeline ID in <filename>recovery.conf</>. You cannot recover into
timelines that branched off earlier than the base backup.
</para>
</sect2>
--- 1188,1196 ----
The default behavior of recovery is to recover along the same timeline
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
! was itself generated after a recovery attempt), you need to set
! <xref linkend="guc-recovery-target-timeline"> to the
! target timeline ID in <filename>postgresql.conf</>. You cannot recover into
timelines that branched off earlier than the base backup.
</para>
</sect2>
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 1963,1968 **** SET ENABLE_SEQSCAN TO OFF;
--- 1963,2240 ----
</variablelist>
</sect2>
+ <sect2 id="runtime-config-wal-archive-recovery">
+ <title>Archive Recovery</title>
+
+ <variablelist>
+ <varlistentry id="guc-restore-command" xreflabel="restore_command">
+ <term><varname>restore_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>restore_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command to execute to retrieve an archived segment of
+ the WAL file series. This parameter is required for archive recovery,
+ but optional for streaming replication.
+ Any <literal>%f</> in the string is
+ replaced by the name of the file to retrieve from the archive,
+ and any <literal>%p</> is replaced by the copy destination path name
+ on the server.
+ (The path name is relative to the current working directory,
+ i.e., the cluster's data directory.)
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point. That is the earliest file that must be kept
+ to allow a restore to be restartable, so this information can be used
+ to truncate the archive to just the minimum required to support
+ restarting from the current restore. <literal>%r</> is typically only
+ used by warm-standby configurations
+ (see <xref linkend="warm-standby">).
+ Write <literal>%%</> to embed an actual <literal>%</> character.
+ </para>
+ <para>
+ It is important for the command to return a zero exit status
+ only if it succeeds. The command <emphasis>will</> be asked for file
+ names that are not present in the archive; it must return nonzero
+ when so asked. Examples:
+ <programlisting>
+ restore_command = 'cp /mnt/server/archivedir/%f "%p"'
+ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
+ </programlisting>
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-archive-cleanup-command" xreflabel="archive_cleanup_command">
+ <term><varname>archive_cleanup_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>archive_cleanup_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command that will be executed at every restartpoint.
+ The purpose of <varname>archive_cleanup_command</> is to
+ provide a mechanism for cleaning up old archived WAL files that
+ are no longer needed by the standby server.
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point.
+ That is the earliest file that must be <emphasis>kept</> to allow a
+ restore to be restartable, and so all files earlier than <literal>%r</>
+ may be safely removed.
+ This information can be used to truncate the archive to just the
+ minimum required to support restart from the current restore.
+ The <xref linkend="pgarchivecleanup"> module
+ is often used in <varname>archive_cleanup_command</> for
+ single-standby configurations, for example:
+ <programlisting>archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'</programlisting>
+ Note however that if multiple standby servers are restoring from the
+ same archive directory, you will need to ensure that you do not delete
+ WAL files until they are no longer needed by any of the servers.
+ <varname>archive_cleanup_command</> would typically be used in a
+ warm-standby configuration (see <xref linkend="warm-standby">).
+ Write <literal>%%</> to embed an actual <literal>%</> character in the
+ command.
+ </para>
+ <para>
+ If the command returns a non-zero exit status then a WARNING log
+ message will be written.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-end-command" xreflabel="recovery_end_command">
+ <term><varname>recovery_end_command</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_end_command</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ The shell command that will be executed once only
+ at the end of recovery. This parameter is optional. The purpose of the
+ <varname>recovery_end_command</> is to provide a mechanism for cleanup
+ following replication or recovery.
+ Any <literal>%r</> is replaced by the name of the file containing the
+ last valid restart point, like in <varname>archive_cleanup_command</>.
+ </para>
+ <para>
+ If the command returns a non-zero exit status then a WARNING log
+ message will be written and the database will proceed to start up
+ anyway. An exception is that if the command was terminated by a
+ signal, the database will not proceed with startup.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="runtime-config-wal-recovery-target">
+ <title>Recovery Target</title>
+
+ <variablelist>
+ <varlistentry id="guc-recovery-target-name" xreflabel="recovery_target_name">
+ <term><varname>recovery_target_name</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_name</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the named restore point, created with
+ <function>pg_create_restore_point()</> to which recovery will proceed.
+ At most one of <varname>recovery_target_name</>,
+ <varname>recovery_target_time</> or
+ <varname>recovery_target_xid</> can be specified. The default
+ value is an empty string, which will recover to the end of the WAL log.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-time" xreflabel="recovery_target_time">
+ <term><varname>recovery_target_time</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_time</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the time stamp up to which recovery will proceed.
+ This parameter must be specified in the date/time format
+ (see <xref linkend="datatype-datetime-input"> for details).
+ At most one of <varname>recovery_target_time</>,
+ <varname>recovery_target_name</> or
+ <varname>recovery_target_xid</> can be specified.
+ The default value is an empty string, which will recover to
+ the end of the WAL log. The precise stopping point is also
+ influenced by <varname>recovery_target_inclusive</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-xid" xreflabel="recovery_target_xid">
+ <term><varname>recovery_target_xid</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_xid</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the transaction ID up to which recovery will proceed.
+ Keep in mind that while transaction IDs are assigned sequentially
+ at transaction start, transactions can complete in a different
+ numeric order. The transactions that will be recovered are
+ those that committed before (and optionally including)
+ the specified one. At most one of <varname>recovery_target_xid</>,
+ <varname>recovery_target_name</> or
+ <varname>recovery_target_time</> can be specified.
+ The default value is an empty string, which will recover to the end of
+ the WAL log. The precise stopping point is also influenced by
+ <varname>recovery_target_inclusive</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-inclusive" xreflabel="recovery_target_inclusive">
+ <term><varname>recovery_target_inclusive</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_inclusive</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether we stop just after the specified recovery target
+ (<literal>on</>), or just before the recovery target (<literal>off</>).
+ Applies to both <varname>recovery_target_time</>
+ and <varname>recovery_target_xid</>, whichever one is
+ specified for this recovery. This indicates whether transactions
+ having exactly the target commit time or ID, respectively, will
+ be included in the recovery. Default is <literal>on</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-recovery-target-timeline" xreflabel="recovery_target_timeline">
+ <term><varname>recovery_target_timeline</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_target_timeline</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies recovering into a particular timeline. The default value is
+ an empty string, which will recover along the same timeline that was
+ current when the base backup was taken. Setting this to
+ <literal>latest</> recovers to the latest timeline found in the archive,
+ which is useful in a standby server. Other than that you only need to
+ set this parameter in complex re-recovery situations, where you need
+ to return to a state that itself was reached after a point-in-time
+ recovery. See <xref linkend="backup-timelines"> for discussion.
+ </para>
+ <para>
+ This parameter can only be set at server start. It only has effect
+ during archive recovery or in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-pause-at-recovery-target" xreflabel="pause_at_recovery_target">
+ <term><varname>pause_at_recovery_target</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>pause_at_recovery_target</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether recovery should pause when the recovery target
+ is reached. The default is <literal>on</>.
+ This is intended to allow queries to be executed against the
+ database to check if this recovery target is the most desirable
+ point for recovery. The paused state can be resumed by using
+ <function>pg_xlog_replay_resume()</> (See
+ <xref linkend="functions-recovery-control-table">), which then
+ causes recovery to end. If this recovery target is not the
+ desired stopping point, then shutdown the server, change the
+ recovery target settings to a later target and restart to
+ continue recovery.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect during archive
+ recovery or in standby mode if recovery target is set.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
</sect1>
<sect1 id="runtime-config-replication">
***************
*** 2184,2189 **** SET ENABLE_SEQSCAN TO OFF;
--- 2456,2548 ----
<variablelist>
+ <varlistentry id="guc-standby-mode" xreflabel="standby_mode">
+ <term><varname>standby_mode</varname> (<type>boolean</type>)</term>
+ <indexterm>
+ <primary><varname>standby_mode</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies whether to start the <productname>PostgreSQL</> server as
+ a standby when recovery trigger file <filename>recovery.trigger</> exists.
+ The default value is <literal>off</>.
+ If this parameter is <literal>on</>, the server will not
+ stop recovery when the end of archived WAL is reached,
+ but will keep trying to continue recovery by fetching new WAL segments
+ using <varname>restore_command</> and/or by connecting to
+ the primary server as specified by the <varname>primary_conninfo</>
+ setting.
+ </para>
+ <para>
+ This parameter can only be set at server start. It only has effect
+ if recovery trigger file <filename>recovery.trigger</> exists.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-primary-conninfo" xreflabel="primary_conninfo">
+ <term><varname>primary_conninfo</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>primary_conninfo</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies a connection string to be used for the standby server
+ to connect with the primary. This string is in the format
+ accepted by the libpq <function>PQconnectdb</function> function,
+ described in <xref linkend="libpq-connect">. If any option is
+ unspecified in this string, then the corresponding environment
+ variable (see <xref linkend="libpq-envars">) is checked. If the
+ environment variable is not set either, then defaults are used.
+ If this parameter is an empty string (the default), no attempt is
+ made to connect to the master.
+ </para>
+ <para>
+ The connection string should specify the host name (or address)
+ of the primary server, as well as the port number if it is not
+ the same as the standby server's default.
+ Also specify a user name corresponding to a role that has the
+ <literal>REPLICATION</> and <literal>LOGIN</> privileges on the
+ primary (see
+ <xref linkend="streaming-replication-authentication">).
+ A password needs to be provided too, if the primary demands password
+ authentication. It can be provided in the
+ <varname>primary_conninfo</varname> string, or in a separate
+ <filename>~/.pgpass</> file on the standby server (use
+ <literal>replication</> as the database name).
+ Do not specify a database name in the
+ <varname>primary_conninfo</varname> string.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect in standby mode.
+ </para>
+ <para>
+ If this parameter is changed while replication is in progress,
+ the standby terminates replication, and then tries to restart
+ replication with new setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-trigger-file" xreflabel="trigger_file">
+ <term><varname>trigger_file</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>trigger_file</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies a trigger file whose presence ends recovery in the
+ standby. Even if this value is not set, you can still promote
+ the standby using <command>pg_ctl promote</>.
+ </para>
+ <para>
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line. It only has effect in standby mode.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-hot-standby" xreflabel="hot_standby">
<term><varname>hot_standby</varname> (<type>boolean</type>)</term>
<indexterm>
*** a/doc/src/sgml/filelist.sgml
--- b/doc/src/sgml/filelist.sgml
***************
*** 42,48 ****
<!ENTITY manage-ag SYSTEM "manage-ag.sgml">
<!ENTITY monitoring SYSTEM "monitoring.sgml">
<!ENTITY regress SYSTEM "regress.sgml">
- <!ENTITY recovery-config SYSTEM "recovery-config.sgml">
<!ENTITY runtime SYSTEM "runtime.sgml">
<!ENTITY config SYSTEM "config.sgml">
<!ENTITY user-manag SYSTEM "user-manag.sgml">
--- 42,47 ----
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***************
*** 14136,14142 **** postgres=# select pg_start_backup('label_goes_here');
<function>pg_create_restore_point</> creates a named transaction log
record that can be used as recovery target, and returns the corresponding
transaction log location. The given name can then be used with
! <xref linkend="recovery-target-name"> to specify the point up to which
recovery will proceed. Avoid creating multiple restore points with the
same name, since recovery will stop at the first one whose name matches
the recovery target.
--- 14136,14142 ----
<function>pg_create_restore_point</> creates a named transaction log
record that can be used as recovery target, and returns the corresponding
transaction log location. The given name can then be used with
! <xref linkend="guc-recovery-target-name"> to specify the point up to which
recovery will proceed. Avoid creating multiple restore points with the
same name, since recovery will stop at the first one whose name matches
the recovery target.
*** a/doc/src/sgml/high-availability.sgml
--- b/doc/src/sgml/high-availability.sgml
***************
*** 591,597 **** protocol to make nodes agree on a serializable transactional order.
<para>
In standby mode, the server continuously applies WAL received from the
master server. The standby server can read WAL from a WAL archive
! (see <xref linkend="restore-command">) or directly from the master
over a TCP connection (streaming replication). The standby server will
also attempt to restore any WAL found in the standby cluster's
<filename>pg_xlog</> directory. That typically happens after a server
--- 591,597 ----
<para>
In standby mode, the server continuously applies WAL received from the
master server. The standby server can read WAL from a WAL archive
! (see <xref linkend="guc-restore-command">) or directly from the master
over a TCP connection (streaming replication). The standby server will
also attempt to restore any WAL found in the standby cluster's
<filename>pg_xlog</> directory. That typically happens after a server
***************
*** 658,665 **** protocol to make nodes agree on a serializable transactional order.
<para>
To set up the standby server, restore the base backup taken from primary
server (see <xref linkend="backup-pitr-recovery">). Create a recovery
! command file <filename>recovery.conf</> in the standby's cluster data
! directory, and turn on <varname>standby_mode</>. Set
<varname>restore_command</> to a simple command to copy files from
the WAL archive. If you plan to have multiple standby servers for high
availability purposes, set <varname>recovery_target_timeline</> to
--- 658,665 ----
<para>
To set up the standby server, restore the base backup taken from primary
server (see <xref linkend="backup-pitr-recovery">). Create a recovery
! trigger file <filename>recovery.trigger</> in the standby's cluster data
! directory. Turn on <varname>standby_mode</> and set
<varname>restore_command</> to a simple command to copy files from
the WAL archive. If you plan to have multiple standby servers for high
availability purposes, set <varname>recovery_target_timeline</> to
***************
*** 695,701 **** protocol to make nodes agree on a serializable transactional order.
<para>
If you're using a WAL archive, its size can be minimized using the <xref
! linkend="archive-cleanup-command"> parameter to remove files that are no
longer required by the standby server.
The <application>pg_archivecleanup</> utility is designed specifically to
be used with <varname>archive_cleanup_command</> in typical single-standby
--- 695,701 ----
<para>
If you're using a WAL archive, its size can be minimized using the <xref
! linkend="guc-archive-cleanup-command"> parameter to remove files that are no
longer required by the standby server.
The <application>pg_archivecleanup</> utility is designed specifically to
be used with <varname>archive_cleanup_command</> in typical single-standby
***************
*** 706,712 **** protocol to make nodes agree on a serializable transactional order.
</para>
<para>
! A simple example of a <filename>recovery.conf</> is:
<programlisting>
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
--- 706,712 ----
</para>
<para>
! A simple example of standby settings is:
<programlisting>
standby_mode = 'on'
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
***************
*** 762,769 **** archive_cleanup_command = 'pg_archivecleanup /path/to/archive %r'
To use streaming replication, set up a file-based log-shipping standby
server as described in <xref linkend="warm-standby">. The step that
turns a file-based log-shipping standby into streaming replication
! standby is setting <varname>primary_conninfo</> setting in the
! <filename>recovery.conf</> file to point to the primary server. Set
<xref linkend="guc-listen-addresses"> and authentication options
(see <filename>pg_hba.conf</>) on the primary so that the standby server
can connect to the <literal>replication</> pseudo-database on the primary
--- 762,769 ----
To use streaming replication, set up a file-based log-shipping standby
server as described in <xref linkend="warm-standby">. The step that
turns a file-based log-shipping standby into streaming replication
! standby is setting <varname>primary_conninfo</> to
! point to the primary server. Set
<xref linkend="guc-listen-addresses"> and authentication options
(see <filename>pg_hba.conf</>) on the primary so that the standby server
can connect to the <literal>replication</> pseudo-database on the primary
***************
*** 832,846 **** host replication foo 192.168.1.100/32 md5
</para>
<para>
The host name and port number of the primary, connection user name,
! and password are specified in the <filename>recovery.conf</> file.
The password can also be set in the <filename>~/.pgpass</> file on the
standby (specify <literal>replication</> in the <replaceable>database</>
field).
For example, if the primary is running on host IP <literal>192.168.1.50</>,
port <literal>5432</literal>, the account name for replication is
<literal>foo</>, and the password is <literal>foopass</>, the administrator
! can add the following line to the <filename>recovery.conf</> file on the
! standby:
<programlisting>
# The standby connects to the primary that is running on host 192.168.1.50
--- 832,845 ----
</para>
<para>
The host name and port number of the primary, connection user name,
! and password are specified in <varname>primary_conninfo</>.
The password can also be set in the <filename>~/.pgpass</> file on the
standby (specify <literal>replication</> in the <replaceable>database</>
field).
For example, if the primary is running on host IP <literal>192.168.1.50</>,
port <literal>5432</literal>, the account name for replication is
<literal>foo</>, and the password is <literal>foopass</>, the administrator
! can set <varname>primary_conninfo</> on the standby like this:
<programlisting>
# The standby connects to the primary that is running on host 192.168.1.50
***************
*** 1205,1212 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<para>
To trigger failover of a log-shipping standby server,
run <command>pg_ctl promote</> or create a trigger
! file with the file name and path specified by the <varname>trigger_file</>
! setting in <filename>recovery.conf</>. If you're planning to use
<command>pg_ctl promote</> to fail over, <varname>trigger_file</> is
not required. If you're setting up the reporting servers that are
only used to offload read-only queries from the primary, not for high
--- 1204,1211 ----
<para>
To trigger failover of a log-shipping standby server,
run <command>pg_ctl promote</> or create a trigger
! file with the file name and path specified by the <varname>trigger_file</>.
! If you're planning to use
<command>pg_ctl promote</> to fail over, <varname>trigger_file</> is
not required. If you're setting up the reporting servers that are
only used to offload read-only queries from the primary, not for high
***************
*** 1251,1258 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
The magic that makes the two loosely coupled servers work together is
simply a <varname>restore_command</> used on the standby that,
when asked for the next WAL file, waits for it to become available from
! the primary. The <varname>restore_command</> is specified in the
! <filename>recovery.conf</> file on the standby server. Normal recovery
processing would request a file from the WAL archive, reporting failure
if the file was unavailable. For standby processing it is normal for
the next WAL file to be unavailable, so the standby must wait for
--- 1250,1256 ----
The magic that makes the two loosely coupled servers work together is
simply a <varname>restore_command</> used on the standby that,
when asked for the next WAL file, waits for it to become available from
! the primary. Normal recovery
processing would request a file from the WAL archive, reporting failure
if the file was unavailable. For standby processing it is normal for
the next WAL file to be unavailable, so the standby must wait for
***************
*** 1339,1346 **** if (!triggered)
</listitem>
<listitem>
<para>
Begin recovery on the standby server from the local WAL
! archive, using a <filename>recovery.conf</> that specifies a
<varname>restore_command</> that waits as described
previously (see <xref linkend="backup-pitr-recovery">).
</para>
--- 1337,1350 ----
</listitem>
<listitem>
<para>
+ Create a recovery trigger file <filename>recovery.trigger</>
+ in the standby's cluster data directory.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
Begin recovery on the standby server from the local WAL
! archive, specifying a
<varname>restore_command</> that waits as described
previously (see <xref linkend="backup-pitr-recovery">).
</para>
***************
*** 1831,1839 **** if (!triggered)
<title>Administrator's Overview</title>
<para>
! If <varname>hot_standby</> is turned <literal>on</> in
! <filename>postgresql.conf</> and there is a <filename>recovery.conf</>
! file present, the server will run in Hot Standby mode.
However, it may take some time for Hot Standby connections to be allowed,
because the server will not accept connections until it has completed
sufficient recovery to provide a consistent state against which queries
--- 1835,1843 ----
<title>Administrator's Overview</title>
<para>
! If <varname>hot_standby</> is turned <literal>on</>
! and there is a recovery trigger file
! <filename>recovery.trigger</> present, the server will run in Hot Standby mode.
However, it may take some time for Hot Standby connections to be allowed,
because the server will not accept connections until it has completed
sufficient recovery to provide a consistent state against which queries
*** a/doc/src/sgml/pgarchivecleanup.sgml
--- b/doc/src/sgml/pgarchivecleanup.sgml
***************
*** 37,44 ****
<para>
To configure a standby
! server to use <application>pg_archivecleanup</>, put this into its
! <filename>recovery.conf</filename> configuration file:
<programlisting>
archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
</programlisting>
--- 37,44 ----
<para>
To configure a standby
! server to use <application>pg_archivecleanup</>, specify
! <xref linkend="guc-archive-cleanup-command"> like this:
<programlisting>
archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
</programlisting>
***************
*** 46,52 **** archive_cleanup_command = 'pg_archivecleanup <replaceable>archivelocation</> %r'
files should be removed.
</para>
<para>
! When used within <xref linkend="archive-cleanup-command">, all WAL files
logically preceding the value of the <literal>%r</> argument will be removed
from <replaceable>archivelocation</>. This minimizes the number of files
that need to be retained, while preserving crash-restart capability. Use of
--- 46,52 ----
files should be removed.
</para>
<para>
! When used within <varname>archive_cleanup_command</>, all WAL files
logically preceding the value of the <literal>%r</> argument will be removed
from <replaceable>archivelocation</>. This minimizes the number of files
that need to be retained, while preserving crash-restart capability. Use of
*** a/doc/src/sgml/pgstandby.sgml
--- b/doc/src/sgml/pgstandby.sgml
***************
*** 48,55 ****
<para>
To configure a standby
! server to use <application>pg_standby</>, put this into its
! <filename>recovery.conf</filename> configuration file:
<programlisting>
restore_command = 'pg_standby <replaceable>archiveDir</> %f %p %r'
</programlisting>
--- 48,55 ----
<para>
To configure a standby
! server to use <application>pg_standby</>, specify
! <xref linkend="guc-restore-command"> like this:
<programlisting>
restore_command = 'pg_standby <replaceable>archiveDir</> %f %p %r'
</programlisting>
*** a/doc/src/sgml/postgres.sgml
--- b/doc/src/sgml/postgres.sgml
***************
*** 155,161 ****
&maintenance;
&backup;
&high-availability;
- &recovery-config;
&monitoring;
&diskusage;
&wal;
--- 155,160 ----
*** a/doc/src/sgml/recovery-config.sgml
--- /dev/null
***************
*** 1,363 ****
- <!-- doc/src/sgml/recovery-config.sgml -->
-
- <chapter id="recovery-config">
- <title>Recovery Configuration</title>
-
- <indexterm>
- <primary>configuration</primary>
- <secondary>of recovery</secondary>
- <tertiary>of a standby server</tertiary>
- </indexterm>
-
- <para>
- This chapter describes the settings available in the
- <filename>recovery.conf</><indexterm><primary>recovery.conf</></>
- file. They apply only for the duration of the
- recovery. They must be reset for any subsequent recovery you wish to
- perform. They cannot be changed once recovery has begun.
- </para>
-
- <para>
- Settings in <filename>recovery.conf</> are specified in the format
- <literal>name = 'value'</>. One parameter is specified per line.
- Hash marks (<literal>#</literal>) designate the rest of the
- line as a comment. To embed a single quote in a parameter
- value, write two quotes (<literal>''</>).
- </para>
-
- <para>
- A sample file, <filename>share/recovery.conf.sample</>,
- is provided in the installation's <filename>share/</> directory.
- </para>
-
- <sect1 id="archive-recovery-settings">
-
- <title>Archive Recovery Settings</title>
- <variablelist>
-
- <varlistentry id="restore-command" xreflabel="restore_command">
- <term><varname>restore_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>restore_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- The shell command to execute to retrieve an archived segment of
- the WAL file series. This parameter is required for archive recovery,
- but optional for streaming replication.
- Any <literal>%f</> in the string is
- replaced by the name of the file to retrieve from the archive,
- and any <literal>%p</> is replaced by the copy destination path name
- on the server.
- (The path name is relative to the current working directory,
- i.e., the cluster's data directory.)
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point. That is the earliest file that must be kept
- to allow a restore to be restartable, so this information can be used
- to truncate the archive to just the minimum required to support
- restarting from the current restore. <literal>%r</> is typically only
- used by warm-standby configurations
- (see <xref linkend="warm-standby">).
- Write <literal>%%</> to embed an actual <literal>%</> character.
- </para>
-
- <para>
- It is important for the command to return a zero exit status
- only if it succeeds. The command <emphasis>will</> be asked for file
- names that are not present in the archive; it must return nonzero
- when so asked. Examples:
- <programlisting>
- restore_command = 'cp /mnt/server/archivedir/%f "%p"'
- restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
- </programlisting>
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="archive-cleanup-command" xreflabel="archive_cleanup_command">
- <term><varname>archive_cleanup_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>archive_cleanup_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This optional parameter specifies a shell command that will be executed
- at every restartpoint. The purpose of
- <varname>archive_cleanup_command</> is to provide a mechanism for
- cleaning up old archived WAL files that are no longer needed by the
- standby server.
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point.
- That is the earliest file that must be <emphasis>kept</> to allow a
- restore to be restartable, and so all files earlier than <literal>%r</>
- may be safely removed.
- This information can be used to truncate the archive to just the
- minimum required to support restart from the current restore.
- The <xref linkend="pgarchivecleanup"> module
- is often used in <varname>archive_cleanup_command</> for
- single-standby configurations, for example:
- <programlisting>archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'</programlisting>
- Note however that if multiple standby servers are restoring from the
- same archive directory, you will need to ensure that you do not delete
- WAL files until they are no longer needed by any of the servers.
- <varname>archive_cleanup_command</> would typically be used in a
- warm-standby configuration (see <xref linkend="warm-standby">).
- Write <literal>%%</> to embed an actual <literal>%</> character in the
- command.
- </para>
- <para>
- If the command returns a non-zero exit status then a WARNING log
- message will be written.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-end-command" xreflabel="recovery_end_command">
- <term><varname>recovery_end_command</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>recovery_end_command</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies a shell command that will be executed once only
- at the end of recovery. This parameter is optional. The purpose of the
- <varname>recovery_end_command</> is to provide a mechanism for cleanup
- following replication or recovery.
- Any <literal>%r</> is replaced by the name of the file containing the
- last valid restart point, like in <xref linkend="archive-cleanup-command">.
- </para>
- <para>
- If the command returns a non-zero exit status then a WARNING log
- message will be written and the database will proceed to start up
- anyway. An exception is that if the command was terminated by a
- signal, the database will not proceed with startup.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
-
- </sect1>
-
- <sect1 id="recovery-target-settings">
-
- <title>Recovery Target Settings</title>
- <variablelist>
-
- <varlistentry id="recovery-target-name" xreflabel="recovery_target_name">
- <term><varname>recovery_target_name</varname>
- (<type>string</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_name</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the named restore point, created with
- <function>pg_create_restore_point()</> to which recovery will proceed.
- At most one of <varname>recovery_target_name</>,
- <xref linkend="recovery-target-time"> or
- <xref linkend="recovery-target-xid"> can be specified. The default is to
- recover to the end of the WAL log.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-time" xreflabel="recovery_target_time">
- <term><varname>recovery_target_time</varname>
- (<type>timestamp</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_time</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the time stamp up to which recovery
- will proceed.
- At most one of <varname>recovery_target_time</>,
- <xref linkend="recovery-target-name"> or
- <xref linkend="recovery-target-xid"> can be specified.
- The default is to recover to the end of the WAL log.
- The precise stopping point is also influenced by
- <xref linkend="recovery-target-inclusive">.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-xid" xreflabel="recovery_target_xid">
- <term><varname>recovery_target_xid</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>recovery_target_xid</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This parameter specifies the transaction ID up to which recovery
- will proceed. Keep in mind
- that while transaction IDs are assigned sequentially at transaction
- start, transactions can complete in a different numeric order.
- The transactions that will be recovered are those that committed
- before (and optionally including) the specified one.
- At most one of <varname>recovery_target_xid</>,
- <xref linkend="recovery-target-name"> or
- <xref linkend="recovery-target-time"> can be specified.
- The default is to recover to the end of the WAL log.
- The precise stopping point is also influenced by
- <xref linkend="recovery-target-inclusive">.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-inclusive"
- xreflabel="recovery_target_inclusive">
- <term><varname>recovery_target_inclusive</varname>
- (<type>boolean</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_inclusive</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether we stop just after the specified recovery target
- (<literal>true</literal>), or just before the recovery target
- (<literal>false</literal>).
- Applies to both <xref linkend="recovery-target-time">
- and <xref linkend="recovery-target-xid">, whichever one is
- specified for this recovery. This indicates whether transactions
- having exactly the target commit time or ID, respectively, will
- be included in the recovery. Default is <literal>true</>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="recovery-target-timeline"
- xreflabel="recovery_target_timeline">
- <term><varname>recovery_target_timeline</varname>
- (<type>string</type>)
- </term>
- <indexterm>
- <primary><varname>recovery_target_timeline</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies recovering into a particular timeline. The default is
- to recover along the same timeline that was current when the
- base backup was taken. Setting this to <literal>latest</> recovers
- to the latest timeline found in the archive, which is useful in
- a standby server. Other than that you only need to set this parameter
- in complex re-recovery situations, where you need to return to
- a state that itself was reached after a point-in-time recovery.
- See <xref linkend="backup-timelines"> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="pause-at-recovery-target"
- xreflabel="pause_at_recovery_target">
- <term><varname>pause_at_recovery_target</varname>
- (<type>boolean</type>)
- </term>
- <indexterm>
- <primary><varname>pause_at_recovery_target</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether recovery should pause when the recovery target
- is reached. The default is true.
- This is intended to allow queries to be executed against the
- database to check if this recovery target is the most desirable
- point for recovery. The paused state can be resumed by using
- <function>pg_xlog_replay_resume()</> (See
- <xref linkend="functions-recovery-control-table">), which then
- causes recovery to end. If this recovery target is not the
- desired stopping point, then shutdown the server, change the
- recovery target settings to a later target and restart to
- continue recovery.
- </para>
- <para>
- This setting has no effect if <xref linkend="guc-hot-standby"> is not
- enabled, or if no recovery target is set.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect1>
-
- <sect1 id="standby-settings">
-
- <title>Standby Server Settings</title>
- <variablelist>
-
- <varlistentry id="standby-mode" xreflabel="standby_mode">
- <term><varname>standby_mode</varname> (<type>boolean</type>)</term>
- <indexterm>
- <primary><varname>standby_mode</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies whether to start the <productname>PostgreSQL</> server as
- a standby. If this parameter is <literal>on</>, the server will
- not stop recovery when the end of archived WAL is reached, but
- will keep trying to continue recovery by fetching new WAL segments
- using <varname>restore_command</>
- and/or by connecting to the primary server as specified by the
- <varname>primary_conninfo</> setting.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry id="primary-conninfo" xreflabel="primary_conninfo">
- <term><varname>primary_conninfo</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>primary_conninfo</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies a connection string to be used for the standby server
- to connect with the primary. This string is in the format
- accepted by the libpq <function>PQconnectdb</function> function,
- described in <xref linkend="libpq-connect">. If any option is
- unspecified in this string, then the corresponding environment
- variable (see <xref linkend="libpq-envars">) is checked. If the
- environment variable is not set either, then
- defaults are used.
- </para>
- <para>
- The connection string should specify the host name (or address)
- of the primary server, as well as the port number if it is not
- the same as the standby server's default.
- Also specify a user name corresponding to a role that has the
- <literal>REPLICATION</> and <literal>LOGIN</> privileges on the
- primary (see
- <xref linkend="streaming-replication-authentication">).
- A password needs to be provided too, if the primary demands password
- authentication. It can be provided in the
- <varname>primary_conninfo</varname> string, or in a separate
- <filename>~/.pgpass</> file on the standby server (use
- <literal>replication</> as the database name).
- Do not specify a database name in the
- <varname>primary_conninfo</varname> string.
- </para>
- <para>
- This setting has no effect if <varname>standby_mode</> is <literal>off</>.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry id="trigger-file" xreflabel="trigger_file">
- <term><varname>trigger_file</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>trigger_file</> recovery parameter</primary>
- </indexterm>
- <listitem>
- <para>
- Specifies a trigger file whose presence ends recovery in the
- standby. Even if this value is not set, you can still promote
- the standby using <command>pg_ctl promote</>.
- This setting has no effect if <varname>standby_mode</> is <literal>off</>.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </sect1>
-
- </chapter>
--- 0 ----
*** a/doc/src/sgml/release-9.1.sgml
--- b/doc/src/sgml/release-9.1.sgml
***************
*** 1078,1084 ****
<listitem>
<para>
Add <filename>recovery.conf</> setting <link
! linkend="pause-at-recovery-target"><varname>pause_at_recovery_target</></link>
to pause recovery at target (Simon Riggs)
</para>
--- 1078,1084 ----
<listitem>
<para>
Add <filename>recovery.conf</> setting <link
! linkend="guc-pause-at-recovery-target"><varname>pause_at_recovery_target</></link>
to pause recovery at target (Simon Riggs)
</para>
***************
*** 1098,1104 ****
<para>
These named restore points can be specified as recovery
targets using the new <filename>recovery.conf</> setting
! <link linkend="recovery-target-name"><varname>recovery_target_name</></link>.
</para>
</listitem>
--- 1098,1104 ----
<para>
These named restore points can be specified as recovery
targets using the new <filename>recovery.conf</> setting
! <link linkend="guc-recovery-target-name"><varname>recovery_target_name</></link>.
</para>
</listitem>
***************
*** 1130,1137 ****
<listitem>
<para>
! Allow <link
! linkend="recovery-config"><filename>recovery.conf</></link>
to use the same quoting behavior as <filename>postgresql.conf</>
(Dimitri Fontaine)
</para>
--- 1130,1136 ----
<listitem>
<para>
! Allow <filename>recovery.conf</>
to use the same quoting behavior as <filename>postgresql.conf</>
(Dimitri Fontaine)
</para>
*** a/doc/src/sgml/release.sgml
--- b/doc/src/sgml/release.sgml
***************
*** 5,12 **** Typical markup:
&<> use & escapes
PostgreSQL <productname>
! postgresql.conf, pg_hba.conf,
! recovery.conf <filename>
[A-Z][A-Z_ ]+[A-Z_] <command>, <literal>, <envar>
[A-Za-z_][A-Za-z0-9_]+() <function>
-[-A-Za-z_]+ <option>
--- 5,12 ----
&<> use & escapes
PostgreSQL <productname>
! postgresql.conf, pg_hba.conf
! recovery.trigger <filename>
[A-Z][A-Z_ ]+[A-Z_] <command>, <literal>, <envar>
[A-Za-z_][A-Za-z0-9_]+() <function>
-[-A-Za-z_]+ <option>
*** a/src/backend/access/transam/recovery.conf.sample
--- b/src/backend/access/transam/recovery.conf.sample
***************
*** 2,24 ****
# PostgreSQL recovery config file
# -------------------------------
#
! # Edit this file to provide the parameters that PostgreSQL needs to
! # perform an archive recovery of a database, or to act as a replication
! # standby.
#
! # If "recovery.conf" is present in the PostgreSQL data directory, it is
! # read on postmaster startup. After successful recovery, it is renamed
! # to "recovery.done" to ensure that we do not accidentally re-enter
! # archive recovery or standby mode.
#
! # This file consists of lines of the form:
! #
! # name = value
! #
! # Comments are introduced with '#'.
! #
! # The complete list of option names and allowed values can be found
! # in the PostgreSQL documentation.
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
--- 2,15 ----
# PostgreSQL recovery config file
# -------------------------------
#
! # PostgreSQL 9.2 or later, recovery.conf is no longer used. Instead,
! # specify all recovery parameters in postgresql.conf and create
! # recovery.trigger to enter archive recovery or standby mode.
#
! # If you must use recovery.conf, specify "include directives" in
! # postgresql.conf like this:
#
! # include 'recovery.conf'
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 63,69 ****
/* File path names (all relative to $PGDATA) */
! #define RECOVERY_COMMAND_FILE "recovery.conf"
#define RECOVERY_COMMAND_DONE "recovery.done"
#define PROMOTE_SIGNAL_FILE "promote"
--- 63,69 ----
/* File path names (all relative to $PGDATA) */
! #define RECOVERY_COMMAND_READY "recovery.trigger"
#define RECOVERY_COMMAND_DONE "recovery.done"
#define PROMOTE_SIGNAL_FILE "promote"
***************
*** 80,85 **** bool fullPageWrites = true;
--- 80,99 ----
bool log_checkpoints = false;
int sync_method = DEFAULT_SYNC_METHOD;
int wal_level = WAL_LEVEL_MINIMAL;
+ char *restore_command = NULL;
+ char *archive_cleanup_command = NULL;
+ char *recovery_end_command = NULL;
+ bool standby_mode = false;
+ char *primary_conninfo = NULL;
+ char *trigger_file = NULL;
+ RecoveryTargetType recovery_target = RECOVERY_TARGET_UNSET;
+ TransactionId recovery_target_xid = InvalidTransactionId;
+ TimestampTz recovery_target_time = 0;
+ char *recovery_target_name = NULL;
+ bool recovery_target_inclusive = true;
+ bool pause_at_recovery_target = true;
+ char *recovery_target_timeline_string = NULL;
+ TimeLineID recovery_target_timeline = 0;
#ifdef WAL_DEBUG
bool XLOG_DEBUG = false;
***************
*** 186,208 **** static bool InArchiveRecovery = false;
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* options taken from recovery.conf for archive recovery */
! static char *recoveryRestoreCommand = NULL;
! static char *recoveryEndCommand = NULL;
! static char *archiveCleanupCommand = NULL;
! static RecoveryTargetType recoveryTarget = RECOVERY_TARGET_UNSET;
! static bool recoveryTargetInclusive = true;
! static bool recoveryPauseAtTarget = true;
! static TransactionId recoveryTargetXid;
! static TimestampTz recoveryTargetTime;
! static char *recoveryTargetName;
!
! /* options taken from recovery.conf for XLOG streaming */
! static bool StandbyMode = false;
! static char *PrimaryConnInfo = NULL;
! static char *TriggerFile = NULL;
!
! /* if recoveryStopsHere returns true, it saves actual stop xid/time/name here */
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
--- 200,207 ----
/* Was the last xlog file restored from archive, or local? */
static bool restoredFromArchive = false;
! /* if recoveryStopsHere returns true, it saves actual stop type/xid/time/name here */
! static RecoveryTargetType recoveryStopTarget;
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
static char recoveryStopName[MAXFNAMELEN];
***************
*** 408,419 **** typedef struct XLogCtlData
TimeLineID RecoveryTargetTLI;
/*
- * archiveCleanupCommand is read from recovery.conf but needs to be in
- * shared memory so that the bgwriter process can access it.
- */
- char archiveCleanupCommand[MAXPGPATH];
-
- /*
* SharedRecoveryInProgress indicates if we're still in crash or archive
* recovery. Protected by info_lck.
*/
--- 407,412 ----
***************
*** 602,608 **** static void XLogArchiveNotifySeg(uint32 log, uint32 seg);
static bool XLogArchiveCheckDone(const char *xlog);
static bool XLogArchiveIsBusy(const char *xlog);
static void XLogArchiveCleanup(const char *xlog);
! static void readRecoveryCommandFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
--- 595,601 ----
static bool XLogArchiveCheckDone(const char *xlog);
static bool XLogArchiveIsBusy(const char *xlog);
static void XLogArchiveCleanup(const char *xlog);
! static void CheckRecoveryReadyFile(void);
static void exitArchiveRecovery(TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
***************
*** 612,617 **** static void SetRecoveryPause(bool recoveryPause);
--- 605,611 ----
static void SetLatestXTime(TimestampTz xtime);
static TimestampTz GetLatestXTime(void);
static void CheckRequiredParameterValues(void);
+ static void CheckRestoreCommandSet(void);
static void XLogReportParameters(void);
static void LocalSetXLogInsertAllowed(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
***************
*** 2932,2938 **** RestoreArchivedFile(char *path, const char *xlogfname,
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (recoveryRestoreCommand == NULL)
goto not_available;
/*
--- 2926,2932 ----
uint32 restartSeg;
/* In standby mode, restore_command might not be supplied */
! if (!restore_command[0])
goto not_available;
/*
***************
*** 2952,2958 **** RestoreArchivedFile(char *path, const char *xlogfname,
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the recoveryRestoreCommand may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
--- 2946,2952 ----
* of log segments that weren't yet transferred to the archive.
*
* Notice that we don't actually overwrite any files when we copy back
! * from archive because the restore_command may inadvertently
* restore inappropriate xlogs, or they may be corrupt, so we may wish to
* fallback to the segments remaining in current XLOGDIR later. The
* copy-from-archive filename is always the same, ensuring that we don't
***************
*** 3016,3022 **** RestoreArchivedFile(char *path, const char *xlogfname,
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = recoveryRestoreCommand; *sp; sp++)
{
if (*sp == '%')
{
--- 3010,3016 ----
endp = xlogRestoreCmd + MAXPGPATH - 1;
*endp = '\0';
! for (sp = restore_command; *sp; sp++)
{
if (*sp == '%')
{
***************
*** 3107,3113 **** RestoreArchivedFile(char *path, const char *xlogfname,
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (StandbyMode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
--- 3101,3107 ----
* incorrectly conclude we've reached the end of WAL and we're
* done recovering ...
*/
! if (standby_mode && stat_buf.st_size < expectedSize)
elevel = DEBUG1;
else
elevel = FATAL;
***************
*** 3283,3289 **** ExecuteRecoveryCommand(char *command, char *commandName, bool failOnSignal)
ereport((signaled && failOnSignal) ? FATAL : WARNING,
/*------
! translator: First %s represents a recovery.conf parameter name like
"recovery_end_command", and the 2nd is the value of that parameter. */
(errmsg("%s \"%s\": return code %d", commandName,
command, rc)));
--- 3277,3283 ----
ereport((signaled && failOnSignal) ? FATAL : WARNING,
/*------
! translator: First %s represents a recovery parameter name like
"recovery_end_command", and the 2nd is the value of that parameter. */
(errmsg("%s \"%s\": return code %d", commandName,
command, rc)));
***************
*** 4064,4070 **** next_record_is_invalid:
}
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return NULL;
--- 4058,4064 ----
}
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return NULL;
***************
*** 4523,4529 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
--- 4517,4523 ----
* Write comment to history file to explain why and where timeline
* changed. Comment varies according to the recovery target used.
*/
! if (recoveryStopTarget == RECOVERY_TARGET_XID)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s transaction %u\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4531,4537 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
--- 4525,4531 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
! else if (recoveryStopTarget == RECOVERY_TARGET_TIME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\t%s %s\n",
(srcfd < 0) ? "" : "\n",
***************
*** 4539,4545 **** writeTimeLineHistory(TimeLineID newTLI, TimeLineID parentTLI,
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
--- 4533,4539 ----
xlogfname,
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
! else if (recoveryStopTarget == RECOVERY_TARGET_NAME)
snprintf(buffer, sizeof(buffer),
"%s%u\t%s\tat restore point \"%s\"\n",
(srcfd < 0) ? "" : "\n",
***************
*** 5268,5500 **** str_time(pg_time_t tnow)
}
/*
! * See if there is a recovery command file (recovery.conf), and if so
! * read in parameters for archive recovery and XLOG streaming.
! *
! * The file is parsed using the main configuration parser.
*/
static void
! readRecoveryCommandFile(void)
{
! FILE *fd;
! TimeLineID rtli = 0;
! bool rtliGiven = false;
! ConfigVariable *item,
! *head = NULL,
! *tail = NULL;
!
! fd = AllocateFile(RECOVERY_COMMAND_FILE, "r");
! if (fd == NULL)
! {
! if (errno == ENOENT)
! return; /* not there, so no archive recovery */
! ereport(FATAL,
! (errcode_for_file_access(),
! errmsg("could not open recovery command file \"%s\": %m",
! RECOVERY_COMMAND_FILE)));
! }
!
! /*
! * Since we're asking ParseConfigFp() to report errors as FATAL, there's
! * no need to check the return value.
! */
! (void) ParseConfigFp(fd, RECOVERY_COMMAND_FILE, 0, FATAL, &head, &tail);
!
! FreeFile(fd);
!
! for (item = head; item; item = item->next)
! {
! if (strcmp(item->name, "restore_command") == 0)
! {
! recoveryRestoreCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("restore_command = '%s'",
! recoveryRestoreCommand)));
! }
! else if (strcmp(item->name, "recovery_end_command") == 0)
! {
! recoveryEndCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("recovery_end_command = '%s'",
! recoveryEndCommand)));
! }
! else if (strcmp(item->name, "archive_cleanup_command") == 0)
! {
! archiveCleanupCommand = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("archive_cleanup_command = '%s'",
! archiveCleanupCommand)));
! }
! else if (strcmp(item->name, "pause_at_recovery_target") == 0)
! {
! if (!parse_bool(item->value, &recoveryPauseAtTarget))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value", "pause_at_recovery_target")));
! ereport(DEBUG2,
! (errmsg_internal("pause_at_recovery_target = '%s'",
! item->value)));
! }
! else if (strcmp(item->name, "recovery_target_timeline") == 0)
! {
! rtliGiven = true;
! if (strcmp(item->value, "latest") == 0)
! rtli = 0;
! else
! {
! errno = 0;
! rtli = (TimeLineID) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_timeline is not a valid number: \"%s\"",
! item->value)));
! }
! if (rtli)
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = %u", rtli)));
! else
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_timeline = latest")));
! }
! else if (strcmp(item->name, "recovery_target_xid") == 0)
! {
! errno = 0;
! recoveryTargetXid = (TransactionId) strtoul(item->value, NULL, 0);
! if (errno == EINVAL || errno == ERANGE)
! ereport(FATAL,
! (errmsg("recovery_target_xid is not a valid number: \"%s\"",
! item->value)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_xid = %u",
! recoveryTargetXid)));
! recoveryTarget = RECOVERY_TARGET_XID;
! }
! else if (strcmp(item->name, "recovery_target_time") == 0)
! {
! /*
! * if recovery_target_xid or recovery_target_name specified, then
! * this overrides recovery_target_time
! */
! if (recoveryTarget == RECOVERY_TARGET_XID ||
! recoveryTarget == RECOVERY_TARGET_NAME)
! continue;
! recoveryTarget = RECOVERY_TARGET_TIME;
!
! /*
! * Convert the time string given by the user to TimestampTz form.
! */
! recoveryTargetTime =
! DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
! CStringGetDatum(item->value),
! ObjectIdGetDatum(InvalidOid),
! Int32GetDatum(-1)));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_time = '%s'",
! timestamptz_to_str(recoveryTargetTime))));
! }
! else if (strcmp(item->name, "recovery_target_name") == 0)
! {
! /*
! * if recovery_target_xid specified, then this overrides
! * recovery_target_name
! */
! if (recoveryTarget == RECOVERY_TARGET_XID)
! continue;
! recoveryTarget = RECOVERY_TARGET_NAME;
!
! recoveryTargetName = pstrdup(item->value);
! if (strlen(recoveryTargetName) >= MAXFNAMELEN)
! ereport(FATAL,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("recovery_target_name is too long (maximum %d characters)",
! MAXFNAMELEN - 1)));
!
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_name = '%s'",
! recoveryTargetName)));
! }
! else if (strcmp(item->name, "recovery_target_inclusive") == 0)
! {
! /*
! * does nothing if a recovery_target is not also set
! */
! if (!parse_bool(item->value, &recoveryTargetInclusive))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "recovery_target_inclusive")));
! ereport(DEBUG2,
! (errmsg_internal("recovery_target_inclusive = %s",
! item->value)));
! }
! else if (strcmp(item->name, "standby_mode") == 0)
! {
! if (!parse_bool(item->value, &StandbyMode))
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("parameter \"%s\" requires a Boolean value",
! "standby_mode")));
! ereport(DEBUG2,
! (errmsg_internal("standby_mode = '%s'", item->value)));
! }
! else if (strcmp(item->name, "primary_conninfo") == 0)
! {
! PrimaryConnInfo = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("primary_conninfo = '%s'",
! PrimaryConnInfo)));
! }
! else if (strcmp(item->name, "trigger_file") == 0)
! {
! TriggerFile = pstrdup(item->value);
! ereport(DEBUG2,
! (errmsg_internal("trigger_file = '%s'",
! TriggerFile)));
! }
! else
! ereport(FATAL,
! (errmsg("unrecognized recovery parameter \"%s\"",
! item->name)));
! }
! /*
! * Check for compulsory parameters
! */
! if (StandbyMode)
! {
! if (PrimaryConnInfo == NULL && recoveryRestoreCommand == NULL)
! ereport(WARNING,
! (errmsg("recovery command file \"%s\" specified neither primary_conninfo nor restore_command",
! RECOVERY_COMMAND_FILE),
! errhint("The database server will regularly poll the pg_xlog subdirectory to check for files placed there.")));
! }
! else
! {
! if (recoveryRestoreCommand == NULL)
! ereport(FATAL,
! (errmsg("recovery command file \"%s\" must specify restore_command when standby mode is not enabled",
! RECOVERY_COMMAND_FILE)));
! }
/* Enable fetching from archive recovery area */
InArchiveRecovery = true;
/*
* If user specified recovery_target_timeline, validate it or compute the
* "latest" value. We can't do this until after we've gotten the restore
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (rtliGiven)
{
! if (rtli)
{
/* Timeline 1 does not have a history file, all else should */
! if (rtli != 1 && !existsTimeLineHistory(rtli))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! rtli)));
! recoveryTargetTLI = rtli;
recoveryTargetIsLatest = false;
}
else
--- 5262,5305 ----
}
/*
! * Check to see if there is a recovery trigger file (recovery.trigger), and if so
! * validate recovery parameters and determine recovery target timeline
*/
static void
! CheckRecoveryReadyFile(void)
{
! struct stat stat_buf;
! if (stat(RECOVERY_COMMAND_READY, &stat_buf) != 0)
! return; /* not there, so no archive recovery */
/* Enable fetching from archive recovery area */
InArchiveRecovery = true;
+ /* Check for compulsory parameters */
+ if (standby_mode && !restore_command[0] && !primary_conninfo[0])
+ ereport(WARNING,
+ (errmsg("neither primary_conninfo nor restore_command is specified"),
+ errhint("The database server will regularly poll the pg_xlog subdirectory to "
+ "check for files placed there until either of them is set in postgresql.conf.")));
+ CheckRestoreCommandSet();
+
/*
* If user specified recovery_target_timeline, validate it or compute the
* "latest" value. We can't do this until after we've gotten the restore
* command and set InArchiveRecovery, because we need to fetch timeline
* history files from the archive.
*/
! if (strcmp(recovery_target_timeline_string, "") != 0)
{
! if (recovery_target_timeline)
{
/* Timeline 1 does not have a history file, all else should */
! if (recovery_target_timeline != 1 && !existsTimeLineHistory(recovery_target_timeline))
ereport(FATAL,
(errmsg("recovery target timeline %u does not exist",
! recovery_target_timeline)));
! recoveryTargetTLI = recovery_target_timeline;
recoveryTargetIsLatest = false;
}
else
***************
*** 5504,5511 **** readRecoveryCommandFile(void)
recoveryTargetIsLatest = true;
}
}
-
- FreeConfigVariables(head);
}
/*
--- 5309,5314 ----
***************
*** 5581,5591 **** exitArchiveRecovery(TimeLineID endTLI, uint32 endLogId, uint32 endLogSeg)
* re-enter archive recovery mode in a subsequent crash.
*/
unlink(RECOVERY_COMMAND_DONE);
! if (rename(RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE) != 0)
ereport(FATAL,
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
! RECOVERY_COMMAND_FILE, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
--- 5384,5394 ----
* re-enter archive recovery mode in a subsequent crash.
*/
unlink(RECOVERY_COMMAND_DONE);
! if (rename(RECOVERY_COMMAND_READY, RECOVERY_COMMAND_DONE) != 0)
ereport(FATAL,
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
! RECOVERY_COMMAND_READY, RECOVERY_COMMAND_DONE)));
ereport(LOG,
(errmsg("archive recovery complete")));
***************
*** 5648,5654 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
/* Do we have a PITR target at all? */
! if (recoveryTarget == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
--- 5451,5457 ----
return false;
/* Do we have a PITR target at all? */
! if (recovery_target == RECOVERY_TARGET_UNSET)
{
/*
* Save timestamp of latest transaction commit/abort if this is a
***************
*** 5659,5665 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
return false;
}
! if (recoveryTarget == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
--- 5462,5468 ----
return false;
}
! if (recovery_target == RECOVERY_TARGET_XID)
{
/*
* There can be only one transaction end record with this exact
***************
*** 5670,5689 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recoveryTargetXid);
if (stopsHere)
! *includeThis = recoveryTargetInclusive;
}
! else if (recoveryTarget == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recoveryTargetName) == 0);
/*
! * Ignore recoveryTargetInclusive because this is not a transaction
* record
*/
*includeThis = false;
--- 5473,5492 ----
* they complete. A higher numbered xid will complete before you about
* 50% of the time...
*/
! stopsHere = (record->xl_xid == recovery_target_xid);
if (stopsHere)
! *includeThis = recovery_target_inclusive;
}
! else if (recovery_target == RECOVERY_TARGET_NAME)
{
/*
* There can be many restore points that share the same name, so we
* stop at the first one
*/
! stopsHere = (strcmp(recordRPName, recovery_target_name) == 0);
/*
! * Ignore recovery_target_inclusive because this is not a transaction
* record
*/
*includeThis = false;
***************
*** 5695,5710 **** recoveryStopsHere(XLogRecord *record, bool *includeThis)
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recoveryTargetInclusive)
! stopsHere = (recordXtime > recoveryTargetTime);
else
! stopsHere = (recordXtime >= recoveryTargetTime);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
--- 5498,5514 ----
* we stop after the last one, if we are inclusive, or stop at the
* first one if we are exclusive
*/
! if (recovery_target_inclusive)
! stopsHere = (recordXtime > recovery_target_time);
else
! stopsHere = (recordXtime >= recovery_target_time);
if (stopsHere)
*includeThis = false;
}
if (stopsHere)
{
+ recoveryStopTarget = recovery_target;
recoveryStopXid = record->xl_xid;
recoveryStopTime = recordXtime;
recoveryStopAfter = *includeThis;
***************
*** 6005,6010 **** CheckRequiredParameterValues(void)
--- 5809,5827 ----
}
/*
+ * Check to see if restore_command must be set for archive recovery
+ * when standby mode is not enabled
+ */
+ static void
+ CheckRestoreCommandSet(void)
+ {
+ if (InArchiveRecovery && !standby_mode && !restore_command[0])
+ ereport(FATAL,
+ (errmsg("restore_command must be specified for archive recovery "
+ "when standby mode is not enabled")));
+ }
+
+ /*
* This must be called ONCE during postmaster or standalone-backend startup
*/
void
***************
*** 6098,6107 **** StartupXLOG(void)
recoveryTargetTLI = ControlFile->checkPointCopy.ThisTimeLineID;
/*
! * Check for recovery control file, and if so set up state for offline
* recovery
*/
! readRecoveryCommandFile();
/* Now we can determine the list of expected TLIs */
expectedTLIs = readTimeLineHistory(recoveryTargetTLI);
--- 5915,5924 ----
recoveryTargetTLI = ControlFile->checkPointCopy.ThisTimeLineID;
/*
! * Check for recovery trigger file, and if so set up state for offline
* recovery
*/
! CheckRecoveryReadyFile();
/* Now we can determine the list of expected TLIs */
expectedTLIs = readTimeLineHistory(recoveryTargetTLI);
***************
*** 6119,6150 **** StartupXLOG(void)
ControlFile->checkPointCopy.ThisTimeLineID)));
/*
! * Save the selected recovery target timeline ID and
! * archive_cleanup_command in shared memory so that other processes can
! * see them
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
- strncpy(XLogCtl->archiveCleanupCommand,
- archiveCleanupCommand ? archiveCleanupCommand : "",
- sizeof(XLogCtl->archiveCleanupCommand));
if (InArchiveRecovery)
{
! if (StandbyMode)
ereport(LOG,
(errmsg("entering standby mode")));
- else if (recoveryTarget == RECOVERY_TARGET_XID)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to XID %u",
- recoveryTargetXid)));
- else if (recoveryTarget == RECOVERY_TARGET_TIME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to %s",
- timestamptz_to_str(recoveryTargetTime))));
- else if (recoveryTarget == RECOVERY_TARGET_NAME)
- ereport(LOG,
- (errmsg("starting point-in-time recovery to \"%s\"",
- recoveryTargetName)));
else
ereport(LOG,
(errmsg("starting archive recovery")));
--- 5936,5951 ----
ControlFile->checkPointCopy.ThisTimeLineID)));
/*
! * Save the selected recovery target timeline ID in shared memory
! * so that other processes can see it
*/
XLogCtl->RecoveryTargetTLI = recoveryTargetTLI;
if (InArchiveRecovery)
{
! if (standby_mode)
ereport(LOG,
(errmsg("entering standby mode")));
else
ereport(LOG,
(errmsg("starting archive recovery")));
***************
*** 6154,6160 **** StartupXLOG(void)
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (StandbyMode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
--- 5955,5961 ----
* Take ownership of the wakeup latch if we're going to sleep during
* recovery.
*/
! if (standby_mode)
OwnLatch(&XLogCtl->recoveryWakeupLatch);
if (read_backup_label(&checkPointLoc, &backupEndRequired))
***************
*** 6212,6218 **** StartupXLOG(void)
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (StandbyMode)
{
/*
* The last valid checkpoint record required for a streaming
--- 6013,6019 ----
(errmsg("checkpoint record is at %X/%X",
checkPointLoc.xlogid, checkPointLoc.xrecoff)));
}
! else if (standby_mode)
{
/*
* The last valid checkpoint record required for a streaming
***************
*** 6281,6287 **** StartupXLOG(void)
/*
* Check whether we need to force recovery from WAL. If it appears to
! * have been a clean shutdown and we did not have a recovery.conf file,
* then assume no recovery needed.
*/
if (XLByteLT(checkPoint.redo, RecPtr))
--- 6082,6088 ----
/*
* Check whether we need to force recovery from WAL. If it appears to
! * have been a clean shutdown and we did not have a recovery.trigger file,
* then assume no recovery needed.
*/
if (XLByteLT(checkPoint.redo, RecPtr))
***************
*** 6295,6301 **** StartupXLOG(void)
InRecovery = true;
else if (InArchiveRecovery)
{
! /* force recovery due to presence of recovery.conf */
InRecovery = true;
}
--- 6096,6102 ----
InRecovery = true;
else if (InArchiveRecovery)
{
! /* force recovery due to presence of recovery.trigger */
InRecovery = true;
}
***************
*** 6565,6571 **** StartupXLOG(void)
* Pause only if users can connect to send a resume
* message
*/
! if (recoveryPauseAtTarget && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
--- 6366,6372 ----
* Pause only if users can connect to send a resume
* message
*/
! if (pause_at_recovery_target && standbyState == STANDBY_SNAPSHOT_READY)
{
SetRecoveryPause(true);
recoveryPausesHere();
***************
*** 6664,6670 **** StartupXLOG(void)
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (StandbyMode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
--- 6465,6471 ----
* We don't need the latch anymore. It's not strictly necessary to disown
* it, but let's do it for the sake of tidiness.
*/
! if (standby_mode)
DisownLatch(&XLogCtl->recoveryWakeupLatch);
/*
***************
*** 6672,6678 **** StartupXLOG(void)
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! StandbyMode = false;
/*
* Re-fetch the last valid or last applied record, so we can identify the
--- 6473,6479 ----
* recovery to force fetching the files (which would be required at end of
* recovery, e.g., timeline history file) from archive or pg_xlog.
*/
! SetConfigOption("standby_mode", "false", PGC_POSTMASTER, PGC_S_OVERRIDE);
/*
* Re-fetch the last valid or last applied record, so we can identify the
***************
*** 6865,6872 **** StartupXLOG(void)
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recoveryEndCommand)
! ExecuteRecoveryCommand(recoveryEndCommand,
"recovery_end_command",
true);
}
--- 6666,6673 ----
/*
* And finally, execute the recovery_end_command, if any.
*/
! if (recovery_end_command[0])
! ExecuteRecoveryCommand(recovery_end_command,
"recovery_end_command",
true);
}
***************
*** 8189,8196 **** CreateRestartPoint(int flags)
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (XLogCtl->archiveCleanupCommand[0])
! ExecuteRecoveryCommand(XLogCtl->archiveCleanupCommand,
"archive_cleanup_command",
false);
--- 7990,7997 ----
/*
* Finally, execute archive_cleanup_command, if any.
*/
! if (archive_cleanup_command[0])
! ExecuteRecoveryCommand(archive_cleanup_command,
"archive_cleanup_command",
false);
***************
*** 10021,10026 **** HandleStartupProcInterrupts(void)
--- 9822,9830 ----
{
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /* Check for compulsory parameter */
+ CheckRestoreCommandSet();
}
/*
***************
*** 10145,10151 **** XLogPageRead(XLogRecPtr *RecPtr, int emode, bool fetching_ckpt,
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (StandbyMode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
--- 9949,9955 ----
* Signal bgwriter to start a restartpoint if we've replayed too much
* xlog since the last one.
*/
! if (standby_mode && bgwriterLaunched)
{
if (XLogCheckpointNeeded(readId, readSeg))
{
***************
*** 10167,10173 **** retry:
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (StandbyMode)
{
/*
* In standby mode, wait for the requested record to become
--- 9971,9977 ----
if (readFile < 0 ||
(readSource == XLOG_FROM_STREAM && !XLByteLT(*RecPtr, receivedUpto)))
{
! if (standby_mode)
{
/*
* In standby mode, wait for the requested record to become
***************
*** 10331,10341 **** retry:
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (PrimaryConnInfo)
{
RequestXLogStreaming(
! fetching_ckpt ? RedoStartLSN : *RecPtr,
! PrimaryConnInfo);
continue;
}
}
--- 10135,10144 ----
* backwards to start redo at RedoStartLSN, we will
* have the logs streamed already.
*/
! if (primary_conninfo[0])
{
RequestXLogStreaming(
! fetching_ckpt ? RedoStartLSN : *RecPtr);
continue;
}
}
***************
*** 10478,10484 **** next_record_is_invalid:
readSource = 0;
/* In standby-mode, keep trying */
! if (StandbyMode)
goto retry;
else
return false;
--- 10281,10287 ----
readSource = 0;
/* In standby-mode, keep trying */
! if (standby_mode)
goto retry;
else
return false;
***************
*** 10550,10564 **** CheckForStandbyTrigger(void)
return true;
}
! if (TriggerFile == NULL)
return false;
! if (stat(TriggerFile, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", TriggerFile)));
ShutdownWalRcv();
! unlink(TriggerFile);
triggered = true;
return true;
}
--- 10353,10367 ----
return true;
}
! if (!trigger_file[0])
return false;
! if (stat(trigger_file, &stat_buf) == 0)
{
ereport(LOG,
! (errmsg("trigger file found: %s", trigger_file)));
ShutdownWalRcv();
! unlink(trigger_file);
triggered = true;
return true;
}
*** a/src/backend/commands/extension.c
--- b/src/backend/commands/extension.c
***************
*** 9,15 ****
* dependent objects can be associated with it. An extension is created by
* populating the pg_extension catalog from a "control" file.
* The extension control file is parsed with the same parser we use for
! * postgresql.conf and recovery.conf. An extension also has an installation
* script file, containing SQL commands to create the extension's objects.
*
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
--- 9,15 ----
* dependent objects can be associated with it. An extension is created by
* populating the pg_extension catalog from a "control" file.
* The extension control file is parsed with the same parser we use for
! * postgresql.conf. An extension also has an installation
* script file, containing SQL commands to create the extension's objects.
*
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
*** a/src/backend/replication/walreceiver.c
--- b/src/backend/replication/walreceiver.c
***************
*** 168,174 **** DisableWalRcvImmediateExit(void)
void
WalReceiverMain(void)
{
- char conninfo[MAXCONNINFO];
XLogRecPtr startpoint;
/* use volatile pointer to prevent code rearrangement */
--- 168,173 ----
***************
*** 216,222 **** WalReceiverMain(void)
walrcv->walRcvState = WALRCV_RUNNING;
/* Fetch information required to start streaming */
- strlcpy(conninfo, (char *) walrcv->conninfo, MAXCONNINFO);
startpoint = walrcv->receiveStart;
SpinLockRelease(&walrcv->mutex);
--- 215,220 ----
***************
*** 272,278 **** WalReceiverMain(void)
/* Establish the connection to the primary for XLOG streaming */
EnableWalRcvImmediateExit();
! walrcv_connect(conninfo, startpoint);
DisableWalRcvImmediateExit();
/* Loop until end-of-streaming or error */
--- 270,276 ----
/* Establish the connection to the primary for XLOG streaming */
EnableWalRcvImmediateExit();
! walrcv_connect(primary_conninfo, startpoint);
DisableWalRcvImmediateExit();
/* Loop until end-of-streaming or error */
***************
*** 302,309 **** WalReceiverMain(void)
--- 300,320 ----
if (got_SIGHUP)
{
+ char *conninfo = pstrdup(primary_conninfo);
+
got_SIGHUP = false;
ProcessConfigFile(PGC_SIGHUP);
+
+ /*
+ * If primary_conninfo has been changed while walreceiver is running,
+ * shut down walreceiver so that new walreceiver is started and
+ * it initiates replication with new primary_conninfo.
+ */
+ if (strcmp(conninfo, primary_conninfo) != 0)
+ ereport(FATAL,
+ (errcode(ERRCODE_ADMIN_SHUTDOWN),
+ errmsg("terminating walreceiver process because primary_conninfo was changed")));
+ pfree(conninfo);
}
/* Wait a while for data to arrive */
*** a/src/backend/replication/walreceiverfuncs.c
--- b/src/backend/replication/walreceiverfuncs.c
***************
*** 166,176 **** ShutdownWalRcv(void)
/*
* Request postmaster to start walreceiver.
*
! * recptr indicates the position where streaming should begin, and conninfo
! * is a libpq connection string to use.
*/
void
! RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo)
{
/* use volatile pointer to prevent code rearrangement */
volatile WalRcvData *walrcv = WalRcv;
--- 166,175 ----
/*
* Request postmaster to start walreceiver.
*
! * recptr indicates the position where streaming should begin.
*/
void
! RequestXLogStreaming(XLogRecPtr recptr)
{
/* use volatile pointer to prevent code rearrangement */
volatile WalRcvData *walrcv = WalRcv;
***************
*** 190,199 **** RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo)
/* It better be stopped before we try to restart it */
Assert(walrcv->walRcvState == WALRCV_STOPPED);
- if (conninfo != NULL)
- strlcpy((char *) walrcv->conninfo, conninfo, MAXCONNINFO);
- else
- walrcv->conninfo[0] = '\0';
walrcv->walRcvState = WALRCV_STARTING;
walrcv->startTime = now;
--- 189,194 ----
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 30,35 ****
--- 30,36 ----
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
+ #include "access/xlog_internal.h"
#include "catalog/namespace.h"
#include "commands/async.h"
#include "commands/prepare.h"
***************
*** 205,210 **** static bool check_application_name(char **newval, void **extra, GucSource source
--- 206,219 ----
static void assign_application_name(const char *newval, void *extra);
static const char *show_unix_socket_permissions(void);
static const char *show_log_file_mode(void);
+ static bool check_recovery_target_xid(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_xid(const char *newval, void *extra);
+ static bool check_recovery_target_name(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_name(const char *newval, void *extra);
+ static bool check_recovery_target_time(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_time(const char *newval, void *extra);
+ static bool check_recovery_target_timeline(char **newval, void **extra, GucSource source);
+ static void assign_recovery_target_timeline(const char *newval, void *extra);
static char *config_enum_get_options(struct config_enum * record,
const char *prefix, const char *suffix,
***************
*** 475,480 **** static int wal_block_size;
--- 484,491 ----
static int wal_segment_size;
static bool integer_datetimes;
static int effective_io_concurrency;
+ static char *recovery_target_xid_string;
+ static char *recovery_target_time_string;
/* should be static, but commands/variable.c needs to get at this */
char *role_string;
***************
*** 554,559 **** const char *const config_group_names[] =
--- 565,574 ----
gettext_noop("Write-Ahead Log / Checkpoints"),
/* WAL_ARCHIVING */
gettext_noop("Write-Ahead Log / Archiving"),
+ /* WAL_ARCHIVE_RECOVERY */
+ gettext_noop("Write-Ahead Log / Archive Recovery"),
+ /* WAL_RECOVERY_TARGET */
+ gettext_noop("Write-Ahead Log / Recovery Target"),
/* REPLICATION */
gettext_noop("Replication"),
/* REPLICATION_SENDING */
***************
*** 1373,1378 **** static struct config_bool ConfigureNamesBool[] =
--- 1388,1423 ----
},
{
+ {"recovery_target_inclusive", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether to include or exclude transaction with recovery target."),
+ NULL
+ },
+ &recovery_target_inclusive,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"pause_at_recovery_target", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets whether recovery should pause when the recovery target is reached."),
+ NULL
+ },
+ &pause_at_recovery_target,
+ true,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"standby_mode", PGC_POSTMASTER, REPLICATION_STANDBY,
+ gettext_noop("Sets whether to start the server as a standby."),
+ NULL
+ },
+ &standby_mode,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"hot_standby", PGC_POSTMASTER, REPLICATION_STANDBY,
gettext_noop("Allows connections and queries during recovery."),
NULL
***************
*** 2529,2534 **** static struct config_string ConfigureNamesString[] =
--- 2574,2670 ----
},
{
+ {"restore_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will retrieve an archived WAL file."),
+ NULL
+ },
+ &restore_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"archive_cleanup_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed at every restartpoint."),
+ NULL
+ },
+ &archive_cleanup_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_end_command", PGC_SIGHUP, WAL_ARCHIVE_RECOVERY,
+ gettext_noop("Sets the shell command that will be executed once only at the end of recovery."),
+ NULL
+ },
+ &recovery_end_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"recovery_target_xid", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the transaction ID up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_xid_string,
+ "",
+ check_recovery_target_xid, assign_recovery_target_xid, NULL
+ },
+
+ {
+ {"recovery_target_name", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the named restore point."),
+ NULL
+ },
+ &recovery_target_name,
+ "",
+ check_recovery_target_name, assign_recovery_target_name, NULL
+ },
+
+ {
+ {"recovery_target_time", PGC_SIGHUP, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets the time stamp up to which recovery will proceed."),
+ NULL
+ },
+ &recovery_target_time_string,
+ "",
+ check_recovery_target_time, assign_recovery_target_time, NULL
+ },
+
+ {
+ {"recovery_target_timeline", PGC_POSTMASTER, WAL_RECOVERY_TARGET,
+ gettext_noop("Sets recoverying into a particular timeline."),
+ NULL
+ },
+ &recovery_target_timeline_string,
+ "",
+ check_recovery_target_timeline, assign_recovery_target_timeline, NULL
+ },
+
+ {
+ {"primary_conninfo", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the connection string to be used to connect with the primary."),
+ NULL,
+ GUC_SUPERUSER_ONLY
+ },
+ &primary_conninfo,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"trigger_file", PGC_SIGHUP, REPLICATION_STANDBY,
+ gettext_noop("Sets the trigger file whose presence ends recovery in the standby."),
+ NULL
+ },
+ &trigger_file,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
{"client_encoding", PGC_USERSET, CLIENT_CONN_LOCALE,
gettext_noop("Sets the client's character set encoding."),
NULL,
***************
*** 8679,8682 **** show_log_file_mode(void)
--- 8815,8972 ----
return buf;
}
+ static bool
+ check_recovery_target_xid(char **newval, void **extra, GucSource source)
+ {
+ TransactionId xid;
+ TransactionId *myextra;
+
+ if (strcmp(*newval, "") == 0)
+ xid = InvalidTransactionId;
+ else
+ {
+ errno = 0;
+ xid = (TransactionId) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_xid is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+ }
+
+ myextra = (TransactionId *) guc_malloc(ERROR, sizeof(TransactionId));
+ *myextra = xid;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_xid(const char *newval, void *extra)
+ {
+ recovery_target_xid = *((TransactionId *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_name(char **newval, void **extra, GucSource source)
+ {
+ if (strlen(*newval) >= MAXFNAMELEN)
+ {
+ GUC_check_errdetail("\"recovery_target_name\" is too long (maximum %d characters)",
+ MAXFNAMELEN - 1);
+ return false;
+ }
+ return true;
+ }
+
+ static void
+ assign_recovery_target_name(const char *newval, void *extra)
+ {
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (newval[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_time(char **newval, void **extra, GucSource source)
+ {
+ TimestampTz time;
+ TimestampTz *myextra;
+ MemoryContext oldcontext = CurrentMemoryContext;
+
+ PG_TRY();
+ {
+ time = (strcmp(*newval, "") == 0) ?
+ 0 :
+ DatumGetTimestampTz(DirectFunctionCall3(timestamptz_in,
+ CStringGetDatum(*newval),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ }
+ PG_CATCH();
+ {
+ ErrorData *edata;
+
+ /* Save error info */
+ MemoryContextSwitchTo(oldcontext);
+ edata = CopyErrorData();
+ FlushErrorState();
+
+ /* Pass the error message */
+ GUC_check_errdetail("%s", edata->message);
+ FreeErrorData(edata);
+ return false;
+ }
+ PG_END_TRY();
+
+ myextra = (TimestampTz *) guc_malloc(ERROR, sizeof(TimestampTz));
+ *myextra = time;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_time(const char *newval, void *extra)
+ {
+ recovery_target_time = *((TimestampTz *) extra);
+
+ if (recovery_target_xid != InvalidTransactionId)
+ recovery_target = RECOVERY_TARGET_XID;
+ else if (recovery_target_name[0])
+ recovery_target = RECOVERY_TARGET_NAME;
+ else if (recovery_target_time != 0)
+ recovery_target = RECOVERY_TARGET_TIME;
+ else
+ recovery_target = RECOVERY_TARGET_UNSET;
+ }
+
+ static bool
+ check_recovery_target_timeline(char **newval, void **extra, GucSource source)
+ {
+ TimeLineID tli = 0;
+ TimeLineID *myextra;
+
+ if (strcmp(*newval, "") == 0 || strcmp(*newval, "latest") == 0)
+ tli = 0;
+ else
+ {
+ errno = 0;
+ tli = (TimeLineID) strtoul(*newval, NULL, 0);
+ if (errno == EINVAL || errno == ERANGE)
+ {
+ GUC_check_errdetail("recovery_target_timeline is not a valid number: \"%s\"",
+ *newval);
+ return false;
+ }
+ }
+
+ myextra = (TimeLineID *) guc_malloc(ERROR, sizeof(TimeLineID));
+ *myextra = tli;
+ *extra = (void *) myextra;
+
+ return true;
+ }
+
+ static void
+ assign_recovery_target_timeline(const char *newval, void *extra)
+ {
+ recovery_target_timeline = *((TimeLineID *) extra);
+ }
+
#include "guc-file.c"
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 192,197 ****
--- 192,214 ----
#archive_timeout = 0 # force a logfile segment switch after this
# number of seconds; 0 disables
+ # - Archive Recovery -
+ #restore_command = '' # command to use to restore an archived logfile segment
+ # placeholders: %p = path of file to restore
+ # %f = file name only
+ # e.g. 'cp /mnt/server/archivedir/%f %p'
+ #archive_cleanup_command = '' # command to execute at every restartpoint
+ #recovery_end_command = '' # command to execute at completion of recovery
+
+ # - Recovery Target -
+ #recovery_target_xid = ''
+ #recovery_target_name = ''
+ #recovery_target_time = ''
+ #recovery_target_inclusive = on
+ #pause_at_recovery_target = on
+ #recovery_target_timeline = '' # timeline ID or 'latest'
+ # (change requires restart)
+
#------------------------------------------------------------------------------
# REPLICATION
***************
*** 219,224 ****
--- 236,245 ----
# These settings are ignored on a master server
+ #standby_mode = off # "on" starts the server as a standby
+ # (change requires restart)
+ #primary_conninfo = '' # connection string to connect to the master
+ #trigger_file = '' # trigger file to promote the standby
#hot_standby = off # "on" allows queries during recovery
# (change requires restart)
#max_standby_archive_delay = 30s # max delay before canceling queries
*** a/src/bin/pg_ctl/pg_ctl.c
--- b/src/bin/pg_ctl/pg_ctl.c
***************
*** 885,891 **** do_stop(void)
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.conf is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
--- 885,891 ----
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.trigger is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
***************
*** 973,979 **** do_restart(void)
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.conf is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
--- 973,979 ----
/*
* If backup_label exists, an online backup is running. Warn the user
* that smart shutdown will wait for it to finish. However, if
! * recovery.trigger is also present, we're recovering from an online
* backup instead of performing one.
*/
if (shutdown_mode == SMART_MODE &&
***************
*** 1084,1090 **** do_promote(void)
exit(1);
}
! /* If recovery.conf doesn't exist, the server is not in standby mode */
if (stat(recovery_file, &statbuf) != 0)
{
write_stderr(_("%s: cannot promote server; "
--- 1084,1090 ----
exit(1);
}
! /* If recovery.trigger doesn't exist, the server is not in standby mode */
if (stat(recovery_file, &statbuf) != 0)
{
write_stderr(_("%s: cannot promote server; "
***************
*** 2223,2229 **** main(int argc, char **argv)
snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
! snprintf(recovery_file, MAXPGPATH, "%s/recovery.conf", pg_data);
snprintf(promote_file, MAXPGPATH, "%s/promote", pg_data);
}
--- 2223,2229 ----
snprintf(postopts_file, MAXPGPATH, "%s/postmaster.opts", pg_data);
snprintf(pid_file, MAXPGPATH, "%s/postmaster.pid", pg_data);
snprintf(backup_file, MAXPGPATH, "%s/backup_label", pg_data);
! snprintf(recovery_file, MAXPGPATH, "%s/recovery.trigger", pg_data);
snprintf(promote_file, MAXPGPATH, "%s/promote", pg_data);
}
*** a/src/include/access/xlog.h
--- b/src/include/access/xlog.h
***************
*** 198,203 **** extern bool XLogArchiveMode;
--- 198,217 ----
extern char *XLogArchiveCommand;
extern bool EnableHotStandby;
extern bool log_checkpoints;
+ extern char *restore_command;
+ extern char *archive_cleanup_command;
+ extern char *recovery_end_command;
+ extern bool standby_mode;
+ extern char *primary_conninfo;
+ extern char *trigger_file;
+ extern RecoveryTargetType recovery_target;
+ extern TransactionId recovery_target_xid;
+ extern TimestampTz recovery_target_time;
+ extern char *recovery_target_name;
+ extern bool recovery_target_inclusive;
+ extern bool pause_at_recovery_target;
+ extern char *recovery_target_timeline_string;
+ extern TimeLineID recovery_target_timeline;
/* WAL levels */
typedef enum WalLevel
*** a/src/include/replication/walreceiver.h
--- b/src/include/replication/walreceiver.h
***************
*** 110,116 **** extern Size WalRcvShmemSize(void);
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
! extern void RequestXLogStreaming(XLogRecPtr recptr, const char *conninfo);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
#endif /* _WALRECEIVER_H */
--- 110,116 ----
extern void WalRcvShmemInit(void);
extern void ShutdownWalRcv(void);
extern bool WalRcvInProgress(void);
! extern void RequestXLogStreaming(XLogRecPtr recptr);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart);
#endif /* _WALRECEIVER_H */
*** a/src/include/utils/guc_tables.h
--- b/src/include/utils/guc_tables.h
***************
*** 68,73 **** enum config_group
--- 68,75 ----
WAL_SETTINGS,
WAL_CHECKPOINTS,
WAL_ARCHIVING,
+ WAL_ARCHIVE_RECOVERY,
+ WAL_RECOVERY_TARGET,
REPLICATION,
REPLICATION_SENDING,
REPLICATION_MASTER,
On Tue, Oct 11, 2011 at 6:37 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Mon, Oct 10, 2011 at 6:52 PM, Josh Berkus <josh@agliodbs.com> wrote:
Tatsuo/Josh/Robert also discussed how recovery.conf can be used to
provide parameters solely for recovery. That is difficult to do
without causing all downstream tools to make major changes in the ways
they supply parameters.Actually, this case is easily solved by an "include recovery.conf"
parameter. So it's a non-issue.That is what I've suggested and yes, doing that is straightforward.
Even if we do that, you still need to modify the tool so that it can handle
the recovery trigger file. recovery.conf is used as just a configuration file
(not recovery trigger file at all). It's not renamed to recovery.done at the
end of recovery. If the tool depends on the renaming from recovery.conf
to recovery.done, it also would need to be modified. If the tool needs to
be changed anyway, why do you hesitate in changing it so that it adds
"include recovery.conf" into postgresql.conf automatically?
Or you think that, to keep the backward compatibility completely,
recovery.conf should be used as not only a configuration file but also a
recovery trigger one and it should be renamed to recovery.done at
the end of recovery?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Fujii Masao wrote:
On Tue, Oct 11, 2011 at 6:37 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Mon, Oct 10, 2011 at 6:52 PM, Josh Berkus <josh@agliodbs.com> wrote:
Tatsuo/Josh/Robert also discussed how recovery.conf can be used to
provide parameters solely for recovery. That is difficult to do
without causing all downstream tools to make major changes in the ways
they supply parameters.Actually, this case is easily solved by an "include recovery.conf"
parameter. ?So it's a non-issue.That is what I've suggested and yes, doing that is straightforward.
Even if we do that, you still need to modify the tool so that it can handle
the recovery trigger file. recovery.conf is used as just a configuration file
(not recovery trigger file at all). It's not renamed to recovery.done at the
end of recovery. If the tool depends on the renaming from recovery.conf
to recovery.done, it also would need to be modified. If the tool needs to
be changed anyway, why do you hesitate in changing it so that it adds
"include recovery.conf" into postgresql.conf automatically?Or you think that, to keep the backward compatibility completely,
recovery.conf should be used as not only a configuration file but also a
recovery trigger one and it should be renamed to recovery.done at
the end of recovery?
As much as I appreciate Simon's work in this area, I think we are still
unclear if keeping backward-compatibility is worth the complexity
required for future users. Historically we have been bold in changing
postgresql.conf settings to improve clarity, and that approach has
served us well.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On 10/10/11 9:53 PM, Fujii Masao wrote:
Or you think that, to keep the backward compatibility completely,
recovery.conf should be used as not only a configuration file but also a
recovery trigger one and it should be renamed to recovery.done at
the end of recovery?
That's precisely my point. The trigger file nature of recovery.conf is
a problem in itself, and I don't see any way to support that and fix it
at the same time. Maybe Simon can?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Tue, Oct 11, 2011 at 3:29 PM, Bruce Momjian <bruce@momjian.us> wrote:
As much as I appreciate Simon's work in this area, I think we are still
unclear if keeping backward-compatibility is worth the complexity
required for future users. Historically we have been bold in changing
postgresql.conf settings to improve clarity, and that approach has
served us well.
You raise a good point. First, thank you for the respectful comment;
my viewpoint is not formed from resistance to change per se, even if
may appear to be so. Thank you for raising that possibility to allow
me to explain and refute that.
I am genuinely concerned that we show respect to downstream software
that uses our APIs and have no personal or corporate ulterior motive.
Most people are used to the 3 year cycle of development on which
SQLServer and Oracle have now standardised. Our 1 year cycle provides
a considerable benefit in agility, but it also provides for x3
complexity in release management and a continual temptation to change
for no good reason. I want to encourage people to adopt our APIs, not
give them a headache for attempting to do so. We know that software
exists that follows the previous API and we should take steps to
deprecate that across multiple releases, with appropriate notice, just
as we do in other cases, such as standard conforming strings where our
lack of boldness is appropriate.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon Riggs wrote:
On Tue, Oct 11, 2011 at 3:29 PM, Bruce Momjian <bruce@momjian.us> wrote:
As much as I appreciate Simon's work in this area, I think we are still
unclear if keeping backward-compatibility is worth the complexity
required for future users. ?Historically we have been bold in changing
postgresql.conf settings to improve clarity, and that approach has
served us well.You raise a good point. First, thank you for the respectful comment;
my viewpoint is not formed from resistance to change per se, even if
may appear to be so. Thank you for raising that possibility to allow
me to explain and refute that.I am genuinely concerned that we show respect to downstream software
that uses our APIs and have no personal or corporate ulterior motive.Most people are used to the 3 year cycle of development on which
SQLServer and Oracle have now standardised. Our 1 year cycle provides
a considerable benefit in agility, but it also provides for x3
complexity in release management and a continual temptation to change
for no good reason. I want to encourage people to adopt our APIs, not
give them a headache for attempting to do so. We know that software
exists that follows the previous API and we should take steps to
deprecate that across multiple releases, with appropriate notice, just
as we do in other cases, such as standard conforming strings where our
lack of boldness is appropriate.
Well, let me be specific. Around 2003 to 2006, we added many new
configuration parameters for logging, which required renaming or
removing older parameters. There really wasn't a smooth way to allow
for this to be done without impacting users, and the current system we
have enjoyed since 2006 is logical only because we made the changes
necessary.
We can look at trying to phase changes in, but often the phasing becomes
more complicated that just doing the change. Logging parameter changes
were easier because it was assumed logging was an admin-only task, as I
assume pitr and replication are as well. Standard conforming strings
was tricky because it was more user-facing, or certainly SQL-facing.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Tue, Oct 11, 2011 at 9:28 PM, Bruce Momjian <bruce@momjian.us> wrote:
Standard conforming strings
was tricky because it was more user-facing, or certainly SQL-facing.
Why is SQL more important than backup?
There is no good reason to do this so quickly.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon Riggs wrote:
On Tue, Oct 11, 2011 at 9:28 PM, Bruce Momjian <bruce@momjian.us> wrote:
Standard conforming strings
was tricky because it was more user-facing, or certainly SQL-facing.Why is SQL more important than backup?
Because the percentage of database users it affects is different.
Administrators know when they are installing a new version of Postgres
and already are probably changing these configuration files.
Application binaries and perhaps application developers are not as aware
of a change, and there are a far higher percentage of them in an
organization than administrators.
There is no good reason to do this so quickly.
I just gave you a reason above, and as I said, doing backward
compatibility can make the system more complex.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Simon,
I haven't see a response from you on a proposed way to keep backwards
compatibility with recovery.conf as a trigger file, while also
eliminating its trigger status as an unmanagable misfeature. As far as
I can tell, that's the one area where we *cannot* maintain backwards
compatibility.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Fri, Sep 9, 2011 at 10:56 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
In previous discussion, we've reached the consensus that we should unite
recovery.conf and postgresql.conf. The attached patch does that. The
patch is WIP, I'll have to update the document, but if you notice something,
please feel free to comment.
My short summary of the thread is
Fujii proposes we allow parameters currently in recovery.conf to be
specified in postgresql.conf. All agree to that. Fujii suggests that
if we have both postgresql.conf and recovery.conf then recovery.conf
should contain overrides.
Fujii then suggests that if such an override exists, then SHOW would
not work properly. Magnus is rightly horrified and many speak against
allowing recovery.conf to continue to exist for this reason. I note
that if recovery.conf is an include file of postgresql.conf then the
overrides would work correctly, just as if postgresql.conf had
multiple settings for that parameter. So the premise is incorrect, so
the conclusion is not relevant.
Simon, JD, Greg Stark speak in favour of the usefulness of having a
recovery.conf separate from postgresql.conf. Tatsuo confirms pgpool
uses this.
Simon, Fujii, Peter agree an automatic include of recovery.conf would be useful
Robert points out that pg_ctl promote was a good feature
Simon, JD say that backwards compatibility is important
Everybody agrees a neater way of invoking standby mode would be good.
Peter points out that including recovery target parameters in
postgresql.conf would be difficult and require manual editing, and
also that pg_ctl -o is not a suitable interface.
The thread also includes a variety of other alternate ideas,
misunderstandings and other commentary.
- - -
My thoughts on how to resolve this are...
Everybody agrees these two points:
* Allow recovery parameters to be handled same way as other GUCs, and
specified in postgresql.conf if desired.
* Allow parameters to be reloaded at SIGHUP and visible using SHOW.
Those two things do not themselves force us to break backwards compatibility.
We also agree that we want a neater way to startup in standby mode.
In 9.1 we added "pg_ctl promote" as a better way of indicating
failover/switchover. When we did that we kept the trigger_file
parameter added in 9.0, which shows it is possible to add a new API
without breaking backwards compatibility.
We should add a "pg_ctl standby" command as a better way of indicating
starting up (also described as triggering) standby mode. We keep
standby_mode parameter. There is no difference here between file
based and stream based replication: you can have file, stream or both
file and stream (as intended).
In this mode the recovery target parameters are *ignored* even if
specified (explained below).
http://developer.postgresql.org/pgdocs/postgres/recovery-target-settings.html
In 9.2 the presence of recovery.conf can and therefore should continue
to act as it does in 9.1. This should be automatically included at the
end of postgresql.conf, which naturally and with no additional code
allows us to override settings, with overrides visible by SHOW. We
don't make any specific checks to see if someone has added a
postgresql.conf parameter in there. If there is a recovery target
parameter in recovery.conf we enter recovery, otherwise we operate as
a standby. recovery.conf is no longer *required* for standby modes.
These things are announced as deprecated and will be removed when we
go to release 10.0
* trigger_file
* standby_mode
* recovery.conf indicates standby
recovery.conf should continue to be required to perform a PITR. If we
place the recovery_target parameters into postgresql.conf we will have
no way to differentiate between (1) a recovery that has successfully
completed then crashed and (2) a user-specified recovery, which was
the original rationale for its use. This is OK, since we now encourage
people to enter a recovery by creating recovery.conf and for entering
a standby to use a new cleaner API without the confusing use of the
word "recovery".
I think that meets all requirements, as far as technically possible.
Best Regards
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Sat, Oct 29, 2011 at 7:54 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Fri, Sep 9, 2011 at 10:56 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
In previous discussion, we've reached the consensus that we should unite
recovery.conf and postgresql.conf. The attached patch does that. The
patch is WIP, I'll have to update the document, but if you notice something,
please feel free to comment.My short summary of the thread is
Thanks!
In 9.1 we added "pg_ctl promote" as a better way of indicating
failover/switchover. When we did that we kept the trigger_file
parameter added in 9.0, which shows it is possible to add a new API
without breaking backwards compatibility.We should add a "pg_ctl standby" command as a better way of indicating
starting up (also described as triggering) standby mode. We keep
standby_mode parameter. There is no difference here between file
based and stream based replication: you can have file, stream or both
file and stream (as intended).
In this mode the recovery target parameters are *ignored* even if
specified (explained below).
http://developer.postgresql.org/pgdocs/postgres/recovery-target-settings.html
Agreed to add "pg_ctl standby". I think that this can be committed
separately from the change of recovery.conf.
In 9.2 the presence of recovery.conf can and therefore should continue
to act as it does in 9.1.
This means that recovery.conf is renamed to recovery.done at the end of
recovery. IOW, all settings in recovery.conf are reset when recovery ends.
Then if you run "pg_ctl reload" after recovery, you'll get something like
the following error message and the reload will always fail;
LOG: parameter "standby_mode" cannot be changed without restarting
the server
To resolve this issue, I think that we need to introduce new GUC flag
indicating parameters are used only during recovery, and need to define
recovery parameters with that flag. Whenever "pg_ctl reload" is executed,
all the processes check whether recovery is in progress, and ignore
silently the parameters with that flag if not. I'm not sure how easy we
can implement this. In addition to introducing that flag, we might need to
change some processes which cannot access to the shared memory so that
they can. Otherwise, they cannot know whether recovery is in progress.
Or we might need to change them so that they always ignore recovery
parameters.
Another simple but somewhat restricted approach is to read and set
all parameters specified in recovery.conf by using PGC_S_OVERRIDE.
If we do this, those parameters cannot be changed after startup
even if recovery.conf is renamed. But the problem is that a user also
cannot change their settings by reloading the configuration files. This is
obviously a restriction. But it doesn't break any backward compatibility,
I believe. No? If a user prefers new functionality (i.e., reload recovery
parameters) rather than the backward compatibility, he/she can specify
parameters in postgresql.conf. Thought?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Mon, Oct 31, 2011 at 7:38 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
In 9.2 the presence of recovery.conf can and therefore should continue
to act as it does in 9.1.This means that recovery.conf is renamed to recovery.done at the end of
recovery. IOW, all settings in recovery.conf are reset when recovery ends.
Then if you run "pg_ctl reload" after recovery, you'll get something like
the following error message and the reload will always fail;LOG: parameter "standby_mode" cannot be changed without restarting
the server
To resolve this issue,
This issue exists whether or not we have recovery.conf etc., so yes,
we must solve the problem.
I think that we need to introduce new GUC flag
indicating parameters are used only during recovery, and need to define
recovery parameters with that flag. Whenever "pg_ctl reload" is executed,
all the processes check whether recovery is in progress, and ignore
silently the parameters with that flag if not. I'm not sure how easy we
can implement this. In addition to introducing that flag, we might need to
change some processes which cannot access to the shared memory so that
they can. Otherwise, they cannot know whether recovery is in progress.
Or we might need to change them so that they always ignore recovery
parameters.
The postmaster knows whether its in recovery or not without checking
shared memory. Various postmaster states describe this. If not
postmaster, other backends can run recoveryinprogress() as normal.
It makes sense to have a new flag and that is easily created and used.
Another simple but somewhat restricted approach is to read and set
all parameters specified in recovery.conf by using PGC_S_OVERRIDE.
If we do this, those parameters cannot be changed after startup
even if recovery.conf is renamed. But the problem is that a user also
cannot change their settings by reloading the configuration files. This is
obviously a restriction. But it doesn't break any backward compatibility,
I believe. No? If a user prefers new functionality (i.e., reload recovery
parameters) rather than the backward compatibility, he/she can specify
parameters in postgresql.conf. Thought?
No need to create problems.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon,
Everybody agrees a neater way of invoking standby mode would be good.
I don't think this goes far enough. The whole
recovery.conf/recovery.done thing is a serious problem for automated
management of servers and automated failover. So it's not just "a
neater way would be good" but "using recovery.conf as a trigger file is
a broken idea and needs to be changed."
These things are announced as deprecated and will be removed when we
go to release 10.0
* trigger_file
* standby_mode
* recovery.conf indicates standby
So you're idea is that people who don't want recovery.conf to be used as
a trigger file would not have the file at all, but would have something
like "replication.conf" instead?
If it's possible to run a replica without having a recovery.conf file,
then I'm fine with your solution. If it's not, then I find your
solution not to be a solution at all.
recovery.conf should continue to be required to perform a PITR. If we
place the recovery_target parameters into postgresql.conf we will have
no way to differentiate between (1) a recovery that has successfully
completed then crashed and (2) a user-specified recovery, which was
the original rationale for its use. This is OK, since we now encourage
people to enter a recovery by creating recovery.conf and for entering
a standby to use a new cleaner API without the confusing use of the
word "recovery".
Sure. recovery.conf worked fine for PITR. We've just overextended it
for other purposes.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Mon, Oct 31, 2011 at 7:05 PM, Josh Berkus <josh@agliodbs.com> wrote:
If it's possible to run a replica without having a recovery.conf file,
then I'm fine with your solution. If it's not, then I find your
solution not to be a solution at all.
Then you are fine with the solution - not mine alone, just the sum of
everybody's inputs.
So we can teach the new way, while supporting the old way a while longer.
Sure. recovery.conf worked fine for PITR. We've just overextended it
for other purposes.
Agreed.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Oct 31, 2011 at 3:19 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Mon, Oct 31, 2011 at 7:05 PM, Josh Berkus <josh@agliodbs.com> wrote:
If it's possible to run a replica without having a recovery.conf file,
then I'm fine with your solution. If it's not, then I find your
solution not to be a solution at all.Then you are fine with the solution - not mine alone, just the sum of
everybody's inputs.So we can teach the new way, while supporting the old way a while longer.
In most cases we either break backwards compatibility or require some
type of switch to turn on backwards compatibility for those who want
it. While the above plan tries to do one better, it leaves me feeling
that the thing I don't like about this is that it sounds like you are
forcing backwards compatibility on people who would much rather just
do things the new way. Given that, I foresee a whole new generation of
confused users who end up setting their configs one way only to have
someone else set the same config in the other file, or some tool dump
out some config file, overriding what was really intended. This will
also make things *harder* for those tool providers you are trying to
help, as they will be forced to support the behavior *both ways*. I'd
much rather see some type of switch which turns on the old behavior
for those who really want it, because while you can teach the new
behavior, if you can't prevent the old behavior, you're creating
operational headaches for yourself.
Robert Treat
conjecture: xzilla.net
consulting: omniti.com
On Mon, Oct 31, 2011 at 5:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Mon, Oct 31, 2011 at 7:38 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
In 9.2 the presence of recovery.conf can and therefore should continue
to act as it does in 9.1.This means that recovery.conf is renamed to recovery.done at the end of
recovery. IOW, all settings in recovery.conf are reset when recovery ends.
Then if you run "pg_ctl reload" after recovery, you'll get something like
the following error message and the reload will always fail;LOG: parameter "standby_mode" cannot be changed without restarting
the serverTo resolve this issue,
This issue exists whether or not we have recovery.conf etc., so yes,
we must solve the problem.
No. If we don't have recovery.conf, all parameters exist in postgresql.conf.
The above issue would not occur unless a user makes a mistake, e.g.,
change the value of the parameter which cannot be changed without
the server restart.
I think that we need to introduce new GUC flag
indicating parameters are used only during recovery, and need to define
recovery parameters with that flag. Whenever "pg_ctl reload" is executed,
all the processes check whether recovery is in progress, and ignore
silently the parameters with that flag if not. I'm not sure how easy we
can implement this. In addition to introducing that flag, we might need to
change some processes which cannot access to the shared memory so that
they can. Otherwise, they cannot know whether recovery is in progress.
Or we might need to change them so that they always ignore recovery
parameters.The postmaster knows whether its in recovery or not without checking
shared memory. Various postmaster states describe this. If not
postmaster, other backends can run recoveryinprogress() as normal.
AFAIR archiver and syslogger cannot access to the shared memory, i.e.,
they cannot run RecoveryInProgress(). They don't use any recovery
parameters for now, so we can change them so that they always ignore
those parameters. Though I'm not inclined to add the process-specific code
like the following into the GUC mechanism as much as possible.
if (I am postmaster)
{
if (recovery is NOT in progress)
reset the recovery parameters;
}
else if (I am archiver or syslogger)
/* always ignore */
else
{
if (recovery is NOT in progress)
reset the recovery parameters;
}
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Tue, Nov 1, 2011 at 5:59 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Mon, Oct 31, 2011 at 5:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Mon, Oct 31, 2011 at 7:38 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
In 9.2 the presence of recovery.conf can and therefore should continue
to act as it does in 9.1.This means that recovery.conf is renamed to recovery.done at the end of
recovery. IOW, all settings in recovery.conf are reset when recovery ends.
Then if you run "pg_ctl reload" after recovery, you'll get something like
the following error message and the reload will always fail;LOG: parameter "standby_mode" cannot be changed without restarting
the serverTo resolve this issue,
This issue exists whether or not we have recovery.conf etc., so yes,
we must solve the problem.No. If we don't have recovery.conf, all parameters exist in postgresql.conf.
The above issue would not occur unless a user makes a mistake, e.g.,
change the value of the parameter which cannot be changed without
the server restart.
I don't understand what you are saying. You seem to be suggesting that
it would be OK for someone to set standby_mode = on and reload the
config and for that to go completely unremarked. If we are adding new
parameters to postgresql.conf then its clear that some people will
misconfigure them and we need to have error messages for that.
If you change a parameter that only has effect during recovery then
must get an error if it is changed during normal running.
That is the reason we need to mark the recovery parameters with a new
flag, so that we can ignore any errors caused by changing them.
That has nothing at all to do with recovery.conf - you need it even if
you put everything in postgresql.conf.
I think that we need to introduce new GUC flag
indicating parameters are used only during recovery, and need to define
recovery parameters with that flag. Whenever "pg_ctl reload" is executed,
all the processes check whether recovery is in progress, and ignore
silently the parameters with that flag if not. I'm not sure how easy we
can implement this. In addition to introducing that flag, we might need to
change some processes which cannot access to the shared memory so that
they can. Otherwise, they cannot know whether recovery is in progress.
Or we might need to change them so that they always ignore recovery
parameters.The postmaster knows whether its in recovery or not without checking
shared memory. Various postmaster states describe this. If not
postmaster, other backends can run recoveryinprogress() as normal.AFAIR archiver and syslogger cannot access to the shared memory, i.e.,
they cannot run RecoveryInProgress(). They don't use any recovery
parameters for now, so we can change them so that they always ignore
those parameters. Though I'm not inclined to add the process-specific code
like the following into the GUC mechanism as much as possible.if (I am postmaster)
{
if (recovery is NOT in progress)
reset the recovery parameters;
}
else if (I am archiver or syslogger)
/* always ignore */
else
{
if (recovery is NOT in progress)
reset the recovery parameters;
}
I don't see the problem. The code above could easily be simplified and
only needs to exist in one place, not copied around for each GUC.
When we had recovery.conf and postgresql.conf you knew which
parameters took effects at specific times. That metadata needs to be
added back into the system so we can report errors properly.
Considering the amount of code we will be removing, a couple of extra
lines seems trivial.
AFAICS we need to reset all recovery parameters at the end of recovery
anyway. Having SHOW recovery_target_xid return a value other than 0 is
not appropriate during normal running, same for standby_mode and the
other parameters.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Nov 1, 2011 at 7:46 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
If you change a parameter that only has effect during recovery then
must get an error if it is changed during normal running.
I don't see why. If you're in normal running and someone changes a
parameter that is irrelevant during normal running, that should be a
no-op, not an error.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Tue, Nov 1, 2011 at 12:06 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Nov 1, 2011 at 7:46 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
If you change a parameter that only has effect during recovery then
must get an error if it is changed during normal running.I don't see why. If you're in normal running and someone changes a
parameter that is irrelevant during normal running, that should be a
no-op, not an error.
How will it be made into a no-op, except by having a specific flag to
show that it is irrelevant during normal running?
Fujii is saying we only need to mark GUCs if we keep recovery.conf. I
am saying we need to mark them whatever we do elsewhere.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Nov 1, 2011 at 8:14 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Nov 1, 2011 at 12:06 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Nov 1, 2011 at 7:46 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
If you change a parameter that only has effect during recovery then
must get an error if it is changed during normal running.I don't see why. If you're in normal running and someone changes a
parameter that is irrelevant during normal running, that should be a
no-op, not an error.How will it be made into a no-op, except by having a specific flag to
show that it is irrelevant during normal running?
By default, changing a GUC just updates the value of some global
variable inside every backend. But unless there's some code that
makes use of that global variable for some purpose, it doesn't have
any practical effect. Apart from whatever complexities may be imposed
by our choice of implementation, I don't see how this would be any
different from setting maintenance_work_mem in a particular session
and then not running any CREATE INDEX or VACUUM commands in that
session.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Tue, Nov 1, 2011 at 1:23 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Nov 1, 2011 at 8:14 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Nov 1, 2011 at 12:06 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Nov 1, 2011 at 7:46 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
If you change a parameter that only has effect during recovery then
must get an error if it is changed during normal running.I don't see why. If you're in normal running and someone changes a
parameter that is irrelevant during normal running, that should be a
no-op, not an error.How will it be made into a no-op, except by having a specific flag to
show that it is irrelevant during normal running?By default, changing a GUC just updates the value of some global
variable inside every backend. But unless there's some code that
makes use of that global variable for some purpose, it doesn't have
any practical effect. Apart from whatever complexities may be imposed
by our choice of implementation, I don't see how this would be any
different from setting maintenance_work_mem in a particular session
and then not running any CREATE INDEX or VACUUM commands in that
session.
Why do we have this log message then, if it is OK to ignore changes
that have no effect?
LOG: parameter "shared_buffers" cannot be changed without restarting the server
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Nov 1, 2011 at 9:45 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
Why do we have this log message then, if it is OK to ignore changes
that have no effect?LOG: parameter "shared_buffers" cannot be changed without restarting the server
I believe we're logging the fact that we were unable to make the
change, not the fact that it didn't have any effect. Certainly, it
*would* have an effect, if we were able to make it. But we can't,
without a restart, so we tell that to the user.
But, for example, the hot_standby GUC - which already exists - does
not do anything in normal running. We don't need to (and don't)
complain if the user tries to change the value in normal running,
though: they're presumably just hoping it will take effect the next
time they start up, which it will. And the autovacuum parameter does
not do anything during hot standby, but we don't need to (and don't)
complain if the user changes it them; it just takes effect when we
enter normal running.
The only time I think we need to complain about an effort to change a
GUC is when the GUC won't take effect as soon as the user might
normally expect.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Tue, Nov 1, 2011 at 2:04 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Nov 1, 2011 at 9:45 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
Why do we have this log message then, if it is OK to ignore changes
that have no effect?LOG: parameter "shared_buffers" cannot be changed without restarting the server
I believe we're logging the fact that we were unable to make the
change, not the fact that it didn't have any effect. Certainly, it
*would* have an effect, if we were able to make it. But we can't,
without a restart, so we tell that to the user.But, for example, the hot_standby GUC - which already exists - does
not do anything in normal running. We don't need to (and don't)
complain if the user tries to change the value in normal running,
though: they're presumably just hoping it will take effect the next
time they start up, which it will.
If there is precedence then we should follow it. So no error messages.
The only reason this point has any importance is that Fujii is
suggesting the code will be much more complicated if we retain
recovery.conf compatibility, since we need to mark GUCs with a flag as
to whether they can take effect during recovery. I don't think that's
a lot of work to add, and would make the code clearer than it is now.
And the autovacuum parameter does
not do anything during hot standby, but we don't need to (and don't)
complain if the user changes it them; it just takes effect when we
enter normal running.
That one needs to be set on a standby, in case it becomes a primary.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Robert,
In most cases we either break backwards compatibility or require some
type of switch to turn on backwards compatibility for those who want
it. While the above plan tries to do one better, it leaves me feeling
that the thing I don't like about this is that it sounds like you are
forcing backwards compatibility on people who would much rather just
do things the new way. Given that, I foresee a whole new generation
of
confused users who end up setting their configs one way only to have
someone else set the same config in the other file, or some tool dump
out some config file, overriding what was really intended. This will
also make things *harder* for those tool providers you are trying to
help, as they will be forced to support the behavior *both ways*. I'd
much rather see some type of switch which turns on the old behavior
for those who really want it, because while you can teach the new
behavior, if you can't prevent the old behavior, you're creating
operational headaches for yourself.
This is a good point. There's also the second drawback, which is complexity of code, which I believe that Tom Lane has brought up before; having two separate-but-equal paths for configuration is liable to lead to a lot of bugs.
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
2) Offer backwards compatibility only if "recovery_conf='filename'" is set in postgresql.conf, then behave like Simon's compromise.
3) Simon's compromise.
4) Don't ever change how recovery.conf works.
The only two of the above I see as being real options are (1) and (2). (3) would, as Robert points out, cause DBAs to have unpleasant surprises when some third-party tool creates a recovery.conf they weren't expecting. So:
(1) pros:
* new, clean API
* makes everyone update their tools
* no confusion on "how to do failover"
* code simplicity
cons:
* breaks a bunch of 3rd-party tools
* or forces them to maintain separate 9.1 and 9.2 branches
(2) pros:
* allows people to use only new API if they want
* allows gradual update of tools
* can also lump in relocatable recovery.conf as feature
cons:
* puts off the day when vendors pay attention to the new API
(and even more kicking & screaming when that day comes)
* confusion about "how to do failover"
* code complexity
On Tue, Nov 1, 2011 at 5:11 PM, Joshua Berkus <josh@agliodbs.com> wrote:
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
Note that is exactly what I have suggested when using "standby" mode
from pg_ctl.
But you already know that, since you said "If it's possible to run a
replica without having a recovery.conf file,
then I'm fine with your solution", and I already confirmed back to you
that would be possible.
2) Offer backwards compatibility only if "recovery_conf='filename'" is set in postgresql.conf, then behave like Simon's compromise.
3) Simon's compromise.
See above. Calling it a compromise in this way implies nobody has been
given exactly what they ask for, but that is not the case.
4) Don't ever change how recovery.conf works.
The only two of the above I see as being real options are (1) and (2). (3) would, as Robert points out, cause DBAs to have unpleasant surprises when some third-party tool creates a recovery.conf they weren't expecting. So:
Please read my proposal again. I'll be happy to answer questions if
you have any.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Nov 1, 2011 at 1:34 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Nov 1, 2011 at 5:11 PM, Joshua Berkus <josh@agliodbs.com> wrote:
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
Note that is exactly what I have suggested when using "standby" mode
from pg_ctl.But you already know that, since you said "If it's possible to run a
replica without having a recovery.conf file,
then I'm fine with your solution", and I already confirmed back to you
that would be possible.
"It's possible to run a replica without having a recovery.conf file"
is not the same thing as "If someone makes a recovery.conf file, it
won't break my operations". AIUI, you are not supporting the latter.
Robert Treat
conjecture: xzilla.net
consulting: omniti.com
On Tue, Nov 1, 2011 at 6:12 PM, Robert Treat <rob@xzilla.net> wrote:
On Tue, Nov 1, 2011 at 1:34 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Nov 1, 2011 at 5:11 PM, Joshua Berkus <josh@agliodbs.com> wrote:
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
Note that is exactly what I have suggested when using "standby" mode
from pg_ctl.But you already know that, since you said "If it's possible to run a
replica without having a recovery.conf file,
then I'm fine with your solution", and I already confirmed back to you
that would be possible."It's possible to run a replica without having a recovery.conf file"
is not the same thing as "If someone makes a recovery.conf file, it
won't break my operations". AIUI, you are not supporting the latter.
Yes, that is part of the "combined proposal", which allows both old
and new APIs.
New API
pg_ctl standby
will startup a server in standby mode, do not implicitly include
recovery.conf and disallow recovery_target parameters in
postgresql.conf
(you may, if you wish, explicitly have "include recovery.conf" in
your postgresql.conf, should you desire that)
Old API
pg_ctl start
and a recovery.conf has been created
will startup a server in PITR and/or replication, recovery.conf
will be read automatically (as now)
so the presence of the recovery.conf acts as a trigger, only if we
issue "start"
So the existing syntax works exactly as now, but a new API has been
created to simplify things in exactly the way requested. The old and
the new API don't mix, so there is no confusion between them.
You must still use the old API when you wish to perform a PITR, as
explained by me, following comments by Peter.
There is no significant additional code or complexity required to do
this, but it adds considerable usefulness.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 11/1/11 10:34 AM, Simon Riggs wrote:
On Tue, Nov 1, 2011 at 5:11 PM, Joshua Berkus <josh@agliodbs.com> wrote:
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
Note that is exactly what I have suggested when using "standby" mode
from pg_ctl.
I wasn't clear on that from the description of your proposal. So are
you suggesting that, if we start postgresql with "pg_ctl standby" then
recovery.conf would not behave as a trigger file?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Tue, Nov 1, 2011 at 6:36 PM, Josh Berkus <josh@agliodbs.com> wrote:
On 11/1/11 10:34 AM, Simon Riggs wrote:
On Tue, Nov 1, 2011 at 5:11 PM, Joshua Berkus <josh@agliodbs.com> wrote:
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
Note that is exactly what I have suggested when using "standby" mode
from pg_ctl.I wasn't clear on that from the description of your proposal. So are
you suggesting that, if we start postgresql with "pg_ctl standby" then
recovery.conf would not behave as a trigger file?
Yes
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Nov 1, 2011 at 2:34 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Nov 1, 2011 at 6:12 PM, Robert Treat <rob@xzilla.net> wrote:
On Tue, Nov 1, 2011 at 1:34 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Tue, Nov 1, 2011 at 5:11 PM, Joshua Berkus <josh@agliodbs.com> wrote:
So, we have four potential paths regarding recovery.conf:
1) Break backwards compatibility entirely, and stop supporting recovery.conf as a trigger file at all.
Note that is exactly what I have suggested when using "standby" mode
from pg_ctl.But you already know that, since you said "If it's possible to run a
replica without having a recovery.conf file,
then I'm fine with your solution", and I already confirmed back to you
that would be possible."It's possible to run a replica without having a recovery.conf file"
is not the same thing as "If someone makes a recovery.conf file, it
won't break my operations". AIUI, you are not supporting the latter.Yes, that is part of the "combined proposal", which allows both old
and new APIs.New API
pg_ctl standby
will startup a server in standby mode, do not implicitly include
recovery.conf and disallow recovery_target parameters in
postgresql.conf
(you may, if you wish, explicitly have "include recovery.conf" in
your postgresql.conf, should you desire that)Old API
pg_ctl start
and a recovery.conf has been created
will startup a server in PITR and/or replication, recovery.conf
will be read automatically (as now)
so the presence of the recovery.conf acts as a trigger, only if we
issue "start"So the existing syntax works exactly as now, but a new API has been
created to simplify things in exactly the way requested. The old and
the new API don't mix, so there is no confusion between them.You must still use the old API when you wish to perform a PITR, as
explained by me, following comments by Peter.
Ah, thanks for clarifying, your earlier proposal didn't read that way
to me. It still doesn't solve the problem for tool makers of needing
to be able to deal with two possible implementation methods, but it
should be easier for them to make a choice. One thing though, I think
it would be better to have this work the other way around. ISTM we're
going to end up telling people to avoid using pg_ctl start and instead
use pg_ctl standby, which doesn't feel like the right answer. Ie.
"Starting in 9.2, you should use pg_ctl standby to launch your
database for normal operations and/or in cases where you are writing
init scripts to control your production databases. For backwards
compatibility, if you require the old behavior of using a
recovery.conf, we would recommend you use pg_ctl start instead".
Robert Treat
conjecture: xzilla.net
consulting: omniti.com
On 11/1/11 12:29 PM, Robert Treat wrote:
"Starting in 9.2, you should use pg_ctl standby to launch your
database for normal operations and/or in cases where you are writing
init scripts to control your production databases. For backwards
compatibility, if you require the old behavior of using a
recovery.conf, we would recommend you use pg_ctl start instead".
Gah.
There is no way we're getting distro packagers to switch from pg_ctl
start. Also, a lot of distros use the "postgres" command rather than
pg_ctl anything.
Messing with pg_ctl is not really a solution for this, since few people
in production environments call it directly. Nobody I know, anyway.
So Simon's suggested compromise still puts backwards compatibility ahead
of promoting the new API. This would result in nobody supporting the
new API until the day we remove the old one from the code.
I think adding "recovery_conf_location = ''" to postgresql.conf is a
much better compromise. Assuming we can stand the code complexity ...
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Tue, Nov 1, 2011 at 9:11 PM, Josh Berkus <josh@agliodbs.com> wrote:
On 11/1/11 12:29 PM, Robert Treat wrote:
"Starting in 9.2, you should use pg_ctl standby to launch your
database for normal operations and/or in cases where you are writing
init scripts to control your production databases. For backwards
compatibility, if you require the old behavior of using a
recovery.conf, we would recommend you use pg_ctl start instead".Gah.
There is no way we're getting distro packagers to switch from pg_ctl
start. Also, a lot of distros use the "postgres" command rather than
pg_ctl anything.Messing with pg_ctl is not really a solution for this, since few people
in production environments call it directly. Nobody I know, anyway.So Simon's suggested compromise still puts backwards compatibility ahead
of promoting the new API. This would result in nobody supporting the
new API until the day we remove the old one from the code.
Which will never happen, since part of the proposal is that PITR will
only be supported using the old method.
I think adding "recovery_conf_location = ''" to postgresql.conf is a
much better compromise. Assuming we can stand the code complexity ...
I think that might have some possibilities. But how does that work in
detail? If you set it to empty, then the recovery_* parameters are
just GUCs, I suppose: which seems fine. But if you set it to a
non-empty value then what happens, exactly? The recovery.conf
settings clobber anything in postgresql.conf, and when we exit
recovery we reload the config, discarding any settings we got from
recovery.conf? That might not be too bad.
I think we need to back up and figure out what problem we're trying to
solve here. IMV, the answer is "setting up a standby is too
complicated and requiring yet another configuration file to do it
makes it even more complicated". If the mechanism we introduce to
"solve" that problem is more complicated than what we have now, it
might end up being a net regression in terms of usability.
I feel like changing everything that's currently in recovery.conf to
GUCs and implementing SET PERSISTENT would give everyone what they
need, admittedly without perfect backward compatibility, but perhaps
close enough for government work, and a step forward overall.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Nov 2, 2011 at 1:45 AM, Robert Haas <robertmhaas@gmail.com> wrote:
I think that might have some possibilities. But how does that work in
detail?
My thoughts also. I want to see the detail on an alternate proposal so
we can decide things sensibly.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Nov 2, 2011 at 1:11 AM, Josh Berkus <josh@agliodbs.com> wrote:
There is no way we're getting distro packagers to switch from pg_ctl
start. Also, a lot of distros use the "postgres" command rather than
pg_ctl anything.
So backwards compatibility is important for downstream software.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Nov 2, 2011 at 1:11 AM, Josh Berkus <josh@agliodbs.com> wrote:
There is no way we're getting distro packagers to switch from pg_ctl
start. Also, a lot of distros use the "postgres" command rather than
pg_ctl anything.
So backwards compatibility is important for downstream software.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
RH, Simon,
I think that might have some possibilities. But how does that work in
detail? If you set it to empty, then the recovery_* parameters are
just GUCs, I suppose: which seems fine. But if you set it to a
non-empty value then what happens, exactly? The recovery.conf
settings clobber anything in postgresql.conf, and when we exit
recovery we reload the config, discarding any settings we got from
recovery.conf? That might not be too bad.
Yeah, that's what I was picturing. By tying backwards-compatibility to
a setting in pg.conf, you eliminate a lot of changes for a DBA
accidentally enabling it. This also then supports re-locating the
recovery.conf file, which has been an issue for a long time.
I think we need to back up and figure out what problem we're trying to
solve here. IMV, the answer is "setting up a standby is too
complicated and requiring yet another configuration file to do it
makes it even more complicated". If the mechanism we introduce to
"solve" that problem is more complicated than what we have now, it
might end up being a net regression in terms of usability.
Well, as someone who sets up and admins replication for a bunch of
clients, here's what I'd like to see:
1. no more using a configuration file as a trigger
2. ability to put replication configuration in postgresql.conf or in a
manually designated include file
3. having replication configuration show up in pg_settings
The three settings above would make my life as a contract DBA much
easier ... and I presume help a lot of our users like me. Among other
things, fixing the 3 things above would make replication integrate a lot
better with configuration management systems and monitoring.
I feel like changing everything that's currently in recovery.conf to
GUCs and implementing SET PERSISTENT would give everyone what they
need, admittedly without perfect backward compatibility, but perhaps
close enough for government work, and a step forward overall.
Is anyone working on SET PERSISTENT? I thought that got bike-shedded to
death.
So backwards compatibility is important for downstream software.
If it wasn't, we wouldn't be having this discussion. However, backwards
compatibility is not necessarily the *most* important consideration.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Wed, Nov 2, 2011 at 2:48 PM, Josh Berkus <josh@agliodbs.com> wrote:
Is anyone working on SET PERSISTENT? I thought that got bike-shedded to
death.
I think we had a fairly good sketch of how it could work mapped out,
mostly based around adding a postgresql.auto file. I could dig up the
old discussions, if need be.
Basically, I'd be willing to get that implemented and bull it through
to completion in time for 9.2 if having that would make it easier for
everyone to accept the idea of turning the recovery.conf parameters
into GUCs. Otherwise I probably will not get to it this cycle.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Nov 2, 2011 at 6:48 PM, Josh Berkus <josh@agliodbs.com> wrote:
Well, as someone who sets up and admins replication for a bunch of
clients, here's what I'd like to see:
Everyone has their own set of requirements. I've tried hard to fuse
those together into a useful proposal, listening to all. Please bear
in mind that I make my living in exactly the same way you do, so you
must surely be aware I do this solely in the common interest.
I don't force you to accept that proposal, but challenging it does
require somebody to step up to the plate and work out a better
detailed proposal rather than just restate what they personally want.
I don't rule out out that a better proposal exists and would be
incredibly happy if someone worked out the design, wrote it up and
desnaggled it.
We can always do nothing, which is a safe and viable option.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon,
Everyone has their own set of requirements. I've tried hard to fuse
those together into a useful proposal, listening to all. Please bear
in mind that I make my living in exactly the same way you do, so you
must surely be aware I do this solely in the common interest.
Thank you for giving us a place to start. I have seen and commented on
your compromise. Both Robert Treat and I poked holes in it. It was a
good first attempt, but not a final attempt -- your compromise proposal
was heavily skewed towards maintaining the status quo at the expense of
improving functionality. As a compromise, it was "70% Simon, 30%
everyone else".
I don't force you to accept that proposal, but challenging it does
require somebody to step up to the plate and work out a better
detailed proposal rather than just restate what they personally want.
I made a proposal, which was based on modifying (and I believe,
improving) your proposal.
We can always do nothing, which is a safe and viable option.
Not really, no. The whole recovery.conf thing is very broken and
inhibits adoption of PostgreSQL because our users can't figure it out.
You've made it pretty clear that you're personally comfortable with how
replication configuration works now, and aren't really interested in any
changes. That's certainly a valid viewpoint, but the users and
contributors who find the API horribly unusable also have a valid
viewpoint. You don't automatically win arguments because you're on the
side of backwards compatibility.
When we released binary replication in 9.0, I thought everyone knew that
it was a first cut and that we'd be making some dramatic changes --
including ones which broke things -- over the next few versions. There
was simply no way for us to know real user requirements until the
feature was in the field and being deployed in production. We would
discover some things which really didn't work and that we had to break
and remake. And we have.
Now you are arguing for premature senescence, where our first API
becomes our only API now and forever. That's a road to project death.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On 09/24/2011 04:49 PM, Joshua Berkus wrote:
Well, we *did* actually come up with a reasonable way, but it died
under an avalanche of bikeshedding and
"we-must-do-everything-the-way-we-always-have-done". I refer, of
course, to the "configuration directory" patch, which was a fine
solution, and would indeed take care of the recovery.conf issues as
well had we implemented it. We can *still* implement it, for 9.2.
That actually died from a lack of round-tuits, the consensus at the end
of the bike-sheeding was pretty clear. Last night I finally got
motivated to fix the bit rot and feature set on that patch, to match
what seemed to be the easiest path toward community approval. One known
bug left to resolve and I think it's ready to submit for the next CF.
I think includeifexists is also a good improvement, too, on a related
arc to the main topic here. If I can finish off the directory one (or
get someone else to fix my bug) I should be able to follow up with that
one. The patches won't be that different.
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
Josh Berkus wrote:
We can always do nothing, which is a safe and viable option.
Not really, no. The whole recovery.conf thing is very broken and
inhibits adoption of PostgreSQL because our users can't figure it out.You've made it pretty clear that you're personally comfortable with how
replication configuration works now, and aren't really interested in any
changes. That's certainly a valid viewpoint, but the users and
contributors who find the API horribly unusable also have a valid
viewpoint. You don't automatically win arguments because you're on the
side of backwards compatibility.When we released binary replication in 9.0, I thought everyone knew that
it was a first cut and that we'd be making some dramatic changes --
including ones which broke things -- over the next few versions. There
was simply no way for us to know real user requirements until the
feature was in the field and being deployed in production. We would
discover some things which really didn't work and that we had to break
and remake. And we have.Now you are arguing for premature senescence, where our first API
becomes our only API now and forever. That's a road to project death.
Agreed. This thread has already expended too much of our valuable time,
in my opinion.
I think we have enough agreement that we need a new API, so let's design
one. If we can add some backward-compatibility here, great, but let's
not have that driving the discussion. Replication is already complex
enough that having two ways to set this up just adds confusion.
Replication/PITR does not affect SQL or applications --- it affects
admin scripts and tools, so they are just going to have to adjust.
We are not going to make everyone happy, so let's just move forward ---
if people want to pout in the corner, I really don't care.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Agreed. This thread has already expended too much of our valuable time,
in my opinion.
As I said elsewhere, I think that a modified version of Simon's proposal
gets us all what we want except code cleanliness. I'd like to hear from
Tom on that issue.
The proposal is:
1. No changes are expected to PITR mode, unless required by the other
changes below.
2. standby_mode becomes an ENUM: "off,standby,pitr". It can be reset on
server reload or through "pg_ctl promote"
3. we add a "recovery_file" filepath parameter to postgresql.conf. This
points to the location of recovery.conf, if any, but does not error
fatally if the file doesn't exist, allowing recovery.conf/.done if we
want to support it.
3.a. we continue to support recovery.conf/.done trigger file
behavior if recovery_file is set. In this case,
the recovery.conf file would contain the standby_mode GUC.
3.b. should pitr mode require recovery_file? Doesn't seem
like it.
3.c. we begin arguments on whether or not recovery_file
should be set by default
4. Haas implements SET PERSISTENT for postgresql.conf
5. pg_ctl promote uses SET PERSISTENT to change standby_mode so that
former standbys can be permanently promoted.
6. (optional) we add include_if_exists specifier for postgresql.conf,
allowing scripters to handle having a separate recovery.conf file in
other ways.
7. (optional) we add a "pg_ctl standby" command which starts up
postgres and sets standby_mode to "standby". I'm not sure I see the
value in this though.
The above allows DBAs to operate in "backwards compatibility mode"
without forcing authors of new tools and scripts to abide by it.
Question: if both standby_mode and recovery_file are set, what should
happen if the recovery_file is not present? For backwards
compatibility, we would use SET PERSISTENT to set standby_mode after the
server comes up. Arguments?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Mon, Nov 7, 2011 at 2:14 PM, Josh Berkus <josh@agliodbs.com> wrote:
2. standby_mode becomes an ENUM: "off,standby,pitr". It can be reset on
server reload or through "pg_ctl promote"
I'm a little bit confused by the way we're dragging standby_mode into
this conversation. If you're using pg_standby, you can set
standby_mode=off and still have a standby. If you're using a simple
recovery command that just copies files, you need to set
standby_mode=on if you don't want the standby to exit recovery and
promote. But I think of standby_mode as meaning "should we use the
internal standby loop rather than depending on an external tool?"
rather than "should we become a standby?".
Maybe I'm confused.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Josh Berkus <josh@agliodbs.com> writes:
As I said elsewhere, I think that a modified version of Simon's proposal
gets us all what we want except code cleanliness. I'd like to hear from
Tom on that issue.
Well, code complexity is hard to gauge without coding a draft
implementation, but I think this largely fails on UI complexity.
It's still paying too high a price for backwards compatibility IMO.
The more I read about this, the more doubtful I am that unifying PITR
recovery with standby mode is a good idea from the UI perspective. They
may share a lot of common infrastructure but they need to be triggered
in fundamentally different ways.
In particular, one of the reasons that recovery.conf/.done was set up
the way that it was was to have a simple way of letting the system get
out of PITR mode; without that, a crash during PITR recovery is going
to lead to restarting from the PITR start point (because when we
restart, we find configuration settings telling us to do that).
We could possibly move the necessary state into pg_control, but keeping
it as a GUC is going to be a mess. On the whole I still think a trigger
file is a sane design for that. It may make sense to move the
configuration data somewhere else, but we really need to be able to tell
the difference between "starting PITR", "continuing PITR after a
mid-recovery crash", and "finished PITR, up and running normally".
A GUC is not a good way to do that.
The angst around this issue seems to me to largely stem from trying to
use a configuration setup designed for one-shot PITR scenarios for
hot standby scenarios, which are really pretty different. We have to
divorce those two cases before we're going to have something that's
sane and usable ... and AFAICS that means giving up backwards
compatibility to some degree.
We got this wrong in 9.0, which everyone understood at the time was an
unpolished prototype implementation of replication. I don't think it's
sensible to now move the goalposts and decree that we've got to be
fully backward compatible with our first-cut mistakes.
regards, tom lane
configuration data somewhere else, but we really need to be able to tell
the difference between "starting PITR", "continuing PITR after a
mid-recovery crash", and "finished PITR, up and running normally".
A GUC is not a good way to do that.
Does a GUC make sense to you for how to handle standby/master for
replication?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
All:
*ping*
Trying to restart this discussion, since the feature bogged down on
spec. We have consensus that we need to change how replication mode is
mangaged; surely we can reach consensus on how to change it?
On 11/8/11 11:39 AM, Josh Berkus wrote:
configuration data somewhere else, but we really need to be able to tell
the difference between "starting PITR", "continuing PITR after a
mid-recovery crash", and "finished PITR, up and running normally".
A GUC is not a good way to do that.Does a GUC make sense to you for how to handle standby/master for
replication?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
I've submitted two patches for adding new include features to the
postgresql.conf file. While not quite commit quality yet, I hope
everyone will agree their reviews this week suggest both are close
enough that any number of people could finish them off. Before
re-opening this can of worms, I wanted people to be comfortable that we
can expect them to be available as building blocks before 9.2
development is finished. Both of those came out of requests from this
unification thread, and they're a helpful part of what I'd like to propose.
I don't see any path forward here that still expects the recovery.conf
file to function as it used to. The "make replication easy" crowd will
eventually be larger than the pre-9.0 user base, if it isn't already.
And they clearly want no parts of that thing. There's been over a year
of arguing around how to cope with it that will satisfy everyone, so
many messages I couldn't even read them all usefully in our archives and
had to go here:
http://postgresql.1045698.n5.nabble.com/recovery-conf-location-td2854644.html
http://postgresql.1045698.n5.nabble.com/unite-recovery-conf-and-postgresql-conf-td4785717.html
I don't think it's possible. What I would propose is a specification
based on forced failure if there's any sign of recovery.conf, combined
with the simplest migration path we can design to ease upgrades from
older versions. I think we can make the transition easy enough. Users
and tool vendors can make relatively simple changes to support 9.2
without changing everything they're used to just yet--while still being
very clear deprecation has arrived and they should reconsider their
approach. Only bug-compatible levels of backwards compatibility would
make this transparent to them, and there's way too many issues to allow
moving forward that way--a forward path that as far as I can see is
desired by the majority of users, and just as importantly for all of the
potential new users we're impacting with the current mess.
There's another, perhaps under considered, concern I want to highlight
as well. Tom has repeatedly noted that one of the worst problems here
would go away if the "existence means do recovery" nature of
recovery.conf went elsewhere. And we know some packagers want to
separate the necessary to manipulate configuration files from the
database directory, for permissions and management reasons. As Heikki
nicely put it (over a year ago), "You don't want to give monitoring
tools that decide on failover write access to the data directory". Any
information that the user is supplying for the purpose of specifying
things needs to be easy to relocate to a separate config file area,
instead of treating it more like a control file in $PGDATA. Some
chatting this morning with Simon pointed out a second related concern
there, which makes ideas like "specify the path to the recovery.conf
file" infeasible. The data_directory is itself a parameter, so anything
tied to that or a new GUC means that config files specified there we
would need two passes. First identify the data directory, then go back
again to read recovery.conf from somewhere else. And nobody wants to
wander that way. If it doesn't fit cleanly into the existing
postgresql.conf parsing, it's gotta go.
Here's the rough outline of what I think would work here:
-All settings move into the postgresql.conf.
-As of 9.2, relocating the postgresql.conf means there are no user
writable files needed in the data directory.
-Creating a standby.enabled file in the directory that houses the
postgresql.conf (same logic as "include" uses to locate things) puts the
system into recovery mode. That feature needs to save some state, and
making those decisions based on existence of a file is already a thing
we do. Rather than emulating the rename to recovery.done that happens
now, the server can just delete it, to keep from incorrectly returning
to a state it's exited. A UI along the lines of the promote one,
allowing "pg_ctl standby", should fall out of here. I think this is
enough that people who just want to use replication features need never
hear about this file at all, at least during the important to simplify
first run through.
-If startup finds a recovery.conf file where it used to live at,
abort--someone is expecting the old behavior. Hint to RTFM or include a
short migration guide right on the spot. That can have a nice section
about how you might use the various postgresql.conf include* features if
they want to continue managing those files separately. Example: rename
it as replication.conf and use include_if_exists if you want to be able
to rename it to recovery.done like before. Or drop it into a conf.d
directory where the rename will make it then skipped.
-Tools such as pgpool that want to write a simple configuration file,
only touching the things that used to go into recovery.conf, can tell
people to do the same trick. End their postgresql.conf with a call to
\include_if_exists replication.conf as part of setup. While I don't
like pushing problems toward tool vendors, as one I think validating if
this has been done doesn't require the sort of fully GUC compatible
parser people (rightly) want to avoid. A simple scan of the
postgresql.conf looking for the recommended text at the front of a line
could confirm whether that bit is there. And adding a single
"include_if_exists" line to the end of the postgresql.conf is not a
terrible edit job to consider pushing toward tools. None of this is any
more complicated than the little search and replace job that initdb does
right now.
-[Optional: for 9.2 but not forever, report incompatibilities, or
potentially stop working altogether, if the things that used to go into
recovery.conf show up when they aren't going to do anything. That would
match the 'stop if the old way is used' theme, but may introduce its own
new operational issues.]
That's basically it for new ideas. The mechanics of making everything
in recovery.conf turn into a GUC seemed pretty uncontroversial when I
scanned the archives here. Another idea not to forget is getting the
read permissions on the new GUCs correct, like making primary_conninfo
superuser visible only. Seems like some of the restart error handling
might still have a tricky part or two left in it. But if you've
accepted that a clear break from existing behavior is happening, some of
those simplify I think.
If we're moving forward this way for 9.2, we do need to be careful that
a spec is nailed down and people take responsibility for their
respective parts of it. There's a few interlocking parts that won't
work well if partially completed. I marked myself as reviewer for this
patch hoping I could crack the deadlock around the specification. One
open item I'm not going to do is a more serious review of the already
submitted code.
To step back toward the thinking that brought me to here (which has
helped pull Simon a bit closer to the middle of the opinion pack too),
an application facing change to PostgreSQL has serious backward
compatibility requirements. It's really bad if the server runs but will
quietly malfunction, such that you may not notice the change until much
later when a rare code path is hit. But when we're talking about
something that impacts whether the server will start or not, that can be
different. You have a new option. Make sure the break is obvious and
total when it happens: do not function at all if someone tries to
operate the old way. The migration path toward the new one should be as
painless as possible. If people only have to tweak one or two simple
things to get an approximation of the old behavior, provide that, give
migration instructions. But this is a new major version; if people
expected all their old configurations to just move over with zero
changes, that's full of fail in all directions. The best we can do is
try to improve the odds their test migrations will fail early, and make
the path to upgrade as simple and well defined as possible.
(It can be argued that people will not actually run into the proposed
"server blocks if recovery.conf exists" fail-safe during migration
testing. It might hit them only when the first fail-over happens in the
field. Then they've got a production server down and are staring at the
new error message, at best; if they don't see the logs even that may not
be facing them. I would say that someone who migrates a
high-availability system over to a new major version, and doesn't read
the release notes nor do the simplest of fail-over testing, is doomed
regardless of what we do to try and help them.)
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
On Wed, Dec 14, 2011 at 4:16 PM, Greg Smith <greg@2ndquadrant.com> wrote:
[ plan for deprecating recovery.conf ]
+1. I'd be very happy with this plan.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Dec 14, 2011 at 22:16, Greg Smith <greg@2ndquadrant.com> wrote:
I've submitted two patches for adding new include features to the
postgresql.conf file. While not quite commit quality yet, I hope everyone
will agree their reviews this week suggest both are close enough that any
number of people could finish them off. Before re-opening this can of
worms, I wanted people to be comfortable that we can expect them to be
available as building blocks before 9.2 development is finished. Both of
those came out of requests from this unification thread, and they're a
helpful part of what I'd like to propose.I don't see any path forward here that still expects the recovery.conf file
to function as it used to. The "make replication easy" crowd will
eventually be larger than the pre-9.0 user base, if it isn't already. And
they clearly want no parts of that thing. There's been over a year of
arguing around how to cope with it that will satisfy everyone, so many
messages I couldn't even read them all usefully in our archives and had to
go here:http://postgresql.1045698.n5.nabble.com/recovery-conf-location-td2854644.html
http://postgresql.1045698.n5.nabble.com/unite-recovery-conf-and-postgresql-conf-td4785717.htmlI don't think it's possible. What I would propose is a specification based
on forced failure if there's any sign of recovery.conf, combined with the
simplest migration path we can design to ease upgrades from older versions.
I think we can make the transition easy enough. Users and tool vendors can
make relatively simple changes to support 9.2 without changing everything
they're used to just yet--while still being very clear deprecation has
arrived and they should reconsider their approach. Only bug-compatible
levels of backwards compatibility would make this transparent to them, and
there's way too many issues to allow moving forward that way--a forward path
that as far as I can see is desired by the majority of users, and just as
importantly for all of the potential new users we're impacting with the
current mess.There's another, perhaps under considered, concern I want to highlight as
well. Tom has repeatedly noted that one of the worst problems here would go
away if the "existence means do recovery" nature of recovery.conf went
elsewhere. And we know some packagers want to separate the necessary to
manipulate configuration files from the database directory, for permissions
and management reasons. As Heikki nicely put it (over a year ago), "You
don't want to give monitoring tools that decide on failover write access to
the data directory". Any information that the user is supplying for the
purpose of specifying things needs to be easy to relocate to a separate
config file area, instead of treating it more like a control file in
$PGDATA. Some chatting this morning with Simon pointed out a second related
concern there, which makes ideas like "specify the path to the recovery.conf
file" infeasible. The data_directory is itself a parameter, so anything
tied to that or a new GUC means that config files specified there we would
need two passes. First identify the data directory, then go back again to
read recovery.conf from somewhere else. And nobody wants to wander that
way. If it doesn't fit cleanly into the existing postgresql.conf parsing,
it's gotta go.Here's the rough outline of what I think would work here:
-All settings move into the postgresql.conf.
-As of 9.2, relocating the postgresql.conf means there are no user writable
files needed in the data directory.-Creating a standby.enabled file in the directory that houses the
postgresql.conf (same logic as "include" uses to locate things) puts the
system into recovery mode. That feature needs to save some state, and
making those decisions based on existence of a file is already a thing we
do. Rather than emulating the rename to recovery.done that happens now, the
server can just delete it, to keep from incorrectly returning to a state
it's exited. A UI along the lines of the promote one, allowing "pg_ctl
standby", should fall out of here. I think this is enough that people who
just want to use replication features need never hear about this file at
all, at least during the important to simplify first run through.
You say "the server can just delete it". But how will this work if the
file is *not* in the data directory? If you are on a Debian style
system for example, where all these files go in /etc/postgresql -
wouldn't that suddenly require the postgres user to have write access
in this directory? If it actually has to be the server that modifies
the file, I think it *does* make sense for this file to live in the
data directory...
[cutting lots of good explanations]
Other than that consideration, +1 for this proposal.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 12/14/11 1:16 PM, Greg Smith wrote:
-Creating a standby.enabled file in the directory that houses the
postgresql.conf (same logic as "include" uses to locate things) puts the
system into recovery mode. That feature needs to save some state, and
making those decisions based on existence of a file is already a thing
we do. Rather than emulating the rename to recovery.done that happens
now, the server can just delete it, to keep from incorrectly returning
to a state it's exited. A UI along the lines of the promote one,
allowing "pg_ctl standby", should fall out of here. I think this is
enough that people who just want to use replication features need never
hear about this file at all, at least during the important to simplify
first run through.
How will things work for PITR?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On 12/14/2011 04:47 PM, Magnus Hagander wrote:
You say "the server can just delete it". But how will this work if the
file is *not* in the data directory? If you are on a Debian style
system for example, where all these files go in /etc/postgresql -
wouldn't that suddenly require the postgres user to have write access
in this directory? If it actually has to be the server that modifies
the file, I think it *does* make sense for this file to live in the
data directory...
Perhaps I should have softened the suggestion to "relocating the
postgresql.conf makes it *possible* to have no user writable files in
the data directory". That was one of the later additions I made, it
didn't bake overnight before sending like the bulk did.
A Debian system might want it to stay in the data directory. If we
consider this not normally touched by the user state information--they
can poke it by hand, but the preferred way is to use pg_ctl--perhaps it
could live in /var/run/postgresql instead. [Surely I'll find out
momentarily, now that I've trolled everyone here who is more familiar
than me with the rules around what goes into /var]
I think the bigger idea I was trying to support in this part is just how
many benefits there are from breaking this role into one decoupled from
the main server configuration. It's not a new suggestion, but I think
it was cut down by backward compatibility concerns before being fully
explored. It seems all of the good ways to provide cleaner UIs need
that, and it surely gives better flexibility to packagers for it to
float free from the config. Who can predict what people will end up
doing in their packages. (And the Gentoo changes have proven it's not
just Debian)
If we drag this conversation back toward the best way to provide that
cleaner UI, but can pick up enough agreement that backward compatibility
limited to the sort of migration ideas I outlined is acceptable, I'd be
happy with that progress. Hopes of reaching that point is the reason I
dumped time into those alternative include options.
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
On 12/14/2011 04:57 PM, Josh Berkus wrote:
How will things work for PITR?
Left that out mainly because I was already running too long there, but I
do think there's a reasonable path. There is one additional wrinkle in
there to consider that I've thought of so far, falling into the "what is
the best UI for this?" category I think.
Put the stuff you used to insert into recovery.conf into postgresql.conf
instead. If you don't like that, use another file and include it with
one of the multiple options for that--same migration option I already
suggested. Run "pg_ctl recovery"; under the hood that's actually
creating standby.enabled instead of recovery.conf, but you don't have to
know that. You'd suggested renaming it to reflect its most common usage
now, and I thought that was quite sensible. It helps with the "things
have changed, please drive carefully" feel too.
It seems possible to have two files for state kickoff/tracking here
instead, maybe have recovery.enabled and standby.enabled. Is that extra
complexity a useful thing? I haven't dug into that new topic much yet.
(Look at that! I think I just found a *new* topic here!)
There are some questions around what to do when it's done. The new
proposed behavior is to delete the standby.enabled file. But that
doesn't remove the changes made for recovery like the old recovery.done
rename did. This is why I commented that some more thinking is likely
needed about how to handle seeing those only-makes-sense-in-recovery
options when not being started for recovery/standby; it's not obvious
that any approach will make everyone happy.
If you want to do something special yourself to clean that up, there's
already recovery_end_command available for that. Let's say you wanted
to force the old name and did "include_if_exists conf.d/recovery.conf",
to trick it even if the patrolling for the name idea goes in. Now you
could do:
recovery_end_command = 'rm -f /tmp/pgsql.trigger.5432 && mv
conf.d/recovery.conf conf.d/recovery.done'
Like some people are used to and might still prefer for some reason.
There'd be time left over to head out to the lawn and yell at the kids
there. Actually, this might be the right approach for tools that are
trying to change as little as possible but add quick 9.2 compatibility.
I think there's enough pluggable bits in every direction here that
people can assemble the setup they'd like out of the available parts,
Maybe these slightly different semantics between archive recovery and
standby mode are exactly why they should be kicked off by differently
named files?
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
Greg,
Put the stuff you used to insert into recovery.conf into postgresql.conf
instead. If you don't like that, use another file and include it with
one of the multiple options for that--same migration option I already
suggested. Run "pg_ctl recovery"; under the hood that's actually
creating standby.enabled instead of recovery.conf, but you don't have to
know that. You'd suggested renaming it to reflect its most common usage
now, and I thought that was quite sensible. It helps with the "things
have changed, please drive carefully" feel too.
So for streaming replication, will I need to have a standby.enabled
file, or will there be a parameter in postgresql.conf (or included
files) which controls whether or not that server is a standby, available?
In the best of all possible worlds, I'd really like to have a GUC which
100% controls whether or not the current server is a standby.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On 12/14/2011 08:56 PM, Josh Berkus wrote:
So for streaming replication, will I need to have a standby.enabled
file, or will there be a parameter in postgresql.conf (or included
files) which controls whether or not that server is a standby, available?In the best of all possible worlds, I'd really like to have a GUC which
100% controls whether or not the current server is a standby
You keep asking the hard questions. Right now, I kind of like that it's
possible to copy a postgresql.conf file from master to standby and just
use it. That should still be possible with the realignment into GUCs:
-"standby_mode = on": Ignored unless you've started the server with
standby.enabled, won't bother the master if you include it.
-"primary_conninfo": This will look funny on the master showing it
connecting to itself, but it will get ignored there too.
I was hoping to just copy over a base backup, "pg_ctl standby" creates
the needed file and starts the server, and I'm done. Isn't that the
easiest replication "hello, world" possible here? If you think there's
an easier way here, please describe it more; I'm missing it so far.
Some settings will look a bit weird in the identical postgresql.conf in
this case, but it think it can be made to work. Now, eventually you
will have to sort this out, but my normal principle here is that any
issue deferred until after people have a working system is automatically
easier for them to stomach. Yes there's complexity, but people are
facing it after the happy dance when the standby works for the first
time. The unavoidable bad situation happens if you promote a standby
made this way. Replicating more standbys from it won't work; you have
to fix primary_conninfo at some point. But once you're the master, you
should be able to change primary_conninfo anytime--even if you SIGHUP to
reload, it will now be ignored--so sorting that out doesn't even require
a server restart. [Problem of how exactly to define a GUC with those
properties while also doing the right thing when you are a standby was
noted then snuck by quietly]
If that is replaced with an edit to the postgresql.conf, that makes the
bar for setting up a standby higher in my mind. Now we have every
clusterware product forced into the position pgpool already finds
itself, where it needs to cope with making at least one change to that
file. I can see a middle ground position where you can have the
standby.enabled file, but you can also set something in the
postgresql.conf, but now we're back to conflict and order resolution
madness. [See: "which of postgresql.conf and recovery.conf should be
read first?"]
[Crazy talk begins here, but without further abuse of parenthetical
brackets]
There is a route this way I wouldn't mind wandering down, but it circles
back to one of the even bigger debates. I would be perfectly happy
fully embracing multiple configuration files for the server by default
on every install. Then the things that vary depending on current role
can all be put into one place, with some expected splits along this
line. Put all the stuff related to standby configuration in one file;
then tools can know "I can overwrite this whole file" and that will be true.
There's an obvious objection here that "having this crap in two files is
the problem we're trying to eliminate!" I would still see this as
forward, because at the very minimum that split should refactor the
replication and recovery target pieces into different files. Different
tools will want to feel they "own" them and can rewrite them, and making
that easy should be a major goal. Also, it will be possible to
rearrange them if you'd like in whatever order makes sense, which you
can't do now for the recovery.conf part. You'd just be breaking tools
that might expect the default split doing that; if you don't care, have
at it.
Wandering any distance down that whole road surely stretches the premise
of "simple migration procedure using include" too far to be true
anymore. I was thinking that for 9.2, it seems feasible to get much of
this legacy stuff sorted better (from the perspective of the person
focused on simple replication), and add some enabling features. No
recovery.conf, everything is a GUC, migration path isn't so bad, people
get exposed to new concepts for include file organization. I'd like to
do some bigger reorganization too, but that seems too much change to
shove all into one release. There's a simple path from there that leads
to both easier tools all around and SET PERSISTENT, and it comes with a
pile of disruption so big I could throw in "standby controls are now
100% GUC" for you plus a unicorn and it would slip right by unnoticed.
That's a tough roadmap to sell unless those promised benefits are proven
first though. And I'm thinking a release doing all that is going to
want to be named 10.0--and what I could really use is a nice, solid 9.2
that doesn't scare enterprises with too much change next.
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
On Wed, Dec 14, 2011 at 8:56 PM, Josh Berkus <josh@agliodbs.com> wrote:
So for streaming replication, will I need to have a standby.enabled
file, or will there be a parameter in postgresql.conf (or included
files) which controls whether or not that server is a standby, available?In the best of all possible worlds, I'd really like to have a GUC which
100% controls whether or not the current server is a standby.
I think that would be a bad idea, because then how would pg_ctl
promote work? It'd have to permanently change the value of that GUC,
which means rewriting a postgresql.conf file with an arbitrary forest
of comments and include files, and we can't do that now or, probably,
ever. At least not reliably enough for this kind of use case. I
think what Greg is going for is this:
1. Get rid of recovery.conf - error out if it is seen
2. For each parameter that was previously a recovery.conf parameter,
make it a GUC
3. For the parameter that was "does recovery.conf exist?", replace it
with "does standby.enabled exist?".
IMHO, as Greg says, that's as much change as we can cope with in one release.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Greg,
You keep asking the hard questions.
I practice. ;-)
Right now, I kind of like that it's
possible to copy a postgresql.conf file from master to standby and just
use it. That should still be possible with the realignment into GUCs:
... long discussion omitted here.
I agree that GUC vs. standby.enabled is a trade-off. I further agree
that where we're going with this eventually is SET PERSISTENT. I feel
that Greg's proposal is a substantial improvement on the current
arrangement and eliminates *my* #1 source of replication-configuration
pain, and we can keep improving it later.
I think we need to give some thought as to how this will play out for
PITR, since there is far less reason to change the operation of PITR,
and much older backup tools which rely on its current operation.
Otherwise, +1.
shove all into one release. There's a simple path from there that leads
to both easier tools all around and SET PERSISTENT, and it comes with a
pile of disruption so big I could throw in "standby controls are now
100% GUC" for you plus a unicorn and it would slip right by unnoticed.
That's a tough roadmap to sell unless those promised benefits are proven
first though. And I'm thinking a release doing all that is going to
want to be named 10.0--and what I could really use is a nice, solid 9.2
that doesn't scare enterprises with too much change next.
I would love to see a writeup on this someday. Blog?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
All,
I'll point out that this patch got sandbagged to death, and never made
it into 9.2. So, for 9.2 replication is just as hard to configure and
manage as it was in 9.1. Are we going to fix it in 9.3, or not?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On Wed, May 09, 2012 at 08:07:52PM -0700, Josh Berkus wrote:
All,
I'll point out that this patch got sandbagged to death, and never made
it into 9.2. So, for 9.2 replication is just as hard to configure and
manage as it was in 9.1. Are we going to fix it in 9.3, or not?
Greg Smith was going to allow for files in configuration directories,
and that was somehow going to make it easier to manage the configuration
files and remove recovery.conf. I don't think Greg did it; I didn't
see it in the release notes I just wrote.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On 05/09/2012 11:15 PM, Bruce Momjian wrote:
On Wed, May 09, 2012 at 08:07:52PM -0700, Josh Berkus wrote:
All,
I'll point out that this patch got sandbagged to death, and never made
it into 9.2. So, for 9.2 replication is just as hard to configure and
manage as it was in 9.1. Are we going to fix it in 9.3, or not?Greg Smith was going to allow for files in configuration directories,
and that was somehow going to make it easier to manage the configuration
files and remove recovery.conf. I don't think Greg did it; I didn't
see it in the release notes I just wrote.
That was actually submitted back in November, and rightly kicked back as
needing more work. I would have updated it and resubmitted if that was
the only blocker. But by the time January rolled around, it was already
obvious that the last CommitFest was going into overtime. Didn't seem
like a great time to add a disruptive change like this one into the mix.
I expect to revisit config directories before the first 9.3 CF, it will
help multiple things I'd like to see happen. Then we can circle back to
the main unification job with a fairly clear path forward from there.
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com
I expect to revisit config directories before the first 9.3 CF, it will
help multiple things I'd like to see happen. Then we can circle back to
the main unification job with a fairly clear path forward from there.
Yeah, let's discuss this next week. "Easier configuration" is one
demand I'm hearing from developers in general, and I don't think that's
nearly as hard a feature as, say, parallel query. We can do it.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
On 10 May 2012 05:44, Josh Berkus <josh@agliodbs.com> wrote:
I expect to revisit config directories before the first 9.3 CF, it will
help multiple things I'd like to see happen. Then we can circle back to
the main unification job with a fairly clear path forward from there.Yeah, let's discuss this next week. "Easier configuration" is one
demand I'm hearing from developers in general, and I don't think that's
nearly as hard a feature as, say, parallel query. We can do it.
A key requirement is to be able to drop in new config files without
needing to $EDIT anything.
OK, its possible to put in lots of includeifexists for non-existent
files just in case you need one, but that sucks.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
A key requirement is to be able to drop in new config files without
needing to $EDIT anything.
Yes, absolutely. I want to move towards the idea that the majority of
our users never edit postgresql.conf by hand.
OK, its possible to put in lots of includeifexists for non-existent
files just in case you need one, but that sucks.
Yeah, seems like we need something more elegant.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com