LSN as a recovery target
Hi all,
Today somebody has pointed me out that it could be interesting to be
able to recovery up to a given LSN position. One argument behind that
was to allow a maximum of things to recover up to the point where a
relation block got corrupted by a specific record because of a broken
segment. So that would be simply having recovery_target_lsn in
recovery.conf similar to what we have now.
Thoughts?
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 24 May 2016 at 09:12, Michael Paquier <michael.paquier@gmail.com> wrote:
Hi all,
Today somebody has pointed me out that it could be interesting to be
able to recovery up to a given LSN position. One argument behind that
was to allow a maximum of things to recover up to the point where a
relation block got corrupted by a specific record because of a broken
segment. So that would be simply having recovery_target_lsn in
recovery.conf similar to what we have now.
Thoughts?
Sounds useful, if somewhat niche. I've needed that in the past, and just
zeroed the WAL segment after the end of the target record. Not super
user-friendly.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Mon, May 23, 2016 at 6:25 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 24 May 2016 at 09:12, Michael Paquier <michael.paquier@gmail.com> wrote:
Hi all,
Today somebody has pointed me out that it could be interesting to be
able to recovery up to a given LSN position. One argument behind that
was to allow a maximum of things to recover up to the point where a
relation block got corrupted by a specific record because of a broken
segment. So that would be simply having recovery_target_lsn in
recovery.conf similar to what we have now.
Thoughts?Sounds useful, if somewhat niche. I've needed that in the past, and just
zeroed the WAL segment after the end of the target record. Not super
user-friendly.
Yeah, that's really something that covers only a narrow case, though
if we don't have it when we need it we're limited to some hacks.
Perhaps people who have the advanced level to use such a thing have
the level to use hacks anyway..
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Michael Paquier 2016-05-24 <CAB7nPqQRXsC8=ozh6GpjLnpZ=MeooUZOaAbzx28n2bjSMv2B4g@mail.gmail.com>
Yeah, that's really something that covers only a narrow case, though
if we don't have it when we need it we're limited to some hacks.
Perhaps people who have the advanced level to use such a thing have
the level to use hacks anyway..
I'd think recovery_target_lsn would be more useful in practice than
the existing recovery_target_xid. So I don't see why it shouldn't just
be added, also given it's likely very unobtrusive to do so.
Christoph
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Christoph Berg wrote:
Re: Michael Paquier 2016-05-24 <CAB7nPqQRXsC8=ozh6GpjLnpZ=MeooUZOaAbzx28n2bjSMv2B4g@mail.gmail.com>
Yeah, that's really something that covers only a narrow case, though
if we don't have it when we need it we're limited to some hacks.
Perhaps people who have the advanced level to use such a thing have
the level to use hacks anyway..I'd think recovery_target_lsn would be more useful in practice than
the existing recovery_target_xid. So I don't see why it shouldn't just
be added, also given it's likely very unobtrusive to do so.
Also, see
/messages/by-id/56BD0E4E.5050503@2ndquadrant.com
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 24, 2016 at 9:29 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
Christoph Berg wrote:
Re: Michael Paquier 2016-05-24 <CAB7nPqQRXsC8=ozh6GpjLnpZ=MeooUZOaAbzx28n2bjSMv2B4g@mail.gmail.com>
Yeah, that's really something that covers only a narrow case, though
if we don't have it when we need it we're limited to some hacks.
Perhaps people who have the advanced level to use such a thing have
the level to use hacks anyway..I'd think recovery_target_lsn would be more useful in practice than
the existing recovery_target_xid. So I don't see why it shouldn't just
be added, also given it's likely very unobtrusive to do so.
Looking at xlog.c it is not that complicated, and we could add tests
in 003_recovery_targets.pl at the same time. Perhaps somebody looking
for a first participation would be interested in this small project?
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, May 25, 2016 at 1:32 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Tue, May 24, 2016 at 9:29 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:Christoph Berg wrote:
Re: Michael Paquier 2016-05-24 <CAB7nPqQRXsC8=ozh6GpjLnpZ=MeooUZOaAbzx28n2bjSMv2B4g@mail.gmail.com>
Yeah, that's really something that covers only a narrow case, though
if we don't have it when we need it we're limited to some hacks.
Perhaps people who have the advanced level to use such a thing have
the level to use hacks anyway..I'd think recovery_target_lsn would be more useful in practice than
the existing recovery_target_xid. So I don't see why it shouldn't just
be added, also given it's likely very unobtrusive to do so.Looking at xlog.c it is not that complicated, and we could add tests
in 003_recovery_targets.pl at the same time. Perhaps somebody looking
for a first participation would be interested in this small project?
Oh, well. I have implemented it as attached by introducing
recovery_target_lsn as a new recovery parameter. This takes into
account recovery_target_inclusive and stops at the precise point of a
record without being influenced by the xact records, in a way similar
recovery_target_name. Tests and documentation are added, and this is
part of the next CF.
--
Michael
Attachments:
recovery-target-lsn.patchbinary/octet-stream; name=recovery-target-lsn.patchDownload
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index 26af221..e131b7d 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -232,6 +232,21 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="recovery-target-lsn" xreflabel="recovery_target_lsn">
+ <term><varname>recovery_target_lsn</varname> (<type>pg_lsn</type>)
+ <indexterm>
+ <primary><varname>recovery_target_lsn</> recovery parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the LSN of the write-ahead log stream up to
+ which recovery will proceed. The precise stopping point is also
+ influenced by <xref linkend="recovery-target-inclusive">.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
<para>
diff --git a/src/backend/access/transam/recovery.conf.sample b/src/backend/access/transam/recovery.conf.sample
index b777400..32f6903 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -67,7 +67,7 @@
# must set a recovery target.
#
# You may set a recovery target either by transactionId, by name,
-# or by timestamp. Recovery may either include or exclude the
+# by timestamp or by LSN. Recovery may either include or exclude the
# transaction(s) with the recovery target value (ie, stop either
# just after or just before the given target, respectively).
#
@@ -78,6 +78,8 @@
#
#recovery_target_xid = ''
#
+#recovery_target_lsn = '' # e.g. '0/70006B8'
+#
#recovery_target_inclusive = true
#
#
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index b473f19..05abbfc 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -67,6 +67,7 @@
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
+#include "utils/pg_lsn.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
@@ -254,6 +255,7 @@ static RecoveryTargetAction recoveryTargetAction = RECOVERY_TARGET_ACTION_PAUSE;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
+static XLogRecPtr recoveryTargetLSN;
static int recovery_min_apply_delay = 0;
static TimestampTz recoveryDelayUntilTime;
@@ -275,6 +277,7 @@ static bool fast_promote = false;
*/
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
+static XLogRecPtr recoveryStopLSN;
static char recoveryStopName[MAXFNAMELEN];
static bool recoveryStopAfter;
@@ -5080,6 +5083,23 @@ readRecoveryCommandFile(void)
(errmsg_internal("recovery_target_name = '%s'",
recoveryTargetName)));
}
+ else if (strcmp(item->name, "recovery_target_lsn") == 0)
+ {
+ recoveryTarget = RECOVERY_TARGET_LSN;
+
+ /*
+ * Convert the LSN string given by the user to XLogRecPtr form.
+ */
+ recoveryTargetLSN =
+ DatumGetLSN(DirectFunctionCall3(pg_lsn_in,
+ CStringGetDatum(item->value),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ ereport(DEBUG2,
+ (errmsg_internal("recovery_target_lsn = '%X/%X'",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
+ }
else if (strcmp(item->name, "recovery_target") == 0)
{
if (strcmp(item->value, "immediate") == 0)
@@ -5390,11 +5410,29 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopTime = 0;
recoveryStopName[0] = '\0';
return true;
}
+ /* Check if target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ !recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = false;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping before LSN \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
/* Otherwise we only consider stopping before COMMIT or ABORT records. */
if (XLogRecGetRmid(record) != RM_XACT_ID)
return false;
@@ -5469,6 +5507,7 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (isCommit)
@@ -5522,6 +5561,7 @@ recoveryStopsAfter(XLogReaderState *record)
{
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
(void) getRecordTimestamp(record, &recoveryStopTime);
strlcpy(recoveryStopName, recordRestorePointData->rp_name, MAXFNAMELEN);
@@ -5533,6 +5573,23 @@ recoveryStopsAfter(XLogReaderState *record)
}
}
+ /* Check if the target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = true;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping after LSN \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
if (rmid != RM_XACT_ID)
return false;
@@ -5588,6 +5645,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (xact_info == XLOG_XACT_COMMIT ||
@@ -5619,6 +5677,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
recoveryStopTime = 0;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
return true;
}
@@ -6045,6 +6104,11 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("starting point-in-time recovery to \"%s\"",
recoveryTargetName)));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ ereport(LOG,
+ (errmsg("starting point-in-time recovery to LSN \"%X/%X\"",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
else if (recoveryTarget == RECOVERY_TARGET_IMMEDIATE)
ereport(LOG,
(errmsg("starting point-in-time recovery to earliest consistent point")));
@@ -7114,6 +7178,11 @@ StartupXLOG(void)
"%s %s\n",
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ snprintf(reason, sizeof(reason),
+ "at LSN %X/%X\n",
+ (uint32 ) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN);
else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(reason, sizeof(reason),
"at restore point \"%s\"",
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index e7e91fc..9eb5ca8 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -83,6 +83,7 @@ typedef enum
RECOVERY_TARGET_XID,
RECOVERY_TARGET_TIME,
RECOVERY_TARGET_NAME,
+ RECOVERY_TARGET_LSN,
RECOVERY_TARGET_IMMEDIATE
} RecoveryTargetType;
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index b20116a..8945a6c 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -3,7 +3,7 @@ use strict;
use warnings;
use PostgresNode;
use TestLib;
-use Test::More tests => 7;
+use Test::More tests => 9;
# Create and test a standby from given backup, with a certain
# recovery target.
@@ -86,6 +86,16 @@ my $lsn4 =
$node_master->safe_psql('postgres',
"SELECT pg_create_restore_point('$recovery_name');");
+# And now for a recovery target LSN
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(4001,5000))");
+my $recovery_lsn = $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location()");
+my $lsn5 =
+ $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location();");
+
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(5001,6000))");
+
# Force archiving of WAL file
$node_master->safe_psql('postgres', "SELECT pg_switch_xlog()");
@@ -102,6 +112,9 @@ test_recovery_standby('Time', 'standby_3', $node_master, \@recovery_params,
@recovery_params = ("recovery_target_name = '$recovery_name'");
test_recovery_standby('Name', 'standby_4', $node_master, \@recovery_params,
"4000", $lsn4);
+@recovery_params = ("recovery_target_lsn = '$recovery_lsn'");
+test_recovery_standby('LSN', 'standby_5', $node_master, \@recovery_params,
+ "5000", $lsn5);
# Multiple targets
# Last entry has priority (note that an array respects the order of items
@@ -111,16 +124,23 @@ test_recovery_standby('Name', 'standby_4', $node_master, \@recovery_params,
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'");
test_recovery_standby('Name + XID + Time',
- 'standby_5', $node_master, \@recovery_params, "3000", $lsn3);
+ 'standby_6', $node_master, \@recovery_params, "3000", $lsn3);
@recovery_params = (
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'",
"recovery_target_xid = '$recovery_txid'");
test_recovery_standby('Time + Name + XID',
- 'standby_6', $node_master, \@recovery_params, "2000", $lsn2);
+ 'standby_7', $node_master, \@recovery_params, "2000", $lsn2);
@recovery_params = (
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'");
test_recovery_standby('XID + Time + Name',
- 'standby_7', $node_master, \@recovery_params, "4000", $lsn4);
+ 'standby_8', $node_master, \@recovery_params, "4000", $lsn4);
+@recovery_params = (
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_lsn = '$recovery_lsn'",);
+test_recovery_standby('XID + Time + Name + LSN',
+ 'standby_9', $node_master, \@recovery_params, "5000", $lsn5);
On 06/09/2016 02:33 PM, Michael Paquier wrote:
On Wed, May 25, 2016 at 1:32 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Tue, May 24, 2016 at 9:29 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:Christoph Berg wrote:
Re: Michael Paquier 2016-05-24 <CAB7nPqQRXsC8=ozh6GpjLnpZ=MeooUZOaAbzx28n2bjSMv2B4g@mail.gmail.com>
Yeah, that's really something that covers only a narrow case, though
if we don't have it when we need it we're limited to some hacks.
Perhaps people who have the advanced level to use such a thing have
the level to use hacks anyway..I'd think recovery_target_lsn would be more useful in practice than
the existing recovery_target_xid. So I don't see why it shouldn't just
be added, also given it's likely very unobtrusive to do so.Looking at xlog.c it is not that complicated, and we could add tests
in 003_recovery_targets.pl at the same time. Perhaps somebody looking
for a first participation would be interested in this small project?Oh, well. I have implemented it as attached by introducing
recovery_target_lsn as a new recovery parameter. This takes into
account recovery_target_inclusive and stops at the precise point of a
record without being influenced by the xact records, in a way similar
recovery_target_name. Tests and documentation are added, and this is
part of the next CF.
Hi,
I reviewed this patch rebased to deal with
f6ced51f9188ad5806219471a0b40a91dde923aa, and minor adjustment (see below)
It do the job. However if you use an incorrect recovery_target_lsn you
get this message :
"FATAL: invalid input syntax for type pg_lsn: "0RT5/4""
I added a PG_TRY/PG_CATCH section in order to get a more explicit message :
FATAL: wrong recovery_target_lsn (must be pg_lsn type)
I am not sure if it is the best solution?
Thanks to Julien Rouhaud for his help.
--
Adrien NAYRAT
Attachments:
recovery-target-lsn-2.patchtext/x-patch; name=recovery-target-lsn-2.patchDownload
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index 26af221..e131b7d 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -232,6 +232,21 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="recovery-target-lsn" xreflabel="recovery_target_lsn">
+ <term><varname>recovery_target_lsn</varname> (<type>pg_lsn</type>)
+ <indexterm>
+ <primary><varname>recovery_target_lsn</> recovery parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the LSN of the write-ahead log stream up to
+ which recovery will proceed. The precise stopping point is also
+ influenced by <xref linkend="recovery-target-inclusive">.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
<para>
diff --git a/src/backend/access/transam/recovery.conf.sample b/src/backend/access/transam/recovery.conf.sample
index b777400..32f6903 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -67,7 +67,7 @@
# must set a recovery target.
#
# You may set a recovery target either by transactionId, by name,
-# or by timestamp. Recovery may either include or exclude the
+# by timestamp or by LSN. Recovery may either include or exclude the
# transaction(s) with the recovery target value (ie, stop either
# just after or just before the given target, respectively).
#
@@ -78,6 +78,8 @@
#
#recovery_target_xid = ''
#
+#recovery_target_lsn = '' # e.g. '0/70006B8'
+#
#recovery_target_inclusive = true
#
#
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f13f9c1..762c7bb 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -67,6 +67,7 @@
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
+#include "utils/pg_lsn.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
@@ -254,6 +255,7 @@ static RecoveryTargetAction recoveryTargetAction = RECOVERY_TARGET_ACTION_PAUSE;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
+static XLogRecPtr recoveryTargetLSN;
static int recovery_min_apply_delay = 0;
static TimestampTz recoveryDelayUntilTime;
@@ -275,6 +277,7 @@ static bool fast_promote = false;
*/
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
+static XLogRecPtr recoveryStopLSN;
static char recoveryStopName[MAXFNAMELEN];
static bool recoveryStopAfter;
@@ -5078,6 +5081,34 @@ readRecoveryCommandFile(void)
(errmsg_internal("recovery_target_name = '%s'",
recoveryTargetName)));
}
+ else if (strcmp(item->name, "recovery_target_lsn") == 0)
+ {
+ recoveryTarget = RECOVERY_TARGET_LSN;
+
+ /*
+ * Convert the LSN string given by the user to XLogRecPtr form.
+ */
+ PG_TRY();
+ {
+ recoveryTargetLSN =
+ DatumGetLSN(DirectFunctionCall3(pg_lsn_in,
+ CStringGetDatum(item->value),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ }
+ PG_CATCH();
+ {
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("wrong recovery_target_lsn (must be pg_lsn type)",
+ MAXFNAMELEN - 1)));
+ }
+ PG_END_TRY();
+ ereport(DEBUG2,
+ (errmsg_internal("recovery_target_lsn = '%X/%X'",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
+ }
else if (strcmp(item->name, "recovery_target") == 0)
{
if (strcmp(item->value, "immediate") == 0)
@@ -5388,8 +5419,26 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ return true;
+ }
+
+ /* Check if target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ !recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = false;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
recoveryStopTime = 0;
recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping before LSN \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
return true;
}
@@ -5467,6 +5516,7 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (isCommit)
@@ -5520,6 +5570,7 @@ recoveryStopsAfter(XLogReaderState *record)
{
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
(void) getRecordTimestamp(record, &recoveryStopTime);
strlcpy(recoveryStopName, recordRestorePointData->rp_name, MAXFNAMELEN);
@@ -5531,6 +5582,23 @@ recoveryStopsAfter(XLogReaderState *record)
}
}
+ /* Check if the target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = true;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping after LSN \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
if (rmid != RM_XACT_ID)
return false;
@@ -5586,6 +5654,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (xact_info == XLOG_XACT_COMMIT ||
@@ -5617,6 +5686,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
recoveryStopTime = 0;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
return true;
}
@@ -6043,6 +6113,11 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("starting point-in-time recovery to \"%s\"",
recoveryTargetName)));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ ereport(LOG,
+ (errmsg("starting point-in-time recovery to LSN \"%X/%X\"",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
else if (recoveryTarget == RECOVERY_TARGET_IMMEDIATE)
ereport(LOG,
(errmsg("starting point-in-time recovery to earliest consistent point")));
@@ -7112,6 +7187,11 @@ StartupXLOG(void)
"%s %s\n",
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ snprintf(reason, sizeof(reason),
+ "at LSN %X/%X\n",
+ (uint32 ) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN);
else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(reason, sizeof(reason),
"at restore point \"%s\"",
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..c9f332c 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -83,6 +83,7 @@ typedef enum
RECOVERY_TARGET_XID,
RECOVERY_TARGET_TIME,
RECOVERY_TARGET_NAME,
+ RECOVERY_TARGET_LSN,
RECOVERY_TARGET_IMMEDIATE
} RecoveryTargetType;
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index d1f6d78..a82545b 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -3,7 +3,7 @@ use strict;
use warnings;
use PostgresNode;
use TestLib;
-use Test::More tests => 7;
+use Test::More tests => 9;
# Create and test a standby from given backup, with a certain
# recovery target.
@@ -86,6 +86,16 @@ my $lsn4 =
$node_master->safe_psql('postgres',
"SELECT pg_create_restore_point('$recovery_name');");
+# And now for a recovery target LSN
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(4001,5000))");
+my $recovery_lsn = $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location()");
+my $lsn5 =
+ $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location();");
+
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(5001,6000))");
+
# Force archiving of WAL file
$node_master->safe_psql('postgres', "SELECT pg_switch_xlog()");
@@ -102,6 +112,9 @@ test_recovery_standby('time', 'standby_3', $node_master, \@recovery_params,
@recovery_params = ("recovery_target_name = '$recovery_name'");
test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"4000", $lsn4);
+@recovery_params = ("recovery_target_lsn = '$recovery_lsn'");
+test_recovery_standby('LSN', 'standby_5', $node_master, \@recovery_params,
+ "5000", $lsn5);
# Multiple targets
# Last entry has priority (note that an array respects the order of items
@@ -111,16 +124,23 @@ test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'");
test_recovery_standby('name + XID + time',
- 'standby_5', $node_master, \@recovery_params, "3000", $lsn3);
+ 'standby_6', $node_master, \@recovery_params, "3000", $lsn3);
@recovery_params = (
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'",
"recovery_target_xid = '$recovery_txid'");
test_recovery_standby('time + name + XID',
- 'standby_6', $node_master, \@recovery_params, "2000", $lsn2);
+ 'standby_7', $node_master, \@recovery_params, "2000", $lsn2);
@recovery_params = (
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'");
test_recovery_standby('XID + time + name',
- 'standby_7', $node_master, \@recovery_params, "4000", $lsn4);
+ 'standby_8', $node_master, \@recovery_params, "4000", $lsn4);
+@recovery_params = (
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_lsn = '$recovery_lsn'",);
+test_recovery_standby('XID + time + name + LSN',
+ 'standby_9', $node_master, \@recovery_params, "5000", $lsn5);
On Fri, Aug 19, 2016 at 10:47 PM, Adrien Nayrat
<adrien.nayrat@dalibo.com> wrote:
I reviewed this patch rebased to deal with
f6ced51f9188ad5806219471a0b40a91dde923aa, and minor adjustment (see below)
Thanks!
It do the job. However if you use an incorrect recovery_target_lsn you
get this message :
"FATAL: invalid input syntax for type pg_lsn: "0RT5/4""I added a PG_TRY/PG_CATCH section in order to get a more explicit message :
FATAL: wrong recovery_target_lsn (must be pg_lsn type)I am not sure if it is the best solution?
Using a PG_TRY/CATCH block the way you do to show to user a different
error message while the original one is actually correct does not
sound like a good idea to me. It complicates the code and the original
pattern matches what is already done for timestamps, where in case of
error you'd get that:
FATAL: invalid input syntax for type timestamp with time zone: "aa"
I think that a better solution would be to make clearer in the docs
that pg_lsn is used here. First, in recovery.conf.sample, let's use
pg_lsn and not LSN. Then, in the sgml docs, let's add a reference to
the page of pg_lsn as well:
https://www.postgresql.org/docs/devel/static/datatype-pg-lsn.html
Now we have that:
+ <para>
+ This parameter specifies the LSN of the write-ahead log stream up to
+ which recovery will proceed. The precise stopping point is also
+ influenced by <xref linkend="recovery-target-inclusive">.
+ </para>
Perhaps we could just add an additional sentence: "This parameter
value is parsed using the system datatype <link=pg_lsn>". Or something
like that. Thoughts?
I'll think about that a bit more and send a new patch. I have switched
the patch to "waiting on author" for now, just to not forget about
that.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 20/08/16 02:13, Michael Paquier wrote:
On Fri, Aug 19, 2016 at 10:47 PM, Adrien Nayrat
<adrien.nayrat@dalibo.com> wrote:I reviewed this patch rebased to deal with
f6ced51f9188ad5806219471a0b40a91dde923aa, and minor adjustment (see below)Thanks!
It do the job. However if you use an incorrect recovery_target_lsn you
get this message :
"FATAL: invalid input syntax for type pg_lsn: "0RT5/4""I added a PG_TRY/PG_CATCH section in order to get a more explicit message :
FATAL: wrong recovery_target_lsn (must be pg_lsn type)I am not sure if it is the best solution?
Using a PG_TRY/CATCH block the way you do to show to user a different
error message while the original one is actually correct does not
sound like a good idea to me. It complicates the code and the original
pattern matches what is already done for timestamps, where in case of
error you'd get that:
FATAL: invalid input syntax for type timestamp with time zone: "aa"I think that a better solution would be to make clearer in the docs
that pg_lsn is used here. First, in recovery.conf.sample, let's use
pg_lsn and not LSN. Then, in the sgml docs, let's add a reference to
the page of pg_lsn as well:
https://www.postgresql.org/docs/devel/static/datatype-pg-lsn.htmlNow we have that: + <para> + This parameter specifies the LSN of the write-ahead log stream up to + which recovery will proceed. The precise stopping point is also + influenced by <xref linkend="recovery-target-inclusive">. + </para> Perhaps we could just add an additional sentence: "This parameter value is parsed using the system datatype <link=pg_lsn>". Or something like that. Thoughts?
+1
If we want to specifically name the recovery_target_lsn in the message,
we could probably do it using context.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Aug 20, 2016 at 10:44 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:
If we want to specifically name the recovery_target_lsn in the message, we
could probably do it using context.
So that would be basically assigning error_context_stack for each item
parsed for recovery.conf? That seems a bit narrow as usually
recovery.conf is not that long..
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 20/08/2016 15:41, Michael Paquier wrote:
On Sat, Aug 20, 2016 at 10:44 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:
If we want to specifically name the recovery_target_lsn in the message, we
could probably do it using context.So that would be basically assigning error_context_stack for each item
parsed for recovery.conf? That seems a bit narrow as usually
recovery.conf is not that long..
That'd still be an improvement, since the error message doesn't even
mention that the error comes from recovery.conf. I agree it can't come
from many places, but it may be useful for some people.
--
Julien Rouhaud
http://dalibo.com - http://dalibo.org
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Aug 20, 2016 at 10:44 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:
On 20/08/16 02:13, Michael Paquier wrote:
On Fri, Aug 19, 2016 at 10:47 PM, Adrien Nayrat
<adrien.nayrat@dalibo.com> wrote:
Using a PG_TRY/CATCH block the way you do to show to user a different
error message while the original one is actually correct does not
sound like a good idea to me. It complicates the code and the original
pattern matches what is already done for timestamps, where in case of
error you'd get that:
FATAL: invalid input syntax for type timestamp with time zone: "aa"I think that a better solution would be to make clearer in the docs
that pg_lsn is used here. First, in recovery.conf.sample, let's use
pg_lsn and not LSN. Then, in the sgml docs, let's add a reference to
the page of pg_lsn as well:
https://www.postgresql.org/docs/devel/static/datatype-pg-lsn.htmlNow we have that: + <para> + This parameter specifies the LSN of the write-ahead log stream up to + which recovery will proceed. The precise stopping point is also + influenced by <xref linkend="recovery-target-inclusive">. + </para> Perhaps we could just add an additional sentence: "This parameter value is parsed using the system datatype <link=pg_lsn>". Or something like that. Thoughts?+1
So here is the patch resulting in that. After thinking more about that
I have arrived at the conclusion to not use LSN in recovery.conf, but
"WAL position (LSN)", and also mention in the docs that pg_lsn is used
to parse the parameter value. A couple of log messages are changed
similarly. It definitely makes it more understandable for users to
speak of WAL positions.
I also noticed that the top block of "Recovery Target Settings" needs
to mention recovery_target_lsn, aka when name, xid, lsn and time are
combined in the same recovery.conf, only the last value will be used.
--
Michael
Attachments:
recovery-target-lsn-3.patchapplication/x-download; name=recovery-target-lsn-3.patchDownload
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index 26af221..ad977d3 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -157,9 +157,10 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
By default, recovery will recover to the end of the WAL log. The
following parameters can be used to specify an earlier stopping point.
At most one of <varname>recovery_target</>,
- <varname>recovery_target_name</>, <varname>recovery_target_time</>, or
- <varname>recovery_target_xid</> can be used; if more than one of these
- is specified in the configuration file, the last entry will be used.
+ <varname>recovery_target_lsn</>, <varname>recovery_target_name</>,
+ <varname>recovery_target_time</>, or <varname>recovery_target_xid</>
+ can be used; if more than one of these is specified in the configuration
+ file, the last entry will be used.
</para>
<variablelist>
@@ -232,6 +233,23 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="recovery-target-lsn" xreflabel="recovery_target_lsn">
+ <term><varname>recovery_target_lsn</varname> (<type>pg_lsn</type>)
+ <indexterm>
+ <primary><varname>recovery_target_lsn</> recovery parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the LSN of the write-ahead log stream up to
+ which recovery will proceed. The precise stopping point is also
+ influenced by <xref linkend="recovery-target-inclusive">. This
+ parameter is parsed using the system data type
+ <link linkend="datatype-pg-lsn"><type>pg_lsn</></link>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
<para>
diff --git a/src/backend/access/transam/recovery.conf.sample b/src/backend/access/transam/recovery.conf.sample
index b777400..7a16751 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -67,8 +67,8 @@
# must set a recovery target.
#
# You may set a recovery target either by transactionId, by name,
-# or by timestamp. Recovery may either include or exclude the
-# transaction(s) with the recovery target value (ie, stop either
+# by timestamp or by WAL position (LSN). Recovery may either include or
+# exclude the transaction(s) with the recovery target value (ie, stop either
# just after or just before the given target, respectively).
#
#
@@ -78,6 +78,8 @@
#
#recovery_target_xid = ''
#
+#recovery_target_lsn = '' # e.g. '0/70006B8'
+#
#recovery_target_inclusive = true
#
#
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f13f9c1..e3985ee 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -67,6 +67,7 @@
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
+#include "utils/pg_lsn.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
@@ -254,6 +255,7 @@ static RecoveryTargetAction recoveryTargetAction = RECOVERY_TARGET_ACTION_PAUSE;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
+static XLogRecPtr recoveryTargetLSN;
static int recovery_min_apply_delay = 0;
static TimestampTz recoveryDelayUntilTime;
@@ -275,6 +277,7 @@ static bool fast_promote = false;
*/
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
+static XLogRecPtr recoveryStopLSN;
static char recoveryStopName[MAXFNAMELEN];
static bool recoveryStopAfter;
@@ -5078,6 +5081,23 @@ readRecoveryCommandFile(void)
(errmsg_internal("recovery_target_name = '%s'",
recoveryTargetName)));
}
+ else if (strcmp(item->name, "recovery_target_lsn") == 0)
+ {
+ recoveryTarget = RECOVERY_TARGET_LSN;
+
+ /*
+ * Convert the LSN string given by the user to XLogRecPtr form.
+ */
+ recoveryTargetLSN =
+ DatumGetLSN(DirectFunctionCall3(pg_lsn_in,
+ CStringGetDatum(item->value),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ ereport(DEBUG2,
+ (errmsg_internal("recovery_target_lsn = '%X/%X'",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
+ }
else if (strcmp(item->name, "recovery_target") == 0)
{
if (strcmp(item->value, "immediate") == 0)
@@ -5388,11 +5408,29 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopTime = 0;
recoveryStopName[0] = '\0';
return true;
}
+ /* Check if target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ !recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = false;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping before WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
/* Otherwise we only consider stopping before COMMIT or ABORT records. */
if (XLogRecGetRmid(record) != RM_XACT_ID)
return false;
@@ -5467,6 +5505,7 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (isCommit)
@@ -5520,6 +5559,7 @@ recoveryStopsAfter(XLogReaderState *record)
{
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
(void) getRecordTimestamp(record, &recoveryStopTime);
strlcpy(recoveryStopName, recordRestorePointData->rp_name, MAXFNAMELEN);
@@ -5531,6 +5571,23 @@ recoveryStopsAfter(XLogReaderState *record)
}
}
+ /* Check if the target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = true;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping after WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
if (rmid != RM_XACT_ID)
return false;
@@ -5586,6 +5643,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (xact_info == XLOG_XACT_COMMIT ||
@@ -5617,6 +5675,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
recoveryStopTime = 0;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
return true;
}
@@ -6043,6 +6102,11 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("starting point-in-time recovery to \"%s\"",
recoveryTargetName)));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ ereport(LOG,
+ (errmsg("starting point-in-time recovery to WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
else if (recoveryTarget == RECOVERY_TARGET_IMMEDIATE)
ereport(LOG,
(errmsg("starting point-in-time recovery to earliest consistent point")));
@@ -7112,6 +7176,11 @@ StartupXLOG(void)
"%s %s\n",
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ snprintf(reason, sizeof(reason),
+ "at LSN %X/%X\n",
+ (uint32 ) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN);
else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(reason, sizeof(reason),
"at restore point \"%s\"",
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..c9f332c 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -83,6 +83,7 @@ typedef enum
RECOVERY_TARGET_XID,
RECOVERY_TARGET_TIME,
RECOVERY_TARGET_NAME,
+ RECOVERY_TARGET_LSN,
RECOVERY_TARGET_IMMEDIATE
} RecoveryTargetType;
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index d1f6d78..a82545b 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -3,7 +3,7 @@ use strict;
use warnings;
use PostgresNode;
use TestLib;
-use Test::More tests => 7;
+use Test::More tests => 9;
# Create and test a standby from given backup, with a certain
# recovery target.
@@ -86,6 +86,16 @@ my $lsn4 =
$node_master->safe_psql('postgres',
"SELECT pg_create_restore_point('$recovery_name');");
+# And now for a recovery target LSN
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(4001,5000))");
+my $recovery_lsn = $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location()");
+my $lsn5 =
+ $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location();");
+
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(5001,6000))");
+
# Force archiving of WAL file
$node_master->safe_psql('postgres', "SELECT pg_switch_xlog()");
@@ -102,6 +112,9 @@ test_recovery_standby('time', 'standby_3', $node_master, \@recovery_params,
@recovery_params = ("recovery_target_name = '$recovery_name'");
test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"4000", $lsn4);
+@recovery_params = ("recovery_target_lsn = '$recovery_lsn'");
+test_recovery_standby('LSN', 'standby_5', $node_master, \@recovery_params,
+ "5000", $lsn5);
# Multiple targets
# Last entry has priority (note that an array respects the order of items
@@ -111,16 +124,23 @@ test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'");
test_recovery_standby('name + XID + time',
- 'standby_5', $node_master, \@recovery_params, "3000", $lsn3);
+ 'standby_6', $node_master, \@recovery_params, "3000", $lsn3);
@recovery_params = (
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'",
"recovery_target_xid = '$recovery_txid'");
test_recovery_standby('time + name + XID',
- 'standby_6', $node_master, \@recovery_params, "2000", $lsn2);
+ 'standby_7', $node_master, \@recovery_params, "2000", $lsn2);
@recovery_params = (
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'");
test_recovery_standby('XID + time + name',
- 'standby_7', $node_master, \@recovery_params, "4000", $lsn4);
+ 'standby_8', $node_master, \@recovery_params, "4000", $lsn4);
+@recovery_params = (
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_lsn = '$recovery_lsn'",);
+test_recovery_standby('XID + time + name + LSN',
+ 'standby_9', $node_master, \@recovery_params, "5000", $lsn5);
On 08/20/2016 04:16 PM, Michael Paquier wrote:
On Sat, Aug 20, 2016 at 10:44 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:
On 20/08/16 02:13, Michael Paquier wrote:
On Fri, Aug 19, 2016 at 10:47 PM, Adrien Nayrat
<adrien.nayrat@dalibo.com> wrote:
Using a PG_TRY/CATCH block the way you do to show to user a different
error message while the original one is actually correct does not
sound like a good idea to me. It complicates the code and the original
pattern matches what is already done for timestamps, where in case of
error you'd get that:
FATAL: invalid input syntax for type timestamp with time zone: "aa"I think that a better solution would be to make clearer in the docs
that pg_lsn is used here. First, in recovery.conf.sample, let's use
pg_lsn and not LSN. Then, in the sgml docs, let's add a reference to
the page of pg_lsn as well:
https://www.postgresql.org/docs/devel/static/datatype-pg-lsn.htmlNow we have that: + <para> + This parameter specifies the LSN of the write-ahead log stream up to + which recovery will proceed. The precise stopping point is also + influenced by <xref linkend="recovery-target-inclusive">. + </para> Perhaps we could just add an additional sentence: "This parameter value is parsed using the system datatype <link=pg_lsn>". Or something like that. Thoughts?+1
So here is the patch resulting in that. After thinking more about that
I have arrived at the conclusion to not use LSN in recovery.conf, but
"WAL position (LSN)", and also mention in the docs that pg_lsn is used
to parse the parameter value. A couple of log messages are changed
similarly. It definitely makes it more understandable for users to
speak of WAL positions.
Good, I prefer this.
I also noticed that the top block of "Recovery Target Settings" needs
to mention recovery_target_lsn, aka when name, xid, lsn and time are
combined in the same recovery.conf, only the last value will be used.
Good catch!
As Julien said, there is nothing to notice that error comes from
recovery.conf.
My fear would be that an user encounters an error like this. Il will be
difficult to link to the recovery.conf.
For the others settings (xid, timeline,name) there is an explicit
message that notice error is in recovery.conf.
I see it is not the case for recovery_target_time.
Should we modify only the documentation or shoud we try to find a
solution to point the origin of error?
--
Adrien NAYRAT
http://dalibo.com - http://dalibo.org
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Aug 22, 2016 at 7:12 PM, Adrien Nayrat <adrien.nayrat@dalibo.com> wrote:
As Julien said, there is nothing to notice that error comes from
recovery.conf.
My fear would be that an user encounters an error like this. Il will be
difficult to link to the recovery.conf.
Thinking a bit wider than that, we may want to know such context for
normal GUC parameters as well, and that's not the case now. Perhaps
there is actually a reason why that's not done for GUCs, but it seems
that it would be useful there as well. That would give another reason
to move all that under the GUC umbrella.
For the others settings (xid, timeline,name) there is an explicit
message that notice error is in recovery.conf.I see it is not the case for recovery_target_time.
Yes, in this case the parameter value is parsed using an datatype _in
function call. And the error message depends on that..
Should we modify only the documentation or should we try to find a
solution to point the origin of error?
The patch as proposed is complicated enough I think, and it would be
good to keep things simple if we can. So having something in the docs
looks fine to me, and that's actually the reference to pg_lsn used to
parse the parameter value.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 8/22/16 8:28 AM, Michael Paquier wrote:
Thinking a bit wider than that, we may want to know such context for
normal GUC parameters as well, and that's not the case now. Perhaps
there is actually a reason why that's not done for GUCs, but it seems
that it would be useful there as well.
GUC parsing generally needs, or used to need, to work under more
constrained circumstances, e.g., no full memory management. That's not
a reason not to try this, but there might be non-obvious problems.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Aug 22, 2016 at 8:28 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Mon, Aug 22, 2016 at 7:12 PM, Adrien Nayrat <adrien.nayrat@dalibo.com> wrote:
As Julien said, there is nothing to notice that error comes from
recovery.conf.
My fear would be that an user encounters an error like this. Il will be
difficult to link to the recovery.conf.Thinking a bit wider than that, we may want to know such context for
normal GUC parameters as well, and that's not the case now. Perhaps
there is actually a reason why that's not done for GUCs, but it seems
that it would be useful there as well. That would give another reason
to move all that under the GUC umbrella.
Maybe so, but that's been tried multiple times without success. If
you think an error context is useful here, and I bet it is, I'd say
just add it and be done with it.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Aug 23, 2016 at 12:49 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 22, 2016 at 8:28 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Mon, Aug 22, 2016 at 7:12 PM, Adrien Nayrat <adrien.nayrat@dalibo.com> wrote:
As Julien said, there is nothing to notice that error comes from
recovery.conf.
My fear would be that an user encounters an error like this. Il will be
difficult to link to the recovery.conf.Thinking a bit wider than that, we may want to know such context for
normal GUC parameters as well, and that's not the case now. Perhaps
there is actually a reason why that's not done for GUCs, but it seems
that it would be useful there as well. That would give another reason
to move all that under the GUC umbrella.Maybe so, but that's been tried multiple times without success. If
you think an error context is useful here, and I bet it is, I'd say
just add it and be done with it.
This has finished by being less ugly than I thought, so I implemented
it as attached. Patch 0001 introduces recovery_target_lsn, and patch
0002 sets up an error context callback generating things like that on
failure:
FATAL: invalid input syntax for type pg_lsn: "popo"
CONTEXT: line 11 of configuration file "recovery.conf", parameter
"recovery_target_lsn"
--
Michael
Attachments:
0001-Introduce-recovery_target_lsn-in-recovery.conf.patchtext/plain; charset=US-ASCII; name=0001-Introduce-recovery_target_lsn-in-recovery.conf.patchDownload
From 6f73e9b02ee9806dd73571e1190983748fcad03d Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@otacoo.com>
Date: Tue, 23 Aug 2016 16:12:03 +0900
Subject: [PATCH 1/2] Introduce recovery_target_lsn in recovery.conf
This new recovery target can be used to stop recovery once a given WAL
position is reached (LSN). Tests and documentation are added.
---
doc/src/sgml/recovery-config.sgml | 24 +++++++--
src/backend/access/transam/recovery.conf.sample | 6 ++-
src/backend/access/transam/xlog.c | 69 +++++++++++++++++++++++++
src/include/access/xlog.h | 1 +
src/test/recovery/t/003_recovery_targets.pl | 28 ++++++++--
5 files changed, 119 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index 26af221..ad977d3 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -157,9 +157,10 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
By default, recovery will recover to the end of the WAL log. The
following parameters can be used to specify an earlier stopping point.
At most one of <varname>recovery_target</>,
- <varname>recovery_target_name</>, <varname>recovery_target_time</>, or
- <varname>recovery_target_xid</> can be used; if more than one of these
- is specified in the configuration file, the last entry will be used.
+ <varname>recovery_target_lsn</>, <varname>recovery_target_name</>,
+ <varname>recovery_target_time</>, or <varname>recovery_target_xid</>
+ can be used; if more than one of these is specified in the configuration
+ file, the last entry will be used.
</para>
<variablelist>
@@ -232,6 +233,23 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="recovery-target-lsn" xreflabel="recovery_target_lsn">
+ <term><varname>recovery_target_lsn</varname> (<type>pg_lsn</type>)
+ <indexterm>
+ <primary><varname>recovery_target_lsn</> recovery parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the LSN of the write-ahead log stream up to
+ which recovery will proceed. The precise stopping point is also
+ influenced by <xref linkend="recovery-target-inclusive">. This
+ parameter is parsed using the system data type
+ <link linkend="datatype-pg-lsn"><type>pg_lsn</></link>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
<para>
diff --git a/src/backend/access/transam/recovery.conf.sample b/src/backend/access/transam/recovery.conf.sample
index b777400..7a16751 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -67,8 +67,8 @@
# must set a recovery target.
#
# You may set a recovery target either by transactionId, by name,
-# or by timestamp. Recovery may either include or exclude the
-# transaction(s) with the recovery target value (ie, stop either
+# by timestamp or by WAL position (LSN). Recovery may either include or
+# exclude the transaction(s) with the recovery target value (ie, stop either
# just after or just before the given target, respectively).
#
#
@@ -78,6 +78,8 @@
#
#recovery_target_xid = ''
#
+#recovery_target_lsn = '' # e.g. '0/70006B8'
+#
#recovery_target_inclusive = true
#
#
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f13f9c1..e3985ee 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -67,6 +67,7 @@
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
+#include "utils/pg_lsn.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
@@ -254,6 +255,7 @@ static RecoveryTargetAction recoveryTargetAction = RECOVERY_TARGET_ACTION_PAUSE;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
+static XLogRecPtr recoveryTargetLSN;
static int recovery_min_apply_delay = 0;
static TimestampTz recoveryDelayUntilTime;
@@ -275,6 +277,7 @@ static bool fast_promote = false;
*/
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
+static XLogRecPtr recoveryStopLSN;
static char recoveryStopName[MAXFNAMELEN];
static bool recoveryStopAfter;
@@ -5078,6 +5081,23 @@ readRecoveryCommandFile(void)
(errmsg_internal("recovery_target_name = '%s'",
recoveryTargetName)));
}
+ else if (strcmp(item->name, "recovery_target_lsn") == 0)
+ {
+ recoveryTarget = RECOVERY_TARGET_LSN;
+
+ /*
+ * Convert the LSN string given by the user to XLogRecPtr form.
+ */
+ recoveryTargetLSN =
+ DatumGetLSN(DirectFunctionCall3(pg_lsn_in,
+ CStringGetDatum(item->value),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ ereport(DEBUG2,
+ (errmsg_internal("recovery_target_lsn = '%X/%X'",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
+ }
else if (strcmp(item->name, "recovery_target") == 0)
{
if (strcmp(item->value, "immediate") == 0)
@@ -5388,11 +5408,29 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopTime = 0;
recoveryStopName[0] = '\0';
return true;
}
+ /* Check if target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ !recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = false;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping before WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
/* Otherwise we only consider stopping before COMMIT or ABORT records. */
if (XLogRecGetRmid(record) != RM_XACT_ID)
return false;
@@ -5467,6 +5505,7 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (isCommit)
@@ -5520,6 +5559,7 @@ recoveryStopsAfter(XLogReaderState *record)
{
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
(void) getRecordTimestamp(record, &recoveryStopTime);
strlcpy(recoveryStopName, recordRestorePointData->rp_name, MAXFNAMELEN);
@@ -5531,6 +5571,23 @@ recoveryStopsAfter(XLogReaderState *record)
}
}
+ /* Check if the target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = true;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping after WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
if (rmid != RM_XACT_ID)
return false;
@@ -5586,6 +5643,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (xact_info == XLOG_XACT_COMMIT ||
@@ -5617,6 +5675,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
recoveryStopTime = 0;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
return true;
}
@@ -6043,6 +6102,11 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("starting point-in-time recovery to \"%s\"",
recoveryTargetName)));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ ereport(LOG,
+ (errmsg("starting point-in-time recovery to WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
else if (recoveryTarget == RECOVERY_TARGET_IMMEDIATE)
ereport(LOG,
(errmsg("starting point-in-time recovery to earliest consistent point")));
@@ -7112,6 +7176,11 @@ StartupXLOG(void)
"%s %s\n",
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ snprintf(reason, sizeof(reason),
+ "at LSN %X/%X\n",
+ (uint32 ) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN);
else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(reason, sizeof(reason),
"at restore point \"%s\"",
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..c9f332c 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -83,6 +83,7 @@ typedef enum
RECOVERY_TARGET_XID,
RECOVERY_TARGET_TIME,
RECOVERY_TARGET_NAME,
+ RECOVERY_TARGET_LSN,
RECOVERY_TARGET_IMMEDIATE
} RecoveryTargetType;
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index d1f6d78..a82545b 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -3,7 +3,7 @@ use strict;
use warnings;
use PostgresNode;
use TestLib;
-use Test::More tests => 7;
+use Test::More tests => 9;
# Create and test a standby from given backup, with a certain
# recovery target.
@@ -86,6 +86,16 @@ my $lsn4 =
$node_master->safe_psql('postgres',
"SELECT pg_create_restore_point('$recovery_name');");
+# And now for a recovery target LSN
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(4001,5000))");
+my $recovery_lsn = $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location()");
+my $lsn5 =
+ $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location();");
+
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(5001,6000))");
+
# Force archiving of WAL file
$node_master->safe_psql('postgres', "SELECT pg_switch_xlog()");
@@ -102,6 +112,9 @@ test_recovery_standby('time', 'standby_3', $node_master, \@recovery_params,
@recovery_params = ("recovery_target_name = '$recovery_name'");
test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"4000", $lsn4);
+@recovery_params = ("recovery_target_lsn = '$recovery_lsn'");
+test_recovery_standby('LSN', 'standby_5', $node_master, \@recovery_params,
+ "5000", $lsn5);
# Multiple targets
# Last entry has priority (note that an array respects the order of items
@@ -111,16 +124,23 @@ test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'");
test_recovery_standby('name + XID + time',
- 'standby_5', $node_master, \@recovery_params, "3000", $lsn3);
+ 'standby_6', $node_master, \@recovery_params, "3000", $lsn3);
@recovery_params = (
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'",
"recovery_target_xid = '$recovery_txid'");
test_recovery_standby('time + name + XID',
- 'standby_6', $node_master, \@recovery_params, "2000", $lsn2);
+ 'standby_7', $node_master, \@recovery_params, "2000", $lsn2);
@recovery_params = (
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'");
test_recovery_standby('XID + time + name',
- 'standby_7', $node_master, \@recovery_params, "4000", $lsn4);
+ 'standby_8', $node_master, \@recovery_params, "4000", $lsn4);
+@recovery_params = (
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_lsn = '$recovery_lsn'",);
+test_recovery_standby('XID + time + name + LSN',
+ 'standby_9', $node_master, \@recovery_params, "5000", $lsn5);
--
2.9.3
0002-Introduce-error-callback-when-parsing-recovery.conf.patchtext/plain; charset=US-ASCII; name=0002-Introduce-error-callback-when-parsing-recovery.conf.patchDownload
From 82a591ca8be457f8be4f13080ededc5daabf3670 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@otacoo.com>
Date: Tue, 23 Aug 2016 16:28:05 +0900
Subject: [PATCH 2/2] Introduce error callback when parsing recovery.conf
At the moment an error occurs when recovery.conf is parsed, generate
a context message related to the parameter used, as follows:
CONTEXT: line 11 of configuration file "recovery.conf", parameter foo
This is useful to understand from where a parsing failure is coming in
recovery.conf, particularly for parameters like recovery_target_time and
recovery_target_lsn that do not have recovery-specific error messages.
---
src/backend/access/transam/xlog.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index e3985ee..9097841 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4935,6 +4935,20 @@ str_time(pg_time_t tnow)
}
/*
+ * error context callback for parsing of recovery.conf
+ *
+ * The argument for the error context must be of type ConfigVariable.
+ */
+static void
+readRecoveryErrorCallback(void *arg)
+{
+ ConfigVariable *item = (ConfigVariable *) arg;
+
+ errcontext("line %d of configuration file \"%s\", parameter \"%s\"",
+ item->sourceline, item->filename, item->name);
+}
+
+/*
* See if there is a recovery command file (recovery.conf), and if so
* read in parameters for archive recovery and XLOG streaming.
*
@@ -4950,6 +4964,7 @@ readRecoveryCommandFile(void)
*head = NULL,
*tail = NULL;
bool recoveryTargetActionSet = false;
+ ErrorContextCallback errcallback;
fd = AllocateFile(RECOVERY_COMMAND_FILE, "r");
@@ -4971,8 +4986,15 @@ readRecoveryCommandFile(void)
FreeFile(fd);
+ /* set up callback to identify file parsing information on failure */
+ errcallback.callback = readRecoveryErrorCallback;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
for (item = head; item; item = item->next)
{
+ errcallback.arg = (void *) item;
+
if (strcmp(item->name, "restore_command") == 0)
{
recoveryRestoreCommand = pstrdup(item->value);
@@ -5179,6 +5201,9 @@ readRecoveryCommandFile(void)
item->name)));
}
+ /* Done, clean up */
+ error_context_stack = errcallback.previous;
+
/*
* Check for compulsory parameters
*/
--
2.9.3
On 23/08/16 09:33, Michael Paquier wrote:
On Tue, Aug 23, 2016 at 12:49 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 22, 2016 at 8:28 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Mon, Aug 22, 2016 at 7:12 PM, Adrien Nayrat <adrien.nayrat@dalibo.com> wrote:
As Julien said, there is nothing to notice that error comes from
recovery.conf.
My fear would be that an user encounters an error like this. Il will be
difficult to link to the recovery.conf.Thinking a bit wider than that, we may want to know such context for
normal GUC parameters as well, and that's not the case now. Perhaps
there is actually a reason why that's not done for GUCs, but it seems
that it would be useful there as well. That would give another reason
to move all that under the GUC umbrella.Maybe so, but that's been tried multiple times without success. If
you think an error context is useful here, and I bet it is, I'd say
just add it and be done with it.This has finished by being less ugly than I thought, so I implemented
it as attached. Patch 0001 introduces recovery_target_lsn, and patch
0002 sets up an error context callback generating things like that on
failure:
FATAL: invalid input syntax for type pg_lsn: "popo"
CONTEXT: line 11 of configuration file "recovery.conf", parameter
"recovery_target_lsn"
Looks very reasonable to me (both patches). Thanks for doing that.
I am inclined to mark this as ready for committer.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 08/23/2016 10:39 AM, Petr Jelinek wrote:
On 23/08/16 09:33, Michael Paquier wrote:
On Tue, Aug 23, 2016 at 12:49 AM, Robert Haas <robertmhaas@gmail.com>
wrote:On Mon, Aug 22, 2016 at 8:28 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Mon, Aug 22, 2016 at 7:12 PM, Adrien Nayrat
<adrien.nayrat@dalibo.com> wrote:As Julien said, there is nothing to notice that error comes from
recovery.conf.
My fear would be that an user encounters an error like this. Il
will be
difficult to link to the recovery.conf.Thinking a bit wider than that, we may want to know such context for
normal GUC parameters as well, and that's not the case now. Perhaps
there is actually a reason why that's not done for GUCs, but it seems
that it would be useful there as well. That would give another reason
to move all that under the GUC umbrella.Maybe so, but that's been tried multiple times without success. If
you think an error context is useful here, and I bet it is, I'd say
just add it and be done with it.This has finished by being less ugly than I thought, so I implemented
it as attached. Patch 0001 introduces recovery_target_lsn, and patch
0002 sets up an error context callback generating things like that on
failure:
FATAL: invalid input syntax for type pg_lsn: "popo"
CONTEXT: line 11 of configuration file "recovery.conf", parameter
"recovery_target_lsn"
Good! Message is clear now for recovery_target_lsn and recovery_target_time.
Thanks for your work.
Looks very reasonable to me (both patches). Thanks for doing that.
I am inclined to mark this as ready for committer.
+1 everything if fine for me.
--
Adrien NAYRAT
http://dalibo.com - http://dalibo.org
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 23 August 2016 at 09:39, Petr Jelinek <petr@2ndquadrant.com> wrote:
Looks very reasonable to me (both patches). Thanks for doing that.
I am inclined to mark this as ready for committer.
Looking at it now.
The messages for recovery_target_lsn don't mention after or before, as
do other targets... e.g.
recoveryStopAfter ? "after" : "before",
My understanding is that if you request an LSN that isn't the exact
end point of a WAL record then it will either stop before or after the
requested point, so that needs to be described in the docs and in the
messages generated prior to starting to search.
Everything else looks in good order.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Aug 23, 2016 at 6:10 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 23 August 2016 at 09:39, Petr Jelinek <petr@2ndquadrant.com> wrote:
Looks very reasonable to me (both patches). Thanks for doing that.
I am inclined to mark this as ready for committer.
Looking at it now.
The messages for recovery_target_lsn don't mention after or before, as
do other targets... e.g.
recoveryStopAfter ? "after" : "before",
My understanding is that if you request an LSN that isn't the exact
end point of a WAL record then it will either stop before or after the
requested point, so that needs to be described in the docs and in the
messages generated prior to starting to search.Everything else looks in good order.
You are right, this message should be completed as such. Do you want
an updated patch?
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Aug 23, 2016 at 8:50 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Tue, Aug 23, 2016 at 6:10 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 23 August 2016 at 09:39, Petr Jelinek <petr@2ndquadrant.com> wrote:
Looks very reasonable to me (both patches). Thanks for doing that.
I am inclined to mark this as ready for committer.
Looking at it now.
The messages for recovery_target_lsn don't mention after or before, as
do other targets... e.g.
recoveryStopAfter ? "after" : "before",
My understanding is that if you request an LSN that isn't the exact
end point of a WAL record then it will either stop before or after the
requested point, so that needs to be described in the docs and in the
messages generated prior to starting to search.Everything else looks in good order.
You are right, this message should be completed as such. Do you want
an updated patch?
Well, I finished by updating the patch anyway.
--
Michael
Attachments:
0001-Introduce-recovery_target_lsn-in-recovery.conf.patchinvalid/octet-stream; name=0001-Introduce-recovery_target_lsn-in-recovery.conf.patchDownload
From a678b7a11a5e4f5a4136274c5044eaf1ed627b27 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@otacoo.com>
Date: Wed, 24 Aug 2016 13:47:43 +0900
Subject: [PATCH 1/2] Introduce recovery_target_lsn in recovery.conf
This new recovery target can be used to stop recovery once a given WAL
position is reached (LSN). Tests and documentation are added.
---
doc/src/sgml/recovery-config.sgml | 24 +++++++--
src/backend/access/transam/recovery.conf.sample | 6 ++-
src/backend/access/transam/xlog.c | 70 +++++++++++++++++++++++++
src/include/access/xlog.h | 1 +
src/test/recovery/t/003_recovery_targets.pl | 28 ++++++++--
5 files changed, 120 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index 26af221..ad977d3 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -157,9 +157,10 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
By default, recovery will recover to the end of the WAL log. The
following parameters can be used to specify an earlier stopping point.
At most one of <varname>recovery_target</>,
- <varname>recovery_target_name</>, <varname>recovery_target_time</>, or
- <varname>recovery_target_xid</> can be used; if more than one of these
- is specified in the configuration file, the last entry will be used.
+ <varname>recovery_target_lsn</>, <varname>recovery_target_name</>,
+ <varname>recovery_target_time</>, or <varname>recovery_target_xid</>
+ can be used; if more than one of these is specified in the configuration
+ file, the last entry will be used.
</para>
<variablelist>
@@ -232,6 +233,23 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="recovery-target-lsn" xreflabel="recovery_target_lsn">
+ <term><varname>recovery_target_lsn</varname> (<type>pg_lsn</type>)
+ <indexterm>
+ <primary><varname>recovery_target_lsn</> recovery parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the LSN of the write-ahead log stream up to
+ which recovery will proceed. The precise stopping point is also
+ influenced by <xref linkend="recovery-target-inclusive">. This
+ parameter is parsed using the system data type
+ <link linkend="datatype-pg-lsn"><type>pg_lsn</></link>.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
<para>
diff --git a/src/backend/access/transam/recovery.conf.sample b/src/backend/access/transam/recovery.conf.sample
index b777400..7a16751 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -67,8 +67,8 @@
# must set a recovery target.
#
# You may set a recovery target either by transactionId, by name,
-# or by timestamp. Recovery may either include or exclude the
-# transaction(s) with the recovery target value (ie, stop either
+# by timestamp or by WAL position (LSN). Recovery may either include or
+# exclude the transaction(s) with the recovery target value (ie, stop either
# just after or just before the given target, respectively).
#
#
@@ -78,6 +78,8 @@
#
#recovery_target_xid = ''
#
+#recovery_target_lsn = '' # e.g. '0/70006B8'
+#
#recovery_target_inclusive = true
#
#
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f13f9c1..4bb4485 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -67,6 +67,7 @@
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/memutils.h"
+#include "utils/pg_lsn.h"
#include "utils/ps_status.h"
#include "utils/relmapper.h"
#include "utils/snapmgr.h"
@@ -254,6 +255,7 @@ static RecoveryTargetAction recoveryTargetAction = RECOVERY_TARGET_ACTION_PAUSE;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
+static XLogRecPtr recoveryTargetLSN;
static int recovery_min_apply_delay = 0;
static TimestampTz recoveryDelayUntilTime;
@@ -275,6 +277,7 @@ static bool fast_promote = false;
*/
static TransactionId recoveryStopXid;
static TimestampTz recoveryStopTime;
+static XLogRecPtr recoveryStopLSN;
static char recoveryStopName[MAXFNAMELEN];
static bool recoveryStopAfter;
@@ -5078,6 +5081,23 @@ readRecoveryCommandFile(void)
(errmsg_internal("recovery_target_name = '%s'",
recoveryTargetName)));
}
+ else if (strcmp(item->name, "recovery_target_lsn") == 0)
+ {
+ recoveryTarget = RECOVERY_TARGET_LSN;
+
+ /*
+ * Convert the LSN string given by the user to XLogRecPtr form.
+ */
+ recoveryTargetLSN =
+ DatumGetLSN(DirectFunctionCall3(pg_lsn_in,
+ CStringGetDatum(item->value),
+ ObjectIdGetDatum(InvalidOid),
+ Int32GetDatum(-1)));
+ ereport(DEBUG2,
+ (errmsg_internal("recovery_target_lsn = '%X/%X'",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
+ }
else if (strcmp(item->name, "recovery_target") == 0)
{
if (strcmp(item->value, "immediate") == 0)
@@ -5388,8 +5408,26 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ return true;
+ }
+
+ /* Check if target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ !recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = false;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
recoveryStopTime = 0;
recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping before WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
return true;
}
@@ -5467,6 +5505,7 @@ recoveryStopsBefore(XLogReaderState *record)
recoveryStopAfter = false;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (isCommit)
@@ -5520,6 +5559,7 @@ recoveryStopsAfter(XLogReaderState *record)
{
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = InvalidXLogRecPtr;
(void) getRecordTimestamp(record, &recoveryStopTime);
strlcpy(recoveryStopName, recordRestorePointData->rp_name, MAXFNAMELEN);
@@ -5531,6 +5571,23 @@ recoveryStopsAfter(XLogReaderState *record)
}
}
+ /* Check if the target LSN has been reached */
+ if (recoveryTarget == RECOVERY_TARGET_LSN &&
+ recoveryTargetInclusive &&
+ record->ReadRecPtr >= recoveryTargetLSN)
+ {
+ recoveryStopAfter = true;
+ recoveryStopXid = InvalidTransactionId;
+ recoveryStopLSN = record->ReadRecPtr;
+ recoveryStopTime = 0;
+ recoveryStopName[0] = '\0';
+ ereport(LOG,
+ (errmsg("recovery stopping after WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN)));
+ return true;
+ }
+
if (rmid != RM_XACT_ID)
return false;
@@ -5586,6 +5643,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = recordXid;
recoveryStopTime = recordXtime;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
if (xact_info == XLOG_XACT_COMMIT ||
@@ -5617,6 +5675,7 @@ recoveryStopsAfter(XLogReaderState *record)
recoveryStopAfter = true;
recoveryStopXid = InvalidTransactionId;
recoveryStopTime = 0;
+ recoveryStopLSN = InvalidXLogRecPtr;
recoveryStopName[0] = '\0';
return true;
}
@@ -6043,6 +6102,11 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("starting point-in-time recovery to \"%s\"",
recoveryTargetName)));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ ereport(LOG,
+ (errmsg("starting point-in-time recovery to WAL position (LSN) \"%X/%X\"",
+ (uint32) (recoveryTargetLSN >> 32),
+ (uint32) recoveryTargetLSN)));
else if (recoveryTarget == RECOVERY_TARGET_IMMEDIATE)
ereport(LOG,
(errmsg("starting point-in-time recovery to earliest consistent point")));
@@ -7112,6 +7176,12 @@ StartupXLOG(void)
"%s %s\n",
recoveryStopAfter ? "after" : "before",
timestamptz_to_str(recoveryStopTime));
+ else if (recoveryTarget == RECOVERY_TARGET_LSN)
+ snprintf(reason, sizeof(reason),
+ "%s LSN %X/%X\n",
+ recoveryStopAfter ? "after" : "before",
+ (uint32 ) (recoveryStopLSN >> 32),
+ (uint32) recoveryStopLSN);
else if (recoveryTarget == RECOVERY_TARGET_NAME)
snprintf(reason, sizeof(reason),
"at restore point \"%s\"",
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..c9f332c 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -83,6 +83,7 @@ typedef enum
RECOVERY_TARGET_XID,
RECOVERY_TARGET_TIME,
RECOVERY_TARGET_NAME,
+ RECOVERY_TARGET_LSN,
RECOVERY_TARGET_IMMEDIATE
} RecoveryTargetType;
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index d1f6d78..a82545b 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -3,7 +3,7 @@ use strict;
use warnings;
use PostgresNode;
use TestLib;
-use Test::More tests => 7;
+use Test::More tests => 9;
# Create and test a standby from given backup, with a certain
# recovery target.
@@ -86,6 +86,16 @@ my $lsn4 =
$node_master->safe_psql('postgres',
"SELECT pg_create_restore_point('$recovery_name');");
+# And now for a recovery target LSN
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(4001,5000))");
+my $recovery_lsn = $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location()");
+my $lsn5 =
+ $node_master->safe_psql('postgres', "SELECT pg_current_xlog_location();");
+
+$node_master->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(5001,6000))");
+
# Force archiving of WAL file
$node_master->safe_psql('postgres', "SELECT pg_switch_xlog()");
@@ -102,6 +112,9 @@ test_recovery_standby('time', 'standby_3', $node_master, \@recovery_params,
@recovery_params = ("recovery_target_name = '$recovery_name'");
test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"4000", $lsn4);
+@recovery_params = ("recovery_target_lsn = '$recovery_lsn'");
+test_recovery_standby('LSN', 'standby_5', $node_master, \@recovery_params,
+ "5000", $lsn5);
# Multiple targets
# Last entry has priority (note that an array respects the order of items
@@ -111,16 +124,23 @@ test_recovery_standby('name', 'standby_4', $node_master, \@recovery_params,
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'");
test_recovery_standby('name + XID + time',
- 'standby_5', $node_master, \@recovery_params, "3000", $lsn3);
+ 'standby_6', $node_master, \@recovery_params, "3000", $lsn3);
@recovery_params = (
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'",
"recovery_target_xid = '$recovery_txid'");
test_recovery_standby('time + name + XID',
- 'standby_6', $node_master, \@recovery_params, "2000", $lsn2);
+ 'standby_7', $node_master, \@recovery_params, "2000", $lsn2);
@recovery_params = (
"recovery_target_xid = '$recovery_txid'",
"recovery_target_time = '$recovery_time'",
"recovery_target_name = '$recovery_name'");
test_recovery_standby('XID + time + name',
- 'standby_7', $node_master, \@recovery_params, "4000", $lsn4);
+ 'standby_8', $node_master, \@recovery_params, "4000", $lsn4);
+@recovery_params = (
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_lsn = '$recovery_lsn'",);
+test_recovery_standby('XID + time + name + LSN',
+ 'standby_9', $node_master, \@recovery_params, "5000", $lsn5);
--
2.9.3
0002-Introduce-error-callback-when-parsing-recovery.conf.patchinvalid/octet-stream; name=0002-Introduce-error-callback-when-parsing-recovery.conf.patchDownload
From 84b211ad56506ea1b9517a71ec3fb4010936e4da Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@otacoo.com>
Date: Tue, 23 Aug 2016 16:28:05 +0900
Subject: [PATCH 2/2] Introduce error callback when parsing recovery.conf
At the moment an error occurs when recovery.conf is parsed, generate
a context message related to the parameter used, as follows:
CONTEXT: line 11 of configuration file "recovery.conf", parameter foo
This is useful to understand from where a parsing failure is coming in
recovery.conf, particularly for parameters like recovery_target_time and
recovery_target_lsn that do not have recovery-specific error messages.
---
src/backend/access/transam/xlog.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 4bb4485..4496f26 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4935,6 +4935,20 @@ str_time(pg_time_t tnow)
}
/*
+ * error context callback for parsing of recovery.conf
+ *
+ * The argument for the error context must be of type ConfigVariable.
+ */
+static void
+readRecoveryErrorCallback(void *arg)
+{
+ ConfigVariable *item = (ConfigVariable *) arg;
+
+ errcontext("line %d of configuration file \"%s\", parameter \"%s\"",
+ item->sourceline, item->filename, item->name);
+}
+
+/*
* See if there is a recovery command file (recovery.conf), and if so
* read in parameters for archive recovery and XLOG streaming.
*
@@ -4950,6 +4964,7 @@ readRecoveryCommandFile(void)
*head = NULL,
*tail = NULL;
bool recoveryTargetActionSet = false;
+ ErrorContextCallback errcallback;
fd = AllocateFile(RECOVERY_COMMAND_FILE, "r");
@@ -4971,8 +4986,15 @@ readRecoveryCommandFile(void)
FreeFile(fd);
+ /* set up callback to identify file parsing information on failure */
+ errcallback.callback = readRecoveryErrorCallback;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
for (item = head; item; item = item->next)
{
+ errcallback.arg = (void *) item;
+
if (strcmp(item->name, "restore_command") == 0)
{
recoveryRestoreCommand = pstrdup(item->value);
@@ -5179,6 +5201,9 @@ readRecoveryCommandFile(void)
item->name)));
}
+ /* Done, clean up */
+ error_context_stack = errcallback.previous;
+
/*
* Check for compulsory parameters
*/
--
2.9.3
On 24 August 2016 at 05:50, Michael Paquier <michael.paquier@gmail.com> wrote:
Everything else looks in good order.
Committed. Thanks.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Sep 4, 2016 at 1:57 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 24 August 2016 at 05:50, Michael Paquier <michael.paquier@gmail.com> wrote:
Everything else looks in good order.
Committed. Thanks.
Thanks for the commit!
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Sep 4, 2016 at 8:05 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Sun, Sep 4, 2016 at 1:57 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 24 August 2016 at 05:50, Michael Paquier <michael.paquier@gmail.com> wrote:
Everything else looks in good order.
Committed. Thanks.
Thanks for the commit!
By the way, what has been committed does not include the patch adding
the parsing context in case of an error as wanted upthread. Perhaps
that's not worth adding now as there is the GUC refactoring
potentially happening for the recovery parameters, so I don't mind
much. Just that's worth mentioning.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 4 September 2016 at 04:50, Michael Paquier <michael.paquier@gmail.com> wrote:
On Sun, Sep 4, 2016 at 8:05 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Sun, Sep 4, 2016 at 1:57 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 24 August 2016 at 05:50, Michael Paquier <michael.paquier@gmail.com> wrote:
Everything else looks in good order.
Committed. Thanks.
Thanks for the commit!
No problem, it was a good patch. Since I moan to others about lack of
docs, tests etc, I'll do the same here and compliment you on providing
a well rounded patch with docs, tests that does what it says in a
clean way.
By the way, what has been committed does not include the patch adding
the parsing context in case of an error as wanted upthread. Perhaps
that's not worth adding now as there is the GUC refactoring
potentially happening for the recovery parameters, so I don't mind
much. Just that's worth mentioning.
Hmm, that was unintentional. If something stalls the recovery
parameter project, please remind me to commit that as well.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Sep 4, 2016 at 3:02 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 4 September 2016 at 04:50, Michael Paquier <michael.paquier@gmail.com> wrote:
On Sun, Sep 4, 2016 at 8:05 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Sun, Sep 4, 2016 at 1:57 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 24 August 2016 at 05:50, Michael Paquier <michael.paquier@gmail.com> wrote:
Everything else looks in good order.
Committed. Thanks.
Thanks for the commit!
No problem, it was a good patch. Since I moan to others about lack of
docs, tests etc, I'll do the same here and compliment you on providing
a well rounded patch with docs, tests that does what it says in a
clean way.
Thanks a lot! That's really motivating! I don't think I deserve that.
By the way, what has been committed does not include the patch adding
the parsing context in case of an error as wanted upthread. Perhaps
that's not worth adding now as there is the GUC refactoring
potentially happening for the recovery parameters, so I don't mind
much. Just that's worth mentioning.Hmm, that was unintentional. If something stalls the recovery
parameter project, please remind me to commit that as well.
That's what is here as 0002:
/messages/by-id/CAB7nPqSG5adk7=nBgKqy8uQ1tXkeX212jboO_hJbZDVehD3w8Q@mail.gmail.com
This makes produce a context message should an error occur, like that:
FATAL: invalid input syntax for type pg_lsn: "popo"
CONTEXT: line 11 of configuration file "recovery.conf", parameter
"recovery_target_lsn"
And as outlined by reviewers upthread this is particularly useful for
recovery_target_time and recovery_target_lsn because those would bump
on a parsing error message from the data type, letting users no
information that it came from recovery.conf :(
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
At 2016-09-04 07:02:01 +0100, simon@2ndquadrant.com wrote:
By the way, what has been committed does not include the patch
adding the parsing context in case of an error as wanted upthread.
Perhaps that's not worth adding now as there is the GUC refactoring
potentially happening for the recovery parameters, so I don't mind
much. Just that's worth mentioning.Hmm, that was unintentional. If something stalls the recovery
parameter project, please remind me to commit that as well.
I'm aware of this, and will adjust accordingly in the GUC patch. Thanks
for the heads up.
-- Abhijit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 4 September 2016 at 14:32, Abhijit Menon-Sen <ams@2ndquadrant.com> wrote:
At 2016-09-04 07:02:01 +0100, simon@2ndquadrant.com wrote:
By the way, what has been committed does not include the patch
adding the parsing context in case of an error as wanted upthread.
Perhaps that's not worth adding now as there is the GUC refactoring
potentially happening for the recovery parameters, so I don't mind
much. Just that's worth mentioning.Hmm, that was unintentional. If something stalls the recovery
parameter project, please remind me to commit that as well.I'm aware of this, and will adjust accordingly in the GUC patch. Thanks
for the heads up.
I noticed we don't mention what LSN is anywhere, so I'd like to apply
the following doc patch also.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
doc_lsn.v1.patchapplication/octet-stream; name=doc_lsn.v1.patchDownload
diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml
index 38f111e..b8f572e 100644
--- a/doc/src/sgml/acronyms.sgml
+++ b/doc/src/sgml/acronyms.sgml
@@ -381,6 +381,16 @@
</varlistentry>
<varlistentry>
+ <term><acronym>LSN</acronym></term>
+ <listitem>
+ <para>
+ Log Sequence Number, a byte offset in the stream of <acronym>WAL</acronym> changes.
+ See <link linkend="wal-internals">WAL Internals</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><acronym>MSVC</acronym></term>
<listitem>
<para>
diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml
index 503ea8a..6de3c2e 100644
--- a/doc/src/sgml/wal.sgml
+++ b/doc/src/sgml/wal.sgml
@@ -724,6 +724,10 @@
<sect1 id="wal-internals">
<title>WAL Internals</title>
+ <indexterm zone="wal-internals">
+ <primary>LSN</primary>
+ </indexterm>
+
<para>
<acronym>WAL</acronym> is automatically enabled; no action is
required from the administrator except ensuring that the
@@ -733,6 +737,16 @@
</para>
<para>
+ <acronym>WAL</acronym> records are appended to the <acronym>WAL</acronym>
+ logs as each new record is written. The insert position is described by
+ a Log Sequence Number (<acronym>LSN</acronym>) that is a byte offset into
+ the logs, increasing monotonically with each new record. Two
+ <acronym>LSN</acronym>s can be compared to calculate the volume of
+ <acronym>WAL</acronym> data that separates them, so are used to measure
+ progress of <acronym>WAL</acronym> during replication and recovery.
+ </para>
+
+ <para>
<acronym>WAL</acronym> logs are stored in the directory
<filename>pg_xlog</filename> under the data directory, as a set of
segment files, normally each 16 MB in size (but the size can be changed
On Sun, Sep 4, 2016 at 11:30 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 4 September 2016 at 14:32, Abhijit Menon-Sen <ams@2ndquadrant.com> wrote:
At 2016-09-04 07:02:01 +0100, simon@2ndquadrant.com wrote:
By the way, what has been committed does not include the patch
adding the parsing context in case of an error as wanted upthread.
Perhaps that's not worth adding now as there is the GUC refactoring
potentially happening for the recovery parameters, so I don't mind
much. Just that's worth mentioning.Hmm, that was unintentional. If something stalls the recovery
parameter project, please remind me to commit that as well.I'm aware of this, and will adjust accordingly in the GUC patch. Thanks
for the heads up.I noticed we don't mention what LSN is anywhere, so I'd like to apply
the following doc patch also.
+1 for the idea. What do you think about adding a mention to the
system data type pg_lsn?
<para>
+ <acronym>WAL</acronym> records are appended to the <acronym>WAL</acronym>
+ logs as each new record is written. The insert position is described by
+ a Log Sequence Number (<acronym>LSN</acronym>) that is a byte offset into
+ the logs, increasing monotonically with each new record. Two
+ <acronym>LSN</acronym>s can be compared to calculate the volume of
+ <acronym>WAL</acronym> data that separates them, so are used to measure
+ progress of <acronym>WAL</acronym> during replication and recovery.
+ </para>
Here we could for example append a sentence like "The system data type
<link linkend="datatype-pg-lsn"><type>pg_lsn</></link> is an internal
representation of the LSN".
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 5 September 2016 at 06:55, Michael Paquier <michael.paquier@gmail.com> wrote:
On Sun, Sep 4, 2016 at 11:30 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
I noticed we don't mention what LSN is anywhere, so I'd like to apply
the following doc patch also.+1 for the idea. What do you think about adding a mention to the
system data type pg_lsn?<para> + <acronym>WAL</acronym> records are appended to the <acronym>WAL</acronym> + logs as each new record is written. The insert position is described by + a Log Sequence Number (<acronym>LSN</acronym>) that is a byte offset into + the logs, increasing monotonically with each new record. Two + <acronym>LSN</acronym>s can be compared to calculate the volume of + <acronym>WAL</acronym> data that separates them, so are used to measure + progress of <acronym>WAL</acronym> during replication and recovery. + </para> Here we could for example append a sentence like "The system data type <link linkend="datatype-pg-lsn"><type>pg_lsn</></link> is an internal representation of the LSN".
Good input, thanks. v2 attached.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
doc_lsn.v2.patchapplication/octet-stream; name=doc_lsn.v2.patchDownload
diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml
index 38f111e..bf2273f 100644
--- a/doc/src/sgml/acronyms.sgml
+++ b/doc/src/sgml/acronyms.sgml
@@ -381,6 +381,16 @@
</varlistentry>
<varlistentry>
+ <term><acronym>LSN</acronym></term>
+ <listitem>
+ <para>
+ Log Sequence Number, see <link linkend="datatype-pg-lsn"><type>pg_lsn</></link>
+ and <link linkend="wal-internals">WAL Internals</link>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><acronym>MSVC</acronym></term>
<listitem>
<para>
diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml
index 503ea8a..9ae6547 100644
--- a/doc/src/sgml/wal.sgml
+++ b/doc/src/sgml/wal.sgml
@@ -724,6 +724,10 @@
<sect1 id="wal-internals">
<title>WAL Internals</title>
+ <indexterm zone="wal-internals">
+ <primary>LSN</primary>
+ </indexterm>
+
<para>
<acronym>WAL</acronym> is automatically enabled; no action is
required from the administrator except ensuring that the
@@ -733,6 +737,18 @@
</para>
<para>
+ <acronym>WAL</acronym> records are appended to the <acronym>WAL</acronym>
+ logs as each new record is written. The insert position is described by
+ a Log Sequence Number (<acronym>LSN</acronym>) that is a byte offset into
+ the logs, increasing monotonically with each new record.
+ <acronym>LSN</acronym> values are returned as the datatype
+ <link linkend="datatype-pg-lsn"><type>pg_lsn</></link>. Values can be
+ compared to calculate the volume of <acronym>WAL</acronym> data that
+ separates them, so they are used to measure the progress of replication
+ and recovery.
+ </para>
+
+ <para>
<acronym>WAL</acronym> logs are stored in the directory
<filename>pg_xlog</filename> under the data directory, as a set of
segment files, normally each 16 MB in size (but the size can be changed
On Mon, Sep 5, 2016 at 4:02 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 5 September 2016 at 06:55, Michael Paquier <michael.paquier@gmail.com> wrote:
On Sun, Sep 4, 2016 at 11:30 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
I noticed we don't mention what LSN is anywhere, so I'd like to apply
the following doc patch also.+1 for the idea. What do you think about adding a mention to the
system data type pg_lsn?<para> + <acronym>WAL</acronym> records are appended to the <acronym>WAL</acronym> + logs as each new record is written. The insert position is described by + a Log Sequence Number (<acronym>LSN</acronym>) that is a byte offset into + the logs, increasing monotonically with each new record. Two + <acronym>LSN</acronym>s can be compared to calculate the volume of + <acronym>WAL</acronym> data that separates them, so are used to measure + progress of <acronym>WAL</acronym> during replication and recovery. + </para> Here we could for example append a sentence like "The system data type <link linkend="datatype-pg-lsn"><type>pg_lsn</></link> is an internal representation of the LSN".Good input, thanks. v2 attached.
Looks good to me. Thanks for the wordsmithing.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers