[PATCH] Transaction traceability - txid_status(bigint)
Started by Craig Ringerover 9 years ago73 messages
Hi all
Following on from
bigint txids vs 'xid' type, new txid_recent(bigint) => xid
/messages/by-id/CAMsr+YFDZMN_iZ7KrRoe+j0KVLQvFVgvZxbcVxR-MLjgtoZugA@mail.gmail.com
here's a patch that implements a txid_status(bigint) function to report the
commit status of a function.
If an application is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
A future protocol enhancement to report txid assignment would be very
useful, but quite separate to this.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From a298bb242716fbf1526bf9f949a94740a9d975fc Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/5] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 11 ++++-
src/backend/access/transam/clog.c | 23 ----------
src/backend/catalog/system_views.sql | 20 +++++++++
src/backend/utils/adt/txid.c | 82 ++++++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 ++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/txid.out | 50 ++++++++++++++++++++++
src/test/regress/sql/txid.sql | 36 ++++++++++++++++
8 files changed, 223 insertions(+), 24 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7830334..c2a0aeb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -16911,6 +16911,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -16929,7 +16933,7 @@ SELECT collation for ('foo' COLLATE "de_DE");
<row>
<entry><literal><function>txid_current()</function></literal></entry>
<entry><type>bigint</type></entry>
- <entry>get current transaction ID, assigning a new one if the current transaction does not have one</entry>
+ <entry>get current 64-bit transaction ID with epoch, assigning a new one if the current transaction does not have one</entry>
</row>
<row>
<entry><literal><function>txid_current_snapshot()</function></literal></entry>
@@ -16956,6 +16960,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - committed, aborted, in-progress, or null if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 4fc5d5a..4194195 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1008,6 +1008,26 @@ LANGUAGE INTERNAL
STRICT IMMUTABLE PARALLEL SAFE
AS 'jsonb_insert';
+CREATE TYPE txid_status AS ENUM ('committed', 'in-progress', 'aborted');
+
+CREATE FUNCTION
+ txid_status(txid bigint)
+RETURNS txid_status
+LANGUAGE sql
+VOLATILE PARALLEL SAFE
+AS $$
+SELECT CASE
+ WHEN s IS NULL THEN NULL::txid_status
+ WHEN s = -1 THEN 'aborted'::txid_status
+ WHEN s = 0 THEN 'in-progress'::txid_status
+ WHEN s = 1 THEN 'committed'::txid_status
+END
+FROM pg_catalog.txid_status_internal($1) s;
+$$;
+
+COMMENT ON FUNCTION txid_status(bigint)
+IS 'get commit status of given recent xid or null if too old';
+
-- The default permissions for functions mean that anyone can execute them.
-- A number of functions shouldn't be executable by just anyone, but rather
-- than use explicit 'superuser()' checks in those functions, we use the GRANT
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index c2069a9..0480c99 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,9 +21,11 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "catalog/pg_proc.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "libpq/pqformat.h"
@@ -117,6 +119,50 @@ convert_xid(TransactionId xid, const TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound detection.
+ * ERRORs if the txid is in the future. Returns permanent XIDs unchanged.
+ * Otherwise returns the 32-bit xid and sets the wraparound param to true
+ * if wraparound is detected, false otherwise.
+ */
+static TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *wraparound)
+{
+ uint32 xid_epoch = (uint32)(xid_with_epoch >>32);
+ TransactionId xid = (TransactionId)(xid_with_epoch);
+ TxidEpoch now_epoch;
+
+ load_xid_epoch(&now_epoch);
+
+ *wraparound = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch && xid < now_epoch.last_xid))
+ {
+ /* xid too far in the past */
+ *wraparound = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
+
+/*
* txid comparator for qsort/bsearch
*/
static int
@@ -354,6 +400,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -637,3 +686,36 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Underlying implementation of txid_status, which is mapped to an enum in
+ * system_views.sql.
+ */
+Datum
+txid_status_internal(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid = get_xid_in_recent_past(xid_with_epoch, &wraparound);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else if (TransactionIdIsCurrentTransactionId(xid) || TransactionIdIsInProgress(xid))
+ PG_RETURN_INT32(0);
+ else if (TransactionIdDidCommit(xid))
+ PG_RETURN_INT32(1);
+ else if (TransactionIdDidAbort(xid))
+ PG_RETURN_INT32(-1);
+ else
+ /* shouldn't happen */
+ ereport(ERROR,
+ (errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 270dd21..b0b25e6 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4903,6 +4903,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index ddd217e..61299c5 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -238,3 +238,53 @@ SELECT txid_snapshot '1:9223372036854775808:3';
ERROR: invalid input syntax for type txid_snapshot: "1:9223372036854775808:3"
LINE 1: SELECT txid_snapshot '1:9223372036854775808:3';
^
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in-progress
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index b6650b9..50d4461 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -52,3 +52,39 @@ select txid_visible_in_snapshot('1000100010001015', '1000100010001000:1000100010
-- test 64bit overflow
SELECT txid_snapshot '1:9223372036854775807:3';
SELECT txid_snapshot '1:9223372036854775808:3';
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
+
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 20 August 2016 at 21:24, Craig Ringer <craig@2ndquadrant.com> wrote:
Hi all
Following on from
bigint txids vs 'xid' type, new txid_recent(bigint) => xid
Ahem. Forgot to squash in a fixup commit. Updated patch of
txid_status(bigint) attachd.
A related patch follows, adding a new txid_current_ifassigned(bigint)
function as suggested by Jim Nasby. It's usefully connected to
txid_status() and might as well be added at the same time.
Since it builds on the same history I've also attached an updated version
of txid_recent(bigint) now called txid_convert_ifrecent(bigint), per the
above-linked thread.
Finally, and not intended for commit, is a useful test function I wrote to
cause extremely rapid xid wraparound, bundled up into a src/test/regress
test case. txid_incinerate() can jump the server about UINT32/2 xids in ~2
seconds if fsync is off, making it handy for testing. Posting so others
can use it for their own test needs later and because it's useful for
testing these patches that touch on the xid epoch.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From fc6174eada3e73f86934cd7e4f4b701c70324ec2 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/4] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 28 +++++++++++-
src/backend/access/transam/clog.c | 23 ----------
src/backend/catalog/system_views.sql | 20 +++++++++
src/backend/utils/adt/txid.c | 82 ++++++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 ++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/txid.out | 50 ++++++++++++++++++++++
src/test/regress/sql/txid.sql | 36 ++++++++++++++++
8 files changed, 240 insertions(+), 24 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7830334..ae01ed3 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -16911,6 +16911,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -16929,7 +16933,7 @@ SELECT collation for ('foo' COLLATE "de_DE");
<row>
<entry><literal><function>txid_current()</function></literal></entry>
<entry><type>bigint</type></entry>
- <entry>get current transaction ID, assigning a new one if the current transaction does not have one</entry>
+ <entry>get current 64-bit transaction ID with epoch, assigning a new one if the current transaction does not have one</entry>
</row>
<row>
<entry><literal><function>txid_current_snapshot()</function></literal></entry>
@@ -16956,6 +16960,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - committed, aborted, in-progress, or null if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17026,6 +17035,23 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Any recent transaction can be identified as one of
+ <itemizedlist>
+ <listitem><para>in-progress</></>
+ <listitem><para>committed</></>
+ <listitem><para>aborted</></>
+ </itemizedlist>
+ Prepared transactions are identified as <literal>in-progress</>.
+ The commit status of transactions older than the transaction ID wrap-around
+ threshold is no longer known by the system, so <function>txid_status</>
+ returns <literal>NULL</> for such transactions. Applications may use
+ <function>txid_status</> to determine whether a transaction committed
+ or aborted when the application and/or database server crashed or lost
+ connection while a <literal>COMMIT</literal> command was in-progress.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 4fc5d5a..4194195 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1008,6 +1008,26 @@ LANGUAGE INTERNAL
STRICT IMMUTABLE PARALLEL SAFE
AS 'jsonb_insert';
+CREATE TYPE txid_status AS ENUM ('committed', 'in-progress', 'aborted');
+
+CREATE FUNCTION
+ txid_status(txid bigint)
+RETURNS txid_status
+LANGUAGE sql
+VOLATILE PARALLEL SAFE
+AS $$
+SELECT CASE
+ WHEN s IS NULL THEN NULL::txid_status
+ WHEN s = -1 THEN 'aborted'::txid_status
+ WHEN s = 0 THEN 'in-progress'::txid_status
+ WHEN s = 1 THEN 'committed'::txid_status
+END
+FROM pg_catalog.txid_status_internal($1) s;
+$$;
+
+COMMENT ON FUNCTION txid_status(bigint)
+IS 'get commit status of given recent xid or null if too old';
+
-- The default permissions for functions mean that anyone can execute them.
-- A number of functions shouldn't be executable by just anyone, but rather
-- than use explicit 'superuser()' checks in those functions, we use the GRANT
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index c2069a9..0480c99 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,9 +21,11 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "catalog/pg_proc.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "libpq/pqformat.h"
@@ -117,6 +119,50 @@ convert_xid(TransactionId xid, const TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound detection.
+ * ERRORs if the txid is in the future. Returns permanent XIDs unchanged.
+ * Otherwise returns the 32-bit xid and sets the wraparound param to true
+ * if wraparound is detected, false otherwise.
+ */
+static TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *wraparound)
+{
+ uint32 xid_epoch = (uint32)(xid_with_epoch >>32);
+ TransactionId xid = (TransactionId)(xid_with_epoch);
+ TxidEpoch now_epoch;
+
+ load_xid_epoch(&now_epoch);
+
+ *wraparound = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch && xid < now_epoch.last_xid))
+ {
+ /* xid too far in the past */
+ *wraparound = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
+
+/*
* txid comparator for qsort/bsearch
*/
static int
@@ -354,6 +400,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -637,3 +686,36 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Underlying implementation of txid_status, which is mapped to an enum in
+ * system_views.sql.
+ */
+Datum
+txid_status_internal(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid = get_xid_in_recent_past(xid_with_epoch, &wraparound);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else if (TransactionIdIsCurrentTransactionId(xid) || TransactionIdIsInProgress(xid))
+ PG_RETURN_INT32(0);
+ else if (TransactionIdDidCommit(xid))
+ PG_RETURN_INT32(1);
+ else if (TransactionIdDidAbort(xid))
+ PG_RETURN_INT32(-1);
+ else
+ /* shouldn't happen */
+ ereport(ERROR,
+ (errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 270dd21..b0b25e6 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4903,6 +4903,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index ddd217e..61299c5 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -238,3 +238,53 @@ SELECT txid_snapshot '1:9223372036854775808:3';
ERROR: invalid input syntax for type txid_snapshot: "1:9223372036854775808:3"
LINE 1: SELECT txid_snapshot '1:9223372036854775808:3';
^
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in-progress
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index b6650b9..50d4461 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -52,3 +52,39 @@ select txid_visible_in_snapshot('1000100010001015', '1000100010001000:1000100010
-- test 64bit overflow
SELECT txid_snapshot '1:9223372036854775807:3';
SELECT txid_snapshot '1:9223372036854775808:3';
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
+
--
2.5.5
0002-Add-txid_current_ifassigned.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_current_ifassigned.patchDownload
From 3dd411732f917d915182b02561aba9e94749267b Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:46:53 +0800
Subject: [PATCH 2/4] Add txid_current_ifassigned()
Add a variant of txid_current() that returns null if no xid is assigned
instead of assigning one or, on a replica, ERRORing.
Per suggestion from Jim Nasby
---
doc/src/sgml/func.sgml | 9 +++++++++
src/backend/utils/adt/txid.c | 21 +++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 ++
src/test/regress/expected/txid.out | 15 +++++++++++++++
src/test/regress/sql/txid.sql | 6 ++++++
5 files changed, 53 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ae01ed3..14cb067 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -16892,6 +16892,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
</indexterm>
<indexterm>
+ <primary>txid_current_ifassigned</primary>
+ </indexterm>
+
+ <indexterm>
<primary>txid_current_snapshot</primary>
</indexterm>
@@ -16936,6 +16940,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>get current 64-bit transaction ID with epoch, assigning a new one if the current transaction does not have one</entry>
</row>
<row>
+ <entry><literal><function>txid_current_ifassigned()</function></literal></entry>
+ <entry><type>bigint</type></entry>
+ <entry>same as <function>txid_current()</function> but returns null instead of assigning an xid if none is already assigned</entry>
+ </row>
+ <row>
<entry><literal><function>txid_current_snapshot()</function></literal></entry>
<entry><type>txid_snapshot</type></entry>
<entry>get current snapshot</entry>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 0480c99..9c3c8da 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -426,6 +426,27 @@ txid_current(PG_FUNCTION_ARGS)
}
/*
+ * Same as txid_current() but doesn't assign a new xid if there isn't one
+ * yet.
+ */
+Datum
+txid_current_ifassigned(PG_FUNCTION_ARGS)
+{
+ txid val;
+ TxidEpoch state;
+ TransactionId topxid = GetTopTransactionIdIfAny();
+
+ if (topxid == InvalidTransactionId)
+ PG_RETURN_NULL();
+
+ load_xid_epoch(&state);
+
+ val = convert_xid(topxid, &state);
+
+ PG_RETURN_INT64(val);
+}
+
+/*
* txid_current_snapshot() returns txid_snapshot
*
* Return current snapshot in TXID format
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b0b25e6..8db47d2 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4893,6 +4893,8 @@ DATA(insert OID = 2942 ( txid_snapshot_send PGNSP PGUID 12 1 0 0 0 f f f f t
DESCR("I/O");
DATA(insert OID = 2943 ( txid_current PGNSP PGUID 12 1 0 0 0 f f f f t f s u 0 0 20 "" _null_ _null_ _null_ _null_ _null_ txid_current _null_ _null_ _null_ ));
DESCR("get current transaction ID");
+DATA(insert OID = 3348 ( txid_current_ifassigned PGNSP PGUID 12 1 0 0 0 f f f f t f s u 0 0 20 "" _null_ _null_ _null_ _null_ _null_ txid_current_ifassigned _null_ _null_ _null_ ));
+DESCR("get current transaction ID");
DATA(insert OID = 2944 ( txid_current_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f s s 0 0 2970 "" _null_ _null_ _null_ _null_ _null_ txid_current_snapshot _null_ _null_ _null_ ));
DESCR("get current snapshot");
DATA(insert OID = 2945 ( txid_snapshot_xmin PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 20 "2970" _null_ _null_ _null_ _null_ _null_ txid_snapshot_xmin _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 61299c5..df09f62 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -238,6 +238,21 @@ SELECT txid_snapshot '1:9223372036854775808:3';
ERROR: invalid input syntax for type txid_snapshot: "1:9223372036854775808:3"
LINE 1: SELECT txid_snapshot '1:9223372036854775808:3';
^
+BEGIN;
+SELECT txid_current_ifassigned() IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_current() \gset
+SELECT txid_current_ifassigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
+ ?column?
+----------
+ t
+(1 row)
+
+COMMIT;
-- test xid status functions
BEGIN;
SELECT txid_current() AS committed \gset
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 50d4461..e831323 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -53,6 +53,12 @@ select txid_visible_in_snapshot('1000100010001015', '1000100010001000:1000100010
SELECT txid_snapshot '1:9223372036854775807:3';
SELECT txid_snapshot '1:9223372036854775808:3';
+BEGIN;
+SELECT txid_current_ifassigned() IS NULL;
+SELECT txid_current() \gset
+SELECT txid_current_ifassigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
+COMMIT;
+
-- test xid status functions
BEGIN;
SELECT txid_current() AS committed \gset
--
2.5.5
0003-Add-txid_convert_ifrecent-to-get-the-32-bit-xid-from.patchtext/x-patch; charset=US-ASCII; name=0003-Add-txid_convert_ifrecent-to-get-the-32-bit-xid-from.patchDownload
From 7f1c0479a972956fadb6a729c0de34bada4a59f7 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 3/4] Add txid_convert_ifrecent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
7 files changed, 183 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 14cb067..d3e44dc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -16970,6 +16970,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_ifrecent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - committed, aborted, in-progress, or null if the xid is too old</entry>
@@ -16982,9 +16987,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_ifrecent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 9c3c8da..5a145c2 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -708,6 +708,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_ifrecent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Underlying implementation of txid_status, which is mapped to an enum in
* system_views.sql.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 8db47d2..ee8ff1c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4905,6 +4905,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3344 ( txid_convert_ifrecent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_ifrecent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
DESCR("commit status of transaction");
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..1f54482 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_ifrecent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_ifrecent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index df09f62..f65a8d1 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -262,6 +262,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_ifrecent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_ifrecent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_ifrecent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_ifrecent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_ifrecent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_ifrecent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_ifrecent(BIGINT '1' << 32);
+ txid_convert_ifrecent
+-----------------------
+ 0
+(1 row)
+
+SELECT txid_convert_ifrecent((BIGINT '1' << 32) + 1);
+ txid_convert_ifrecent
+-----------------------
+ 1
+(1 row)
+
+SELECT txid_convert_ifrecent((BIGINT '1' << 32) + 2);
+ txid_convert_ifrecent
+-----------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -281,6 +338,49 @@ SELECT txid_status(:inprogress) AS inprogress;
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_ifrecent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_ifrecent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_ifrecent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..a646207 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_ifrecent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_ifrecent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index e831323..638ef30 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -71,12 +71,63 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_ifrecent(:committed) = :'committed'::xid;
+SELECT txid_convert_ifrecent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_ifrecent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_ifrecent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_ifrecent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_ifrecent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_ifrecent(BIGINT '1' << 32);
+SELECT txid_convert_ifrecent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_ifrecent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_ifrecent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_ifrecent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_ifrecent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
0004-Add-txid_incinerate-test-function-for-fast-wrap-arou.patchtext/x-patch; charset=US-ASCII; name=0004-Add-txid_incinerate-test-function-for-fast-wrap-arou.patchDownload
From fe0355e275e699c61345cf0bff80387c44594e03 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:57:54 +0800
Subject: [PATCH 4/4] Add txid_incinerate() test function for fast wrap-around
Burn txids, really, really, really, really fast. Especially if fsync
is off.
Good for reaching wraparound conditions quickly.
---
src/backend/utils/adt/txid.c | 170 +++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/wraparound.out | 161 +++++++++++++++++++++++++++++
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/wraparound.sql | 118 +++++++++++++++++++++
6 files changed, 455 insertions(+)
create mode 100644 src/test/regress/expected/wraparound.out
create mode 100644 src/test/regress/sql/wraparound.sql
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 5a145c2..990e7d5 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -752,3 +752,173 @@ txid_status_internal(PG_FUNCTION_ARGS)
ereport(ERROR,
(errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));
}
+
+/*
+ * Internal function for test only use to burn transaction IDs
+ * as fast as possible.
+ *
+ * Forcibly advance to just before wraparound.
+ *
+ * This will cause commit/rollback of our own xact to fail because
+ * the clog page has been truncated away.
+ *
+ * No safety check is performed to ensure nothing else has an xid.
+ * They'll fail on commit. Should really lock procarray.
+ *
+ * There's also no attempt to keep datfrozenxid correct for the other
+ * DBs. The user gets the fun of freezing them.
+ */
+Datum
+txid_incinerate(PG_FUNCTION_ARGS)
+{
+ const char *target;
+ int nreserved;
+ int i;
+
+ TransactionId lastAllocatedXid;
+ TransactionId clogPageFirstXid;
+ TransactionId targetNextXid;
+
+ if (!superuser())
+ elog(ERROR, "txid_incinerate may only be called by the superuser");
+
+ if (PG_ARGISNULL(0))
+ elog(ERROR, "xid argument must be non-null");
+
+ if (PG_ARGISNULL(1))
+ {
+ nreserved = 0;
+ }
+ else
+ {
+ nreserved = PG_GETARG_INT32(1);
+ if (nreserved < 0)
+ elog(ERROR, "nreserved xids must be >= 0");
+ }
+
+ target = text_to_cstring(PG_GETARG_TEXT_PP(0));
+
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (GetTopTransactionIdIfAny() != InvalidTransactionId)
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errmsg_internal("can't burn XIDs in a session with an xid allocated")));
+ }
+
+ lastAllocatedXid = ShmemVariableCache->nextXid;
+ TransactionIdRetreat(lastAllocatedXid);
+
+ if (strcmp(target, "stop"))
+ targetNextXid = ShmemVariableCache->xidStopLimit;
+ else if (strcmp(target, "warn"))
+ targetNextXid = ShmemVariableCache->xidWarnLimit;
+ else if (strcmp(target, "vac"))
+ targetNextXid = ShmemVariableCache->xidVacLimit;
+ else if (strcmp(target, "wrap"))
+ targetNextXid = ShmemVariableCache->xidWrapLimit;
+ else if (strcmp(target, "page"))
+ targetNextXid = ShmemVariableCache->nextXid + CLOG_XACTS_PER_PAGE - nreserved;
+ else
+ {
+ unsigned long parsed;
+ char *endp;
+ parsed = strtol(target, &endp, 10);
+ if (*endp != '\0')
+ elog(ERROR, "Argument must be an xid or one of the strings page, stop, warn, vac or wrap");
+ if (!TransactionIdIsNormal((TransactionId)parsed))
+ elog(ERROR, "Argument xid must be a normal xid, not the invalid/frozen/bootstrap xid");
+ targetNextXid = (TransactionId)parsed;
+ }
+
+ for (i = 0; i < nreserved; i++)
+ TransactionIdRetreat(targetNextXid);
+
+ if (!TransactionIdFollowsOrEquals(targetNextXid, ShmemVariableCache->nextXid))
+ elog(ERROR, "Target xid %u is <= current xid %u in modulo 32",
+ targetNextXid, ShmemVariableCache->nextXid);
+
+ elog(NOTICE, "xid limits are: vac=%u, warn=%u, stop=%u, wrap=%u, oldest=%u, next=%u; target xid is %u",
+ ShmemVariableCache->xidVacLimit,
+ ShmemVariableCache->xidWarnLimit,
+ ShmemVariableCache->xidStopLimit,
+ ShmemVariableCache->xidWrapLimit,
+ ShmemVariableCache->nextXid,
+ ShmemVariableCache->oldestXid,
+ targetNextXid);
+
+ Assert(TransactionIdPrecedes(ShmemVariableCache->nextXid, ShmemVariableCache->xidStopLimit));
+ Assert(TransactionIdPrecedesOrEquals(ShmemVariableCache->nextXid, ShmemVariableCache->xidWrapLimit));
+
+ /* Advance nextXid to the last xid on the current clog page */
+ clogPageFirstXid = ShmemVariableCache->nextXid - TransactionIdToPgIndex(ShmemVariableCache->nextXid);
+ ShmemVariableCache->nextXid = clogPageFirstXid + (CLOG_XACTS_PER_PAGE - 1);
+ elog(DEBUG1, "txid_incinerate: Advanced xid to %u, first %u on page %u",
+ ShmemVariableCache->nextXid, clogPageFirstXid,
+ TransactionIdToPage(ShmemVariableCache->nextXid));
+
+ /*
+ * Write new clog pages and advance to the end of the next page, until
+ * we've allocated the last clog page. This might take a while.
+ *
+ * At each step, force the next xid forward and extend the clog. We must
+ * allocate the first xid on the last page so that ExtendCLOG actually does
+ * some work, since otherwise it just shortcuts out.
+ */
+ do
+ {
+ if (clogPageFirstXid == FirstNormalTransactionId)
+ clogPageFirstXid = CLOG_XACTS_PER_PAGE;
+ else
+ clogPageFirstXid += CLOG_XACTS_PER_PAGE;
+
+ elog(DEBUG1, "txid_incinerate: nextXid %u", ShmemVariableCache->nextXid);
+
+ if (TransactionIdPrecedes(clogPageFirstXid, targetNextXid)
+ && TransactionIdPrecedesOrEquals(targetNextXid, clogPageFirstXid + (CLOG_XACTS_PER_PAGE - 1)))
+ {
+ ShmemVariableCache->nextXid = targetNextXid;
+ elog(DEBUG1, "txid_incinerate: reached target xid, next page %u greater than target %u",
+ targetNextXid, targetNextXid + CLOG_XACTS_PER_PAGE);
+ }
+ else
+ {
+ ShmemVariableCache->nextXid = clogPageFirstXid + (CLOG_XACTS_PER_PAGE - 1);
+ }
+
+ if (clogPageFirstXid < FirstNormalTransactionId)
+ {
+ clogPageFirstXid = FirstNormalTransactionId;
+ }
+
+ Assert(TransactionIdToPgIndex(clogPageFirstXid) == 0 || clogPageFirstXid == FirstNormalTransactionId);
+
+ Assert(TransactionIdPrecedesOrEquals(ShmemVariableCache->nextXid, ShmemVariableCache->xidWrapLimit));
+
+ ExtendCLOG(clogPageFirstXid);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ while (TransactionIdToPage(ShmemVariableCache->nextXid) != (targetNextXid/CLOG_XACTS_PER_PAGE));
+
+ elog(DEBUG1, "txid_incinerate: done extending clog and advancing counter, nextXid is %u",
+ ShmemVariableCache->nextXid);
+
+ Assert(TransactionIdPrecedesOrEquals(ShmemVariableCache->nextXid, ShmemVariableCache->xidWrapLimit));
+
+ /*
+ * We'd really like to totally reset the clog by truncating it and
+ * moving the wraparound pointer, but we can't do that unless all DBs
+ * are already frozen.
+ *
+ * We can't freeze here since we can't access other DBs. So we've got
+ * to let the user do the job.
+ */
+
+ elog(NOTICE, "txid_incinerate: advanced nextXid to %u",
+ ShmemVariableCache->nextXid);
+
+ LWLockRelease(XidGenLock);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index ee8ff1c..289fd57 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4909,6 +4909,8 @@ DATA(insert OID = 3344 ( txid_convert_ifrecent PGNSP PGUID 12 1 0 0 0 f f f f
DESCR("get the xid from a bigint transaction id if not wrapped around");
DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3347 ( txid_incinerate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "25 23" _null_ _null_ _null_ _null_ _null_ txid_incinerate _null_ _null_ _null_ ));
+DESCR("burn xids fast");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/wraparound.out b/src/test/regress/expected/wraparound.out
new file mode 100644
index 0000000..99644aa
--- /dev/null
+++ b/src/test/regress/expected/wraparound.out
@@ -0,0 +1,161 @@
+-- We need to be able to force vacuuming of template0
+UPDATE pg_database
+SET datallowconn = true
+WHERE datname = 'template0';
+-- For debugging these tests you'll find something like this useful:
+-- SELECT txid_current() \gset
+-- SELECT BIGINT :'txid_current' AS txid,
+-- BIGINT :'txid_current' >> 32 AS epoch,
+-- BIGINT :'txid_current' & 4294967295 AS xid32;
+SELECT txid_current() AS before_wrap_xid \gset
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+ txid_incinerate
+-----------------
+
+(1 row)
+
+-- Should be near UINT32_MAX/2 now
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+ txid_current_epoch
+--------------------
+ 0
+(1 row)
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+CHECKPOINT;
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+-- That got us to nearly UINT32_MAX/2, another run will get us to near UINT32_MAX
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+ txid_incinerate
+-----------------
+
+(1 row)
+
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 32);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+ txid_current_epoch
+--------------------
+ 0
+(1 row)
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+CHECKPOINT;
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 32);
+ ?column?
+----------
+ t
+(1 row)
+
+-- We should be near UINT32_MAX now, so the next run will
+-- bring us across the epoch boundary.
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+ txid_incinerate
+-----------------
+
+(1 row)
+
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 32);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+ txid_current_epoch
+--------------------
+ 1
+(1 row)
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+CHECKPOINT;
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+UPDATE pg_database
+SET datallowconn = false
+WHERE datname = 'template0';
+-- Make sure our txid functions handle the epoch wrap
+SELECT txid_convert_ifrecent(BIGINT :'before_wrap_xid');
+ txid_convert_ifrecent
+-----------------------
+
+(1 row)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4ebad04..5ca175d 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -42,6 +42,9 @@ test: create_type
test: create_table
test: create_function_2
+# Force txid wraparound
+test: wraparound
+
# ----------
# Load huge amounts of data
# We should split the data files into single files and then
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 5c7038d..430b6eb 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -55,6 +55,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: wraparound
test: copy
test: copyselect
test: copydml
diff --git a/src/test/regress/sql/wraparound.sql b/src/test/regress/sql/wraparound.sql
new file mode 100644
index 0000000..e849ef5
--- /dev/null
+++ b/src/test/regress/sql/wraparound.sql
@@ -0,0 +1,118 @@
+-- We need to be able to force vacuuming of template0
+UPDATE pg_database
+SET datallowconn = true
+WHERE datname = 'template0';
+
+-- For debugging these tests you'll find something like this useful:
+-- SELECT txid_current() \gset
+-- SELECT BIGINT :'txid_current' AS txid,
+-- BIGINT :'txid_current' >> 32 AS epoch,
+-- BIGINT :'txid_current' & 4294967295 AS xid32;
+
+SELECT txid_current() AS before_wrap_xid \gset
+
+
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+
+-- Should be near UINT32_MAX/2 now
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+CHECKPOINT;
+
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+
+
+
+
+
+
+-- That got us to nearly UINT32_MAX/2, another run will get us to near UINT32_MAX
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 32);
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+CHECKPOINT;
+
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 32);
+
+
+
+
+-- We should be near UINT32_MAX now, so the next run will
+-- bring us across the epoch boundary.
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 32);
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+CHECKPOINT;
+
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+
+UPDATE pg_database
+SET datallowconn = false
+WHERE datname = 'template0';
+
+
+
+-- Make sure our txid functions handle the epoch wrap
+SELECT txid_convert_ifrecent(BIGINT :'before_wrap_xid');
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Sat, Aug 20, 2016 at 9:46 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
Ahem. Forgot to squash in a fixup commit. Updated patch of
txid_status(bigint) attachd.A related patch follows, adding a new txid_current_ifassigned(bigint)
function as suggested by Jim Nasby. It's usefully connected to txid_status()
and might as well be added at the same time.Since it builds on the same history I've also attached an updated version of
txid_recent(bigint) now called txid_convert_ifrecent(bigint), per the
above-linked thread.Finally, and not intended for commit, is a useful test function I wrote to
cause extremely rapid xid wraparound, bundled up into a src/test/regress
test case. txid_incinerate() can jump the server about UINT32/2 xids in ~2
seconds if fsync is off, making it handy for testing. Posting so others can
use it for their own test needs later and because it's useful for testing
these patches that touch on the xid epoch.
I think you should use underscores to separate all of the words
instead of only some of them.
Also, note that the corresponding internal function is
GetTopTransactionIdIfAny(), which might suggest txid_current_if_any()
rather than txid_current_if_assigned(), but you could argue that your
naming is better...
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 August 2016 at 01:03, Robert Haas <robertmhaas@gmail.com> wrote:
I think you should use underscores to separate all of the words
instead of only some of them.
Right. Will fix.
Thanks for taking a look.
Also, note that the corresponding internal function is
GetTopTransactionIdIfAny(), which might suggest txid_current_if_any()
rather than txid_current_if_assigned(), but you could argue that your
naming is better.
Yeah, I do argue that in this case. Not a hugely strong opinion, but we
refer to txid_current() in the docs as:
"get current transaction ID, assigning a new one if the current transaction
does not have one"
so I thought it'd be worth being consistent with that. It's not like
txid_current() mirrors the name of GetTopTransactionId() after all ;) and I
think it's more important to be consistent with what the user sees than
with the code.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 August 2016 at 01:03, Robert Haas <robertmhaas@gmail.com> wrote:
I think you should use underscores to separate all of the words
instead of only some of them.
ifassigned => if_assigned
ifrecent=> if_recent
Updated patch series attached. As before, 0-4 intended for commit, 5 just
because it'll be handy to have around for people doing wraparound related
testing.
Again, thanks for taking a look.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 81cbe525261a15a21415af361b3421038eccc895 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/4] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 28 +++++++++++-
src/backend/access/transam/clog.c | 23 ----------
src/backend/catalog/system_views.sql | 20 +++++++++
src/backend/utils/adt/txid.c | 82 ++++++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 ++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/txid.out | 50 ++++++++++++++++++++++
src/test/regress/sql/txid.sql | 35 +++++++++++++++
8 files changed, 239 insertions(+), 24 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 169a385..8edf490 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17139,6 +17139,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17157,7 +17161,7 @@ SELECT collation for ('foo' COLLATE "de_DE");
<row>
<entry><literal><function>txid_current()</function></literal></entry>
<entry><type>bigint</type></entry>
- <entry>get current transaction ID, assigning a new one if the current transaction does not have one</entry>
+ <entry>get current 64-bit transaction ID with epoch, assigning a new one if the current transaction does not have one</entry>
</row>
<row>
<entry><literal><function>txid_current_snapshot()</function></literal></entry>
@@ -17184,6 +17188,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - committed, aborted, in-progress, or null if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17254,6 +17263,23 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Any recent transaction can be identified as one of
+ <itemizedlist>
+ <listitem><para>in-progress</></>
+ <listitem><para>committed</></>
+ <listitem><para>aborted</></>
+ </itemizedlist>
+ Prepared transactions are identified as <literal>in-progress</>.
+ The commit status of transactions older than the transaction ID wrap-around
+ threshold is no longer known by the system, so <function>txid_status</>
+ returns <literal>NULL</> for such transactions. Applications may use
+ <function>txid_status</> to determine whether a transaction committed
+ or aborted when the application and/or database server crashed or lost
+ connection while a <literal>COMMIT</literal> command was in-progress.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index ada2142..e173da2 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1006,6 +1006,26 @@ LANGUAGE INTERNAL
STRICT IMMUTABLE PARALLEL SAFE
AS 'jsonb_insert';
+CREATE TYPE txid_status AS ENUM ('committed', 'in-progress', 'aborted');
+
+CREATE FUNCTION
+ txid_status(txid bigint)
+RETURNS txid_status
+LANGUAGE sql
+VOLATILE PARALLEL SAFE
+AS $$
+SELECT CASE
+ WHEN s IS NULL THEN NULL::txid_status
+ WHEN s = -1 THEN 'aborted'::txid_status
+ WHEN s = 0 THEN 'in-progress'::txid_status
+ WHEN s = 1 THEN 'committed'::txid_status
+END
+FROM pg_catalog.txid_status_internal($1) s;
+$$;
+
+COMMENT ON FUNCTION txid_status(bigint)
+IS 'get commit status of given recent xid or null if too old';
+
-- The default permissions for functions mean that anyone can execute them.
-- A number of functions shouldn't be executable by just anyone, but rather
-- than use explicit 'superuser()' checks in those functions, we use the GRANT
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index c2069a9..0480c99 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,9 +21,11 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
+#include "catalog/pg_proc.h"
#include "funcapi.h"
#include "miscadmin.h"
#include "libpq/pqformat.h"
@@ -117,6 +119,50 @@ convert_xid(TransactionId xid, const TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound detection.
+ * ERRORs if the txid is in the future. Returns permanent XIDs unchanged.
+ * Otherwise returns the 32-bit xid and sets the wraparound param to true
+ * if wraparound is detected, false otherwise.
+ */
+static TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *wraparound)
+{
+ uint32 xid_epoch = (uint32)(xid_with_epoch >>32);
+ TransactionId xid = (TransactionId)(xid_with_epoch);
+ TxidEpoch now_epoch;
+
+ load_xid_epoch(&now_epoch);
+
+ *wraparound = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch && xid < now_epoch.last_xid))
+ {
+ /* xid too far in the past */
+ *wraparound = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
+
+/*
* txid comparator for qsort/bsearch
*/
static int
@@ -354,6 +400,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -637,3 +686,36 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Underlying implementation of txid_status, which is mapped to an enum in
+ * system_views.sql.
+ */
+Datum
+txid_status_internal(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid = get_xid_in_recent_past(xid_with_epoch, &wraparound);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else if (TransactionIdIsCurrentTransactionId(xid) || TransactionIdIsInProgress(xid))
+ PG_RETURN_INT32(0);
+ else if (TransactionIdDidCommit(xid))
+ PG_RETURN_INT32(1);
+ else if (TransactionIdDidAbort(xid))
+ PG_RETURN_INT32(-1);
+ else
+ /* shouldn't happen */
+ ereport(ERROR,
+ (errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 6fed7a0..d99384c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4914,6 +4914,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index ddd217e..61299c5 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -238,3 +238,53 @@ SELECT txid_snapshot '1:9223372036854775808:3';
ERROR: invalid input syntax for type txid_snapshot: "1:9223372036854775808:3"
LINE 1: SELECT txid_snapshot '1:9223372036854775808:3';
^
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in-progress
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index b6650b9..0bfdfe2 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -52,3 +52,38 @@ select txid_visible_in_snapshot('1000100010001015', '1000100010001000:1000100010
-- test 64bit overflow
SELECT txid_snapshot '1:9223372036854775807:3';
SELECT txid_snapshot '1:9223372036854775808:3';
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_current_ifassigned.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_current_ifassigned.patchDownload
From 8a4d038e354e98c1e5a5fd10ff8822a50ad1b4ae Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:46:53 +0800
Subject: [PATCH 2/4] Add txid_current_ifassigned()
Add a variant of txid_current() that returns null if no xid is assigned
instead of assigning one or, on a replica, ERRORing.
Per suggestion from Jim Nasby
---
doc/src/sgml/func.sgml | 9 +++++++++
src/backend/utils/adt/txid.c | 21 +++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 ++
src/test/regress/expected/txid.out | 15 +++++++++++++++
src/test/regress/sql/txid.sql | 6 ++++++
5 files changed, 53 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8edf490..9434e9b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17120,6 +17120,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
</indexterm>
<indexterm>
+ <primary>txid_current_if_assigned</primary>
+ </indexterm>
+
+ <indexterm>
<primary>txid_current_snapshot</primary>
</indexterm>
@@ -17164,6 +17168,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>get current 64-bit transaction ID with epoch, assigning a new one if the current transaction does not have one</entry>
</row>
<row>
+ <entry><literal><function>txid_current_if_assigned()</function></literal></entry>
+ <entry><type>bigint</type></entry>
+ <entry>same as <function>txid_current()</function> but returns null instead of assigning an xid if none is already assigned</entry>
+ </row>
+ <row>
<entry><literal><function>txid_current_snapshot()</function></literal></entry>
<entry><type>txid_snapshot</type></entry>
<entry>get current snapshot</entry>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 0480c99..05b0c96 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -426,6 +426,27 @@ txid_current(PG_FUNCTION_ARGS)
}
/*
+ * Same as txid_current() but doesn't assign a new xid if there isn't one
+ * yet.
+ */
+Datum
+txid_current_if_assigned(PG_FUNCTION_ARGS)
+{
+ txid val;
+ TxidEpoch state;
+ TransactionId topxid = GetTopTransactionIdIfAny();
+
+ if (topxid == InvalidTransactionId)
+ PG_RETURN_NULL();
+
+ load_xid_epoch(&state);
+
+ val = convert_xid(topxid, &state);
+
+ PG_RETURN_INT64(val);
+}
+
+/*
* txid_current_snapshot() returns txid_snapshot
*
* Return current snapshot in TXID format
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index d99384c..80420ef 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4904,6 +4904,8 @@ DATA(insert OID = 2942 ( txid_snapshot_send PGNSP PGUID 12 1 0 0 0 f f f f t
DESCR("I/O");
DATA(insert OID = 2943 ( txid_current PGNSP PGUID 12 1 0 0 0 f f f f t f s u 0 0 20 "" _null_ _null_ _null_ _null_ _null_ txid_current _null_ _null_ _null_ ));
DESCR("get current transaction ID");
+DATA(insert OID = 3348 ( txid_current_if_assigned PGNSP PGUID 12 1 0 0 0 f f f f t f s u 0 0 20 "" _null_ _null_ _null_ _null_ _null_ txid_current_if_assigned _null_ _null_ _null_ ));
+DESCR("get current transaction ID");
DATA(insert OID = 2944 ( txid_current_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f s s 0 0 2970 "" _null_ _null_ _null_ _null_ _null_ txid_current_snapshot _null_ _null_ _null_ ));
DESCR("get current snapshot");
DATA(insert OID = 2945 ( txid_snapshot_xmin PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 20 "2970" _null_ _null_ _null_ _null_ _null_ txid_snapshot_xmin _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 61299c5..9450d5e 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -238,6 +238,21 @@ SELECT txid_snapshot '1:9223372036854775808:3';
ERROR: invalid input syntax for type txid_snapshot: "1:9223372036854775808:3"
LINE 1: SELECT txid_snapshot '1:9223372036854775808:3';
^
+BEGIN;
+SELECT txid_current_if_assigned() IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_current() \gset
+SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
+ ?column?
+----------
+ t
+(1 row)
+
+COMMIT;
-- test xid status functions
BEGIN;
SELECT txid_current() AS committed \gset
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 0bfdfe2..70e18e3 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -53,6 +53,12 @@ select txid_visible_in_snapshot('1000100010001015', '1000100010001000:1000100010
SELECT txid_snapshot '1:9223372036854775807:3';
SELECT txid_snapshot '1:9223372036854775808:3';
+BEGIN;
+SELECT txid_current_if_assigned() IS NULL;
+SELECT txid_current() \gset
+SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
+COMMIT;
+
-- test xid status functions
BEGIN;
SELECT txid_current() AS committed \gset
--
2.5.5
0003-Add-txid_convert_ifrecent-to-get-the-32-bit-xid-from.patchtext/x-patch; charset=US-ASCII; name=0003-Add-txid_convert_ifrecent-to-get-the-32-bit-xid-from.patchDownload
From f6414c2fcb73800d7320b4673c3976bf67d93d4c Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 3/4] Add txid_convert_ifrecent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
7 files changed, 183 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9434e9b..8ddf652 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - committed, aborted, in-progress, or null if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 05b0c96..ce25093 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -708,6 +708,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Underlying implementation of txid_status, which is mapped to an enum in
* system_views.sql.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 80420ef..226bed7 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4916,6 +4916,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
DESCR("commit status of transaction");
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 9450d5e..ed038b4 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -262,6 +262,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -281,6 +338,49 @@ SELECT txid_status(:inprogress) AS inprogress;
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 70e18e3..53505bc 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -71,12 +71,63 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
0004-Add-txid_incinerate-test-function-for-fast-wrap-arou.patchtext/x-patch; charset=US-ASCII; name=0004-Add-txid_incinerate-test-function-for-fast-wrap-arou.patchDownload
From bd540abf3e31a4f5623449ba0669e61722bf7647 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:57:54 +0800
Subject: [PATCH 4/4] Add txid_incinerate() test function for fast wrap-around
Burn txids, really, really, really, really fast. Especially if fsync
is off.
Good for reaching wraparound conditions quickly.
---
src/backend/utils/adt/txid.c | 170 +++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/wraparound.out | 161 +++++++++++++++++++++++++++++
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/wraparound.sql | 118 +++++++++++++++++++++
6 files changed, 455 insertions(+)
create mode 100644 src/test/regress/expected/wraparound.out
create mode 100644 src/test/regress/sql/wraparound.sql
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index ce25093..1137ddb 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -752,3 +752,173 @@ txid_status_internal(PG_FUNCTION_ARGS)
ereport(ERROR,
(errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));
}
+
+/*
+ * Internal function for test only use to burn transaction IDs
+ * as fast as possible.
+ *
+ * Forcibly advance to just before wraparound.
+ *
+ * This will cause commit/rollback of our own xact to fail because
+ * the clog page has been truncated away.
+ *
+ * No safety check is performed to ensure nothing else has an xid.
+ * They'll fail on commit. Should really lock procarray.
+ *
+ * There's also no attempt to keep datfrozenxid correct for the other
+ * DBs. The user gets the fun of freezing them.
+ */
+Datum
+txid_incinerate(PG_FUNCTION_ARGS)
+{
+ const char *target;
+ int nreserved;
+ int i;
+
+ TransactionId lastAllocatedXid;
+ TransactionId clogPageFirstXid;
+ TransactionId targetNextXid;
+
+ if (!superuser())
+ elog(ERROR, "txid_incinerate may only be called by the superuser");
+
+ if (PG_ARGISNULL(0))
+ elog(ERROR, "xid argument must be non-null");
+
+ if (PG_ARGISNULL(1))
+ {
+ nreserved = 0;
+ }
+ else
+ {
+ nreserved = PG_GETARG_INT32(1);
+ if (nreserved < 0)
+ elog(ERROR, "nreserved xids must be >= 0");
+ }
+
+ target = text_to_cstring(PG_GETARG_TEXT_PP(0));
+
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (GetTopTransactionIdIfAny() != InvalidTransactionId)
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errmsg_internal("can't burn XIDs in a session with an xid allocated")));
+ }
+
+ lastAllocatedXid = ShmemVariableCache->nextXid;
+ TransactionIdRetreat(lastAllocatedXid);
+
+ if (strcmp(target, "stop"))
+ targetNextXid = ShmemVariableCache->xidStopLimit;
+ else if (strcmp(target, "warn"))
+ targetNextXid = ShmemVariableCache->xidWarnLimit;
+ else if (strcmp(target, "vac"))
+ targetNextXid = ShmemVariableCache->xidVacLimit;
+ else if (strcmp(target, "wrap"))
+ targetNextXid = ShmemVariableCache->xidWrapLimit;
+ else if (strcmp(target, "page"))
+ targetNextXid = ShmemVariableCache->nextXid + CLOG_XACTS_PER_PAGE - nreserved;
+ else
+ {
+ unsigned long parsed;
+ char *endp;
+ parsed = strtol(target, &endp, 10);
+ if (*endp != '\0')
+ elog(ERROR, "Argument must be an xid or one of the strings page, stop, warn, vac or wrap");
+ if (!TransactionIdIsNormal((TransactionId)parsed))
+ elog(ERROR, "Argument xid must be a normal xid, not the invalid/frozen/bootstrap xid");
+ targetNextXid = (TransactionId)parsed;
+ }
+
+ for (i = 0; i < nreserved; i++)
+ TransactionIdRetreat(targetNextXid);
+
+ if (!TransactionIdFollowsOrEquals(targetNextXid, ShmemVariableCache->nextXid))
+ elog(ERROR, "Target xid %u is <= current xid %u in modulo 32",
+ targetNextXid, ShmemVariableCache->nextXid);
+
+ elog(NOTICE, "xid limits are: vac=%u, warn=%u, stop=%u, wrap=%u, oldest=%u, next=%u; target xid is %u",
+ ShmemVariableCache->xidVacLimit,
+ ShmemVariableCache->xidWarnLimit,
+ ShmemVariableCache->xidStopLimit,
+ ShmemVariableCache->xidWrapLimit,
+ ShmemVariableCache->nextXid,
+ ShmemVariableCache->oldestXid,
+ targetNextXid);
+
+ Assert(TransactionIdPrecedes(ShmemVariableCache->nextXid, ShmemVariableCache->xidStopLimit));
+ Assert(TransactionIdPrecedesOrEquals(ShmemVariableCache->nextXid, ShmemVariableCache->xidWrapLimit));
+
+ /* Advance nextXid to the last xid on the current clog page */
+ clogPageFirstXid = ShmemVariableCache->nextXid - TransactionIdToPgIndex(ShmemVariableCache->nextXid);
+ ShmemVariableCache->nextXid = clogPageFirstXid + (CLOG_XACTS_PER_PAGE - 1);
+ elog(DEBUG1, "txid_incinerate: Advanced xid to %u, first %u on page %u",
+ ShmemVariableCache->nextXid, clogPageFirstXid,
+ TransactionIdToPage(ShmemVariableCache->nextXid));
+
+ /*
+ * Write new clog pages and advance to the end of the next page, until
+ * we've allocated the last clog page. This might take a while.
+ *
+ * At each step, force the next xid forward and extend the clog. We must
+ * allocate the first xid on the last page so that ExtendCLOG actually does
+ * some work, since otherwise it just shortcuts out.
+ */
+ do
+ {
+ if (clogPageFirstXid == FirstNormalTransactionId)
+ clogPageFirstXid = CLOG_XACTS_PER_PAGE;
+ else
+ clogPageFirstXid += CLOG_XACTS_PER_PAGE;
+
+ elog(DEBUG1, "txid_incinerate: nextXid %u", ShmemVariableCache->nextXid);
+
+ if (TransactionIdPrecedes(clogPageFirstXid, targetNextXid)
+ && TransactionIdPrecedesOrEquals(targetNextXid, clogPageFirstXid + (CLOG_XACTS_PER_PAGE - 1)))
+ {
+ ShmemVariableCache->nextXid = targetNextXid;
+ elog(DEBUG1, "txid_incinerate: reached target xid, next page %u greater than target %u",
+ targetNextXid, targetNextXid + CLOG_XACTS_PER_PAGE);
+ }
+ else
+ {
+ ShmemVariableCache->nextXid = clogPageFirstXid + (CLOG_XACTS_PER_PAGE - 1);
+ }
+
+ if (clogPageFirstXid < FirstNormalTransactionId)
+ {
+ clogPageFirstXid = FirstNormalTransactionId;
+ }
+
+ Assert(TransactionIdToPgIndex(clogPageFirstXid) == 0 || clogPageFirstXid == FirstNormalTransactionId);
+
+ Assert(TransactionIdPrecedesOrEquals(ShmemVariableCache->nextXid, ShmemVariableCache->xidWrapLimit));
+
+ ExtendCLOG(clogPageFirstXid);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ while (TransactionIdToPage(ShmemVariableCache->nextXid) != (targetNextXid/CLOG_XACTS_PER_PAGE));
+
+ elog(DEBUG1, "txid_incinerate: done extending clog and advancing counter, nextXid is %u",
+ ShmemVariableCache->nextXid);
+
+ Assert(TransactionIdPrecedesOrEquals(ShmemVariableCache->nextXid, ShmemVariableCache->xidWrapLimit));
+
+ /*
+ * We'd really like to totally reset the clog by truncating it and
+ * moving the wraparound pointer, but we can't do that unless all DBs
+ * are already frozen.
+ *
+ * We can't freeze here since we can't access other DBs. So we've got
+ * to let the user do the job.
+ */
+
+ elog(NOTICE, "txid_incinerate: advanced nextXid to %u",
+ ShmemVariableCache->nextXid);
+
+ LWLockRelease(XidGenLock);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 226bed7..fde95bf 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4920,6 +4920,8 @@ DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f
DESCR("get the xid from a bigint transaction id if not wrapped around");
DATA(insert OID = 3346 ( txid_status_internal PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 23 "20" _null_ _null_ _null_ _null_ _null_ txid_status_internal _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3347 ( txid_incinerate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "25 23" _null_ _null_ _null_ _null_ _null_ txid_incinerate _null_ _null_ _null_ ));
+DESCR("burn xids fast");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/wraparound.out b/src/test/regress/expected/wraparound.out
new file mode 100644
index 0000000..5638dd6
--- /dev/null
+++ b/src/test/regress/expected/wraparound.out
@@ -0,0 +1,161 @@
+-- We need to be able to force vacuuming of template0
+UPDATE pg_database
+SET datallowconn = true
+WHERE datname = 'template0';
+-- For debugging these tests you'll find something like this useful:
+-- SELECT txid_current() \gset
+-- SELECT BIGINT :'txid_current' AS txid,
+-- BIGINT :'txid_current' >> 32 AS epoch,
+-- BIGINT :'txid_current' & 4294967295 AS xid32;
+SELECT txid_current() AS before_wrap_xid \gset
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+ txid_incinerate
+-----------------
+
+(1 row)
+
+-- Should be near UINT32_MAX/2 now
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+ txid_current_epoch
+--------------------
+ 0
+(1 row)
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+CHECKPOINT;
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+-- That got us to nearly UINT32_MAX/2, another run will get us to near UINT32_MAX
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+ txid_incinerate
+-----------------
+
+(1 row)
+
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 32);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+ txid_current_epoch
+--------------------
+ 0
+(1 row)
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+CHECKPOINT;
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 32);
+ ?column?
+----------
+ t
+(1 row)
+
+-- We should be near UINT32_MAX now, so the next run will
+-- bring us across the epoch boundary.
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+ txid_incinerate
+-----------------
+
+(1 row)
+
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 32);
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+ txid_current_epoch
+--------------------
+ 1
+(1 row)
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+CHECKPOINT;
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+UPDATE pg_database
+SET datallowconn = false
+WHERE datname = 'template0';
+-- Make sure our txid functions handle the epoch wrap
+SELECT txid_convert_if_recent(BIGINT :'before_wrap_xid');
+ txid_convert_if_recent
+------------------------
+
+(1 row)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 1cb5dfc..e5c34d0 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -42,6 +42,9 @@ test: create_type
test: create_table
test: create_function_2
+# Force txid wraparound
+test: wraparound
+
# ----------
# Load huge amounts of data
# We should split the data files into single files and then
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 8958d8c..1fa4852 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -55,6 +55,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: wraparound
test: copy
test: copyselect
test: copydml
diff --git a/src/test/regress/sql/wraparound.sql b/src/test/regress/sql/wraparound.sql
new file mode 100644
index 0000000..f67c435
--- /dev/null
+++ b/src/test/regress/sql/wraparound.sql
@@ -0,0 +1,118 @@
+-- We need to be able to force vacuuming of template0
+UPDATE pg_database
+SET datallowconn = true
+WHERE datname = 'template0';
+
+-- For debugging these tests you'll find something like this useful:
+-- SELECT txid_current() \gset
+-- SELECT BIGINT :'txid_current' AS txid,
+-- BIGINT :'txid_current' >> 32 AS epoch,
+-- BIGINT :'txid_current' & 4294967295 AS xid32;
+
+SELECT txid_current() AS before_wrap_xid \gset
+
+
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+
+-- Should be near UINT32_MAX/2 now
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+CHECKPOINT;
+
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) - (BIGINT '1' << 30);
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 31) + (BIGINT '1' << 30);
+
+
+
+
+
+
+-- That got us to nearly UINT32_MAX/2, another run will get us to near UINT32_MAX
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+SELECT BIGINT :'txid_current' < (BIGINT '1' << 32);
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+CHECKPOINT;
+
+select min(datfrozenxid::text::bigint) AS min_datfrozenxid from pg_database \gset
+SELECT BIGINT :'min_datfrozenxid' > (BIGINT '1' << 31) + (BIGINT '1' << 30);
+SELECT BIGINT :'min_datfrozenxid' < (BIGINT '1' << 32);
+
+
+
+
+-- We should be near UINT32_MAX now, so the next run will
+-- bring us across the epoch boundary.
+SET client_min_messages = 'error';
+SELECT txid_incinerate('stop', 1000);
+SELECT txid_current() \gset
+SELECT BIGINT :'txid_current' > (BIGINT '1' << 32);
+SELECT BIGINT :'txid_current' >> 32 AS txid_current_epoch;
+
+\c template0
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c template1
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+\c postgres
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+CHECKPOINT;
+
+\c regression
+SET client_min_messages = 'error';
+VACUUM FREEZE;
+
+
+UPDATE pg_database
+SET datallowconn = false
+WHERE datname = 'template0';
+
+
+
+-- Make sure our txid functions handle the epoch wrap
+SELECT txid_convert_if_recent(BIGINT :'before_wrap_xid');
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23/08/16 02:55, Craig Ringer wrote:
On 23 August 2016 at 01:03, Robert Haas <robertmhaas@gmail.com
<mailto:robertmhaas@gmail.com>> wrote:I think you should use underscores to separate all of the words
instead of only some of them.ifassigned => if_assigned
ifrecent=> if_recent
Updated patch series attached. As before, 0-4 intended for commit, 5
just because it'll be handy to have around for people doing wraparound
related testing.
I guess you mean 0-3 for commit and 4 is just handy?
From the point of code this patch seems good to me.
I do wonder about the 3rd patch though. I wonder if it would not be
better to have the opposite function instead - converting xid to txid as
that will always work and does not have to have the NULL case and would
be simpler in terms of code.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 Aug 2016 16:02, "Petr Jelinek" <petr@2ndquadrant.com> wrote:
On 23/08/16 02:55, Craig Ringer wrote:
On 23 August 2016 at 01:03, Robert Haas <robertmhaas@gmail.com
<mailto:robertmhaas@gmail.com>> wrote:I think you should use underscores to separate all of the words
instead of only some of them.ifassigned => if_assigned
ifrecent=> if_recent
Updated patch series attached. As before, 0-4 intended for commit, 5
just because it'll be handy to have around for people doing wraparound
related testing.I guess you mean 0-3 for commit and 4 is just handy?
Er. Right. 1-3. 4 just as handy test/tool.
1 most important and useful. Then 2. Then 3.
From the point of code this patch seems good to me.
Thanks.
I do wonder about the 3rd patch though. I wonder if it would not be
better to have the opposite function instead - converting xid to txid as
that will always work and does not have to have the NULL case and would be
simpler in terms of code.
Yeah, but it wouldn't solve the need to take txid_current() output and do
stuff with it other than ordinal comparison. Like pass to commit ts
functions and others that take xid. If we extend all funcs that take xid to
take bigint instead, they just get to use the same epoch logic in them,
complete with some way to deal with wrapped xids sensibly. It has to be
done somewhere. Though it's prettier if hidden from the user.
More importantly imo, txid => bigint has to assume the current epoch. We
have no way to make sure the user doesn't try to use something already
wrapped.
I don't mind if everyone decides it's better to make xid go away and use
bigint everywhere user facing. Or even a new bigxid type. More work than I
can really afford but can manage; shouldn't block #1 and #2 though as they
already use bigint.
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23/08/16 11:27, Craig Ringer wrote:
On 23 Aug 2016 16:02, "Petr Jelinek" <petr@2ndquadrant.com
<mailto:petr@2ndquadrant.com>> wrote:On 23/08/16 02:55, Craig Ringer wrote:
On 23 August 2016 at 01:03, Robert Haas <robertmhaas@gmail.com
<mailto:robertmhaas@gmail.com>
<mailto:robertmhaas@gmail.com <mailto:robertmhaas@gmail.com>>> wrote:
I think you should use underscores to separate all of the words
instead of only some of them.ifassigned => if_assigned
ifrecent=> if_recent
Updated patch series attached. As before, 0-4 intended for commit, 5
just because it'll be handy to have around for people doing wraparound
related testing.I guess you mean 0-3 for commit and 4 is just handy?
Er. Right. 1-3. 4 just as handy test/tool.
1 most important and useful. Then 2. Then 3.
From the point of code this patch seems good to me.
Thanks.
I do wonder about the 3rd patch though. I wonder if it would not be
better to have the opposite function instead - converting xid to txid as
that will always work and does not have to have the NULL case and would
be simpler in terms of code.Yeah, but it wouldn't solve the need to take txid_current() output and
do stuff with it other than ordinal comparison. Like pass to commit ts
functions and others that take xid. If we extend all funcs that take xid
to take bigint instead, they just get to use the same epoch logic in
them, complete with some way to deal with wrapped xids sensibly. It has
to be done somewhere. Though it's prettier if hidden from the user.More importantly imo, txid => bigint has to assume the current epoch. We
have no way to make sure the user doesn't try to use something already
wrapped.
Okay, fair points.
I don't mind if everyone decides it's better to make xid go away and use
bigint everywhere user facing. Or even a new bigxid type. More work than
I can really afford but can manage; shouldn't block #1 and #2 though as
they already use bigint.
I don't think that would be very straightforward to be honest. I guess
for what you want to achieve the approach chosen is the best one then.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Mon, Aug 22, 2016 at 8:55 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Updated patch series attached. As before, 0-4 intended for commit, 5 just
because it'll be handy to have around for people doing wraparound related
testing.Again, thanks for taking a look.
/me reviews a bit more deeply.
In 0001, it seems to me that "in-progress" should be "in progress". I
don't think it's normal to hyphenate that. We have admittedly
sometimes done so, but:
[rhaas pgsql]$ git grep 'in-progress' | wc -l
63
[rhaas pgsql]$ git grep 'in progress' | wc -l
346
It may make sense to speak of an "in-progress transaction" but I would
say "the transaction is in progress" not "the transaction is
in-progress", which seems to me to argue for a space as the proper
separator here.
Also:
+CREATE TYPE txid_status AS ENUM ('committed', 'in-progress', 'aborted');
+
+CREATE FUNCTION
+ txid_status(txid bigint)
+RETURNS txid_status
+LANGUAGE sql
+VOLATILE PARALLEL SAFE
+AS $$
+SELECT CASE
+ WHEN s IS NULL THEN NULL::txid_status
+ WHEN s = -1 THEN 'aborted'::txid_status
+ WHEN s = 0 THEN 'in-progress'::txid_status
+ WHEN s = 1 THEN 'committed'::txid_status
+END
+FROM pg_catalog.txid_status_internal($1) s;
+$$;
+
+COMMENT ON FUNCTION txid_status(bigint)
+IS 'get commit status of given recent xid or null if too old';I'm not really that keen on this approach. I don't think we need to
introduce a new data type for this, and I would rather not use SQL,
either. It would be faster and simpler to just return the appropriate
string from a C function defined directly.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Tue, Aug 23, 2016 at 10:18 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 22, 2016 at 8:55 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Updated patch series attached. As before, 0-4 intended for commit, 5 just
because it'll be handy to have around for people doing wraparound related
testing.Again, thanks for taking a look.
/me reviews a bit more deeply.
In 0001, ...
0002 looks good, so I committed it. You forgot a function prototype
for the new SQL-callable function, though, so I added that. For me,
it generates a compiler warning if that's missing; you might want to
try to achieve a similar setup.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 August 2016 at 22:18, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 22, 2016 at 8:55 PM, Craig Ringer <craig@2ndquadrant.com>
wrote:Updated patch series attached. As before, 0-4 intended for commit, 5 just
because it'll be handy to have around for people doing wraparound related
testing.Again, thanks for taking a look.
/me reviews a bit more deeply.
In 0001, it seems to me that "in-progress" should be "in progress".
Fine by me. I was on the fence about it anyway.
+CREATE TYPE txid_status AS ENUM ('committed', 'in-progress', 'aborted');
I'm not really that keen on this approach. I don't think we need to
introduce a new data type for this, and I would rather not use SQL,
either. It would be faster and simpler to just return the appropriate
string from a C function defined directly.
Also fine by me. You're right, keep it simple. It means the potential set
of values isn't discoverable the same way, but ... meh. Using it usefully
means reading the docs anyway.
The remaining 2 patches of interest are attached - txid_status() and
txid_convert_if_recent(). Thanks for committing txid_current_if_assigned().
Now I'd best stop pretending I'm in a sensible timezone.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-txid_status-v3.patchtext/x-patch; charset=US-ASCII; name=0001-txid_status-v3.patchDownload
From 737fe0a0f8e76fc24c19e2e56a2890ff56f37075 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 26 ++++++++++++
src/backend/access/transam/clog.c | 23 -----------
src/backend/utils/adt/txid.c | 85 ++++++++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 +++++++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 50 ++++++++++++++++++++++
src/test/regress/sql/txid.sql | 35 ++++++++++++++++
8 files changed, 222 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6355300..420cced 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,23 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Any recent transaction can be identified as one of
+ <itemizedlist>
+ <listitem><para>in progress</></>
+ <listitem><para>committed</></>
+ <listitem><para>aborted</></>
+ </itemizedlist>
+ Prepared transactions are identified as <literal>in progress</>.
+ The commit status of transactions older than the transaction ID wrap-around
+ threshold is no longer known by the system, so <function>txid_status</>
+ returns <literal>NULL</> for such transactions. Applications may use
+ <function>txid_status</> to determine whether a transaction committed
+ or aborted when the application and/or database server crashed or lost
+ connection while a <literal>COMMIT</literal> command was in progress.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..70fdffb 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -117,6 +118,50 @@ convert_xid(TransactionId xid, const TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound detection.
+ * ERRORs if the txid is in the future. Returns permanent XIDs unchanged.
+ * Otherwise returns the 32-bit xid and sets the wraparound param to true
+ * if wraparound is detected, false otherwise.
+ */
+static TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *wraparound)
+{
+ uint32 xid_epoch = (uint32)(xid_with_epoch >>32);
+ TransactionId xid = (TransactionId)(xid_with_epoch);
+ TxidEpoch now_epoch;
+
+ load_xid_epoch(&now_epoch);
+
+ *wraparound = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch && xid < now_epoch.last_xid))
+ {
+ /* xid too far in the past */
+ *wraparound = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
+
+/*
* txid comparator for qsort/bsearch
*/
static int
@@ -354,6 +399,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +706,40 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Underlying implementation of txid_status, which is mapped to an enum in
+ * system_views.sql.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid = get_xid_in_recent_past(xid_with_epoch, &wraparound);
+ const char *status;
+
+ if (!TransactionIdIsValid(xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (wraparound)
+ PG_RETURN_NULL();
+
+ if (TransactionIdIsCurrentTransactionId(xid) || TransactionIdIsInProgress(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /* shouldn't happen */
+ ereport(ERROR,
+ (errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 050a98c..472bf4e 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4916,6 +4916,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..466b7a9 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,53 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..fb4ef37 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,38 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-txid_convert_if_recent-v3.patchtext/x-patch; charset=US-ASCII; name=0002-txid_convert_if_recent-v3.patchDownload
From 04ba167c124939e53d92f060ec11a634ca34f7fa Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 4 +-
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
7 files changed, 184 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 420cced..8bf3eec 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 70fdffb..2f1b614 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -707,6 +707,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Underlying implementation of txid_status, which is mapped to an enum in
* system_views.sql.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 472bf4e..1873893 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4916,8 +4916,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 466b7a9..5dee554 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -282,6 +339,49 @@ SELECT txid_status(:inprogress) AS inprogress;
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index fb4ef37..2d451e6 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,12 +72,63 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Tue, Aug 23, 2016 at 12:59 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Also fine by me. You're right, keep it simple. It means the potential set of
values isn't discoverable the same way, but ... meh. Using it usefully means
reading the docs anyway.The remaining 2 patches of interest are attached - txid_status() and
txid_convert_if_recent(). Thanks for committing txid_current_if_assigned().Now I'd best stop pretending I'm in a sensible timezone.
I reviewed this version some more and found some more problems.
+ uint32 xid_epoch = (uint32)(xid_with_epoch >>32);
+ TransactionId xid = (TransactionId)(xid_with_epoch);I think this is not project style. In particular, I think that the
first one needs a space after the cast and another space before the
32; and I think the second one has an unnecessary set of parentheses
and needs a space added.
+/*
+ * Underlying implementation of txid_status, which is mapped to an enum in
+ * system_views.sql.
+ */Not any more...
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));Spacing.
+ if (TransactionIdIsCurrentTransactionId(xid) ||
TransactionIdIsInProgress(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /* shouldn't happen */
+ ereport(ERROR,
+ (errmsg_internal("unable to determine commit status of
xid "UINT64_FORMAT, xid)));Maybe I'm all wet here, but it seems like there might be a problem
here if the XID is older than the CLOG truncation point but less than
one epoch old. get_xid_in_recent_past only guarantees that the
transaction is less than one epoch old, not that we still have CLOG
data for it. And there's nothing to keep NextXID from advancing under
us, so if somebody asks about a transaction that's just under 2^32
transactions old, then get_xid_in_recent_past could say it's valid,
then NextXID advances and we look up the XID extracted from the txid
and get the status of the new transaction instead of the old one!
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 24 August 2016 at 03:10, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Aug 23, 2016 at 12:59 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Also fine by me. You're right, keep it simple. It means the potential set of
values isn't discoverable the same way, but ... meh. Using it usefully means
reading the docs anyway.The remaining 2 patches of interest are attached - txid_status() and
txid_convert_if_recent(). Thanks for committing txid_current_if_assigned().Now I'd best stop pretending I'm in a sensible timezone.
I reviewed this version some more and found some more problems.
Thanks. It took me a few days to prep a new patch as I found another
issue in the process. Updated patch attached.
The updated series starts (0001) with a change to slru.c to release
the control lock when throwing an exception so that we don't deadlock
with ourselves when re-entering slru.c; explanation below.
Then there's the txid_status (0002) patch with fixes, then
txid_convert_if_recent(0003).
I omitted txid_incinerate() ; I have an updated version that sets xact
status to aborted for burned xacts instead of leaving them at 0
(in-progress), but haven't had time to finish it so it doesn't try to
blindly set the status of xacts on pages where it didn't hold
XidGenLock when the page was added to the clog.
+ uint32 xid_epoch = (uint32)(xid_with_epoch >>32); + TransactionId xid = (TransactionId)(xid_with_epoch);I think this is not project style. In particular, I think that the
first one needs a space after the cast and another space before the
32; and I think the second one has an unnecessary set of parentheses
and needs a space added.
OK, no problems. I didn't realise spacing around casts was specified.
+/* + * Underlying implementation of txid_status, which is mapped to an enum in + * system_views.sql. + */Not any more...
That's why I shouldn't revise a patch at 1am ;)
+ if (TransactionIdIsCurrentTransactionId(xid) || TransactionIdIsInProgress(xid)) + status = gettext_noop("in progress"); + else if (TransactionIdDidCommit(xid)) + status = gettext_noop("committed"); + else if (TransactionIdDidAbort(xid)) + status = gettext_noop("aborted"); + else + /* shouldn't happen */ + ereport(ERROR, + (errmsg_internal("unable to determine commit status of xid "UINT64_FORMAT, xid)));Maybe I'm all wet here, but it seems like there might be a problem
here if the XID is older than the CLOG truncation point but less than
one epoch old. get_xid_in_recent_past only guarantees that the
transaction is less than one epoch old, not that we still have CLOG
data for it.
Good point. The call would then fail with something like
ERROR: could not access status of transaction 778793573
DETAIL: could not open file "pg_clog/02E6": No such file or directory
This probably didn't come up in my wraparound testing because I'm
aggressively forcing wraparound by writing a lot of clog very quickly
under XidGenLock, and because I'm mostly looking at xacts that are
either recent or past the xid boundary. I've added better add coverage
for xacts around 2^30 behind the nextXid to the wraparound tests;
can't add them to txid.sql since the xid never gets that far in normal
regression testing.
What I'd really like is to be able to ask transam.c to handle the
xid_in_recent_past logic, treating an attempt to read an xid from
beyond the clog truncation threshold as a soft error indicating
unknown xact state. But that involves delving into slru.c, and I
really, really don't want to touch that for what should be a simple
and pretty noninvasive utility function.
A PG_TRY to trap ERRCODE_UNDEFINED_FILE seems like it'd be sufficient,
except for two issues:
* I see no accepted way to access the errcode etc from within the
PG_CATCH block, though. PG_RE_THROW() can do it because pg_re_throw()
is in elog.c. I couldn't find any existing code that seems to check
details about an error thrown in a PG_TRY block, only SPI calls. I
don't want to ignore all types of errors and potentially hide
problems, so I just used geterrcode() - which is meant for errcontext
callbacks - and changed the comment to say it can be used in PG_CATCH
too. I don't see why it shouldn't be.
We should probably have some sort of PG_CATCH_INFO(varname) that
exposes the top ErrorData, but that's not needed for this patch so I
left it alone.
* TransactionIdGetStatus() releases the CLogControlLock taken by
SimpleLruReadPage_ReadOnly() on normal exit but not after an exception
thrown from SlruReportIOError(). It seems appropriate for
SimpleLruReadPage() to release the LWLock before calling
SlruReportIOError(), so I've done that as a separate patch (now 0001).
I also removed the TransactionIdInProgress check in txid_status and
just assume it's in progress if it isn't our xid, committed or
aborted. TransactionIdInProgress looks like it's potentially more
expensive, and most of the time we'll be looking at committed or
aborted xacts anyway. I can't sanity-check TransactionIdInProgress
after commited/aborted, because there's then a race where the xact can
commit or abort after we decide it's not committed/aborted so it's not
in progress when we go to check that.
And there's nothing to keep NextXID from advancing under
us, so if somebody asks about a transaction that's just under 2^32
transactions old, then get_xid_in_recent_past could say it's valid,
then NextXID advances and we look up the XID extracted from the txid
and get the status of the new transaction instead of the old one!
Hm, yeah. Though due to the clog truncation issue you noticed it
probably won't happen.
We could require that XidGenLock be held at least as LW_SHARED when
entering get_xid_in_recent_past(), but I'd rather not since that'd be
an otherwise-unnecessary lwlock for txid_convert_ifrecent().
Instead, I think I'll rename the wraparound flag to too_old and set it
if the xact is more than say 2^30 from the epoch struct's last_xid,
leaving a race window so ridiculously improbable that the nearly
impossible chance of failing with a clog access error is not a worry.
If the server's managing to have a proc stuck that long then it's
already on fire. We're only interested in reasonably recent xacts
since we can only work with xacts before wraparound / clog truncation.
This just moves the threshold for "reasonably recent" a bit closer.
All this certainly reinforces my view that users handling 'xid'
directly or trying to extract it from a bigint epoch-extended xid is a
bad idea that needs to go away soon.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Release-SLRU-control-lock-before-reporting-I-O-error.patchtext/x-patch; charset=US-ASCII; name=0001-Release-SLRU-control-lock-before-reporting-I-O-error.patchDownload
From 0b928fb91b5b3aa527372495e9fa4778c5b5bfab Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Thu, 25 Aug 2016 10:59:57 +0800
Subject: [PATCH 1/4] Release SLRU control lock before reporting I/O error
To allow callers to trap SLRU I/O errors with a PG_TRY() / PG_CATCH() block,
slru.c functions that acquire the SLRU control lock now also release the SLRU
control LWLock before reporting I/O errors.
Many SLRU functions previously took the LWLock and threw exceptions with it
still held.
---
src/backend/access/transam/slru.c | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/src/backend/access/transam/slru.c b/src/backend/access/transam/slru.c
index bbae584..f020522 100644
--- a/src/backend/access/transam/slru.c
+++ b/src/backend/access/transam/slru.c
@@ -366,7 +366,8 @@ SimpleLruWaitIO(SlruCtl ctl, int slotno)
* Return value is the shared-buffer slot number now holding the page.
* The buffer's LRU access info is updated.
*
- * Control lock must be held at entry, and will be held at exit.
+ * Control lock must be held at entry, and will be held at normal exit.
+ * The control lock is released if an exception is thrown.
*/
int
SimpleLruReadPage(SlruCtl ctl, int pageno, bool write_ok,
@@ -439,7 +440,10 @@ SimpleLruReadPage(SlruCtl ctl, int pageno, bool write_ok,
/* Now it's okay to ereport if we failed */
if (!ok)
+ {
+ LWLockRelease(shared->ControlLock);
SlruReportIOError(ctl, pageno, xid);
+ }
SlruRecentlyUsed(shared, slotno);
return slotno;
@@ -457,8 +461,9 @@ SimpleLruReadPage(SlruCtl ctl, int pageno, bool write_ok,
* Return value is the shared-buffer slot number now holding the page.
* The buffer's LRU access info is updated.
*
- * Control lock must NOT be held at entry, but will be held at exit.
- * It is unspecified whether the lock will be shared or exclusive.
+ * Control lock must NOT be held at entry, but will be held at exit
+ * unless an exception is thrown. It is unspecified whether the lock
+ * will be shared or exclusive.
*/
int
SimpleLruReadPage_ReadOnly(SlruCtl ctl, int pageno, TransactionId xid)
@@ -498,7 +503,8 @@ SimpleLruReadPage_ReadOnly(SlruCtl ctl, int pageno, TransactionId xid)
* the write). However, we *do* attempt a fresh write even if the page
* is already being written; this is for checkpoints.
*
- * Control lock must be held at entry, and will be held at exit.
+ * Control lock must be held at entry, and will be held at normal exit.
+ * The lock is released if an exception is thrown.
*/
static void
SlruInternalWritePage(SlruCtl ctl, int slotno, SlruFlush fdata)
@@ -564,7 +570,10 @@ SlruInternalWritePage(SlruCtl ctl, int slotno, SlruFlush fdata)
/* Now it's okay to ereport if we failed */
if (!ok)
+ {
+ LWLockRelease(shared->ControlLock);
SlruReportIOError(ctl, pageno, InvalidTransactionId);
+ }
}
/*
--
2.5.5
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 37e3f6195a7ba58de0706f00bfec630c85d3d530 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 2/4] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
To handle failure to access the clog when the xid specified is newer
than the wraparound threshold but older than the clog truncation
threshold we need to catch exceptions and check to see if the failure
was an I/O error. The only documented way to get the error code is for
use in errcontext callbacks, the errcode() call. There's no good
reason it can't also be used in PG_CATCH() blocks so amend the
comment in elog.c.
---
doc/src/sgml/func.sgml | 26 ++++++++
src/backend/access/transam/clog.c | 23 -------
src/backend/utils/adt/txid.c | 132 +++++++++++++++++++++++++++++++++++++
src/backend/utils/error/elog.c | 5 +-
src/include/access/clog.h | 23 +++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++
9 files changed, 293 insertions(+), 25 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5148095..db11f29 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,23 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Any recent transaction can be identified as one of
+ <itemizedlist>
+ <listitem><para>in progress</></>
+ <listitem><para>committed</></>
+ <listitem><para>aborted</></>
+ </itemizedlist>
+ Prepared transactions are identified as <literal>in progress</>.
+ The commit status of transactions older than the transaction ID wrap-around
+ threshold is no longer known by the system, so <function>txid_status</>
+ returns <literal>NULL</> for such transactions. Applications may use
+ <function>txid_status</> to determine whether a transaction committed
+ or aborted when the application and/or database server crashed or lost
+ connection while a <literal>COMMIT</literal> command was in progress.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..a0214a5 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -117,6 +118,64 @@ convert_xid(TransactionId xid, const TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection. ERRORs if the txid is in the future. Returns permanent
+ * XIDs unchanged. Otherwise returns the 32-bit xid and sets the
+ * too_old param to true if wraparound for this xid is close,
+ * otherwise false.
+ *
+ * To guard against wraparound occurring just after we check the xid,
+ * such that the xid is not wrapped when we return it but wraps
+ * shortly afterwards, treat xids more than 2^30 from the current
+ * nextXid as too old. That way there's a large margin of nextXid
+ * growth before TransactionIdPrecedes would start reporting the
+ * queried xid as in the future relative to new xacts as it crosses
+ * the 2^31 difference boundary.
+ *
+ * In practice the clog will usually be truncated away for such xids
+ * already so they're of limited utility.
+ */
+static TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *too_old)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ TxidEpoch now_epoch;
+ const int wraparound_safety_margin = 1 << 30;
+
+ load_xid_epoch(&now_epoch);
+
+ *too_old = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch
+ && (now_epoch.last_xid - xid) > wraparound_safety_margin))
+ {
+ /* xid too far in the past */
+ *too_old = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
+
+/*
* txid comparator for qsort/bsearch
*/
static int
@@ -354,6 +413,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +720,73 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ bool too_old;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid = get_xid_in_recent_past(xid_with_epoch, &too_old);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (too_old)
+ status = NULL;
+ else if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else
+ {
+ /*
+ * clog access might fail here if the requested xid is past the
+ * clog truncation threshold but not wrapped around. We want to
+ * return null in such cases rather than raise an error.
+ */
+ PG_TRY();
+ {
+ if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ PG_CATCH();
+ {
+ if (geterrcode() == ERRCODE_UNDEFINED_FILE)
+ {
+ /*
+ * The clog SLRU reports that it can't find the segment for
+ * the xid of interest. Presumably it has been truncated
+ * away, so the xact is too old to report status for.
+ */
+ elog(INFO, "clog access failed with access error");
+ status = NULL;
+ }
+ else
+ PG_RE_THROW();
+
+ }
+ PG_END_TRY();
+ }
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 224ee78..10bca81 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1233,8 +1233,9 @@ set_errdata_field(MemoryContextData *cxt, char **ptr, const char *str)
/*
* geterrcode --- return the currently set SQLSTATE error code
*
- * This is only intended for use in error callback subroutines, since there
- * is no other place outside elog.c where the concept is meaningful.
+ * This is only intended for use in error callback subroutines and PG_CATCH
+ * blocks, since there is no other place outside elog.c where the concept is
+ * meaningful.
*/
int
geterrcode(void)
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..1a0b84c 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- FirstNormalTransactionId is always committed for the first epoch because of initdb work
+ txid_status
+-------------
+ committed
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..1c56ec1 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- FirstNormalTransactionId is always committed for the first epoch because of initdb work
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0003-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0003-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From 65faf9c9e930a1e1aa3267dd03606f30fbb740bf Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 3/4] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 4 +-
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
7 files changed, 184 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index db11f29..1c6f830 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index a0214a5..6302a70 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -721,6 +721,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 1a0b84c..01c5673 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- FirstNormalTransactionId is always committed for the f
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 1c56ec1..454aba4 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- FirstNormalTransactionId is always committed for the f
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
Hi,
On 2016-08-29 11:25:39 +0800, Craig Ringer wrote:
ERROR: could not access status of transaction 778793573
DETAIL: could not open file "pg_clog/02E6": No such file or directoryWhat I'd really like is to be able to ask transam.c to handle the
xid_in_recent_past logic, treating an attempt to read an xid from
beyond the clog truncation threshold as a soft error indicating
unknown xact state. But that involves delving into slru.c, and I
really, really don't want to touch that for what should be a simple
and pretty noninvasive utility function.
Can't you "just" check this against ShmemVariableCache->oldestXid while
holding appropriate locks?
A PG_TRY to trap ERRCODE_UNDEFINED_FILE seems like it'd be sufficient,
except for two issues:
It seems like a bad idea to PG_CATCH and not re-throw an error. That
generally is quite error prone. At the very least locking and such gets
a lot more complicated (as you noticed below).
* TransactionIdGetStatus() releases the CLogControlLock taken by
SimpleLruReadPage_ReadOnly() on normal exit but not after an exception
thrown from SlruReportIOError(). It seems appropriate for
SimpleLruReadPage() to release the LWLock before calling
SlruReportIOError(), so I've done that as a separate patch (now 0001).
We normally prefer to handle this via the "bulk" releases in the error
handlers. It's otherwise hard to write code that handles these
situations reliably. It's different for spinlocks, but those normally
protect far smaller regions of code.
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Sun, Aug 28, 2016 at 11:25 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
What I'd really like is to be able to ask transam.c to handle the
xid_in_recent_past logic, treating an attempt to read an xid from
beyond the clog truncation threshold as a soft error indicating
unknown xact state. But that involves delving into slru.c, and I
really, really don't want to touch that for what should be a simple
and pretty noninvasive utility function.
I think you're going to have to bite the bullet and do that, though, because ...
A PG_TRY to trap ERRCODE_UNDEFINED_FILE seems like it'd be sufficient,
...I don't think this has any chance of being acceptable. You can't
catch errors and not re-throw them. That's bad news.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 29 August 2016 at 11:45, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2016-08-29 11:25:39 +0800, Craig Ringer wrote:
ERROR: could not access status of transaction 778793573
DETAIL: could not open file "pg_clog/02E6": No such file or directoryWhat I'd really like is to be able to ask transam.c to handle the
xid_in_recent_past logic, treating an attempt to read an xid from
beyond the clog truncation threshold as a soft error indicating
unknown xact state. But that involves delving into slru.c, and I
really, really don't want to touch that for what should be a simple
and pretty noninvasive utility function.Can't you "just" check this against ShmemVariableCache->oldestXid while
holding appropriate locks?
Hm. Yeah, I should've thought of that. Thank you.
A PG_TRY to trap ERRCODE_UNDEFINED_FILE seems like it'd be sufficient,
except for two issues:It seems like a bad idea to PG_CATCH and not re-throw an error. That
generally is quite error prone. At the very least locking and such gets
a lot more complicated (as you noticed below).
Yeah, and as I remember from the "fun" of trying to write apply errors
to tables in BDR. It wasn't my first choice.
* TransactionIdGetStatus() releases the CLogControlLock taken by
SimpleLruReadPage_ReadOnly() on normal exit but not after an exception
thrown from SlruReportIOError(). It seems appropriate for
SimpleLruReadPage() to release the LWLock before calling
SlruReportIOError(), so I've done that as a separate patch (now 0001).We normally prefer to handle this via the "bulk" releases in the error
handlers. It's otherwise hard to write code that handles these
situations reliably. It's different for spinlocks, but those normally
protect far smaller regions of code.
Fair enough. It's not a complex path, but there are a _lot_ of
callers, and while I can't really imagine any of them relying on the
CLogControLock being held on error it's not something I was keen to
change. I thought complicating the clog with a soft-error interface
was worse and didn't come up with a better approach.
Said better approach attached in revised series. Thanks.
My only real complaint with doing this is that it's a bit more
conservative. But in practice clog truncation probably won't follow
that far behind oldestXmin so except in fairly contrived circumstances
it won't hurt. Apps that need guarantees about how old an xid they can
get status on can hold down xmin with a replication slot, a dummy
prepared xact, or whatever. If we find that becomes a common need that
should be made simpler then appropriate API to allow apps to hold down
clog truncation w/o blocking vacuuming can be added down the track.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From b69f99b63f667e745ccdee2130ee1b50690109d4 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 31 ++++++++++
src/backend/access/transam/clog.c | 23 -------
src/backend/utils/adt/txid.c | 119 +++++++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 +++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 ++++++++++++
8 files changed, 282 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5148095..d8b086f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..69148a5 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -117,6 +118,67 @@ convert_xid(TransactionId xid, const TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns permanent XIDs
+ * unchanged. Otherwise returns the 32-bit xid and sets the too_old
+ * param to true if status for this xid cannot be reliably determined.
+ * It's only safe to use the returned xid for most purposes if too_old
+ * is false on return.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+static TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *too_old)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ TxidEpoch now_epoch;
+
+ load_xid_epoch(&now_epoch);
+
+ *too_old = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch && xid < now_epoch.last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ *too_old = true;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ *too_old = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
+
+/*
* txid comparator for qsort/bsearch
*/
static int
@@ -354,6 +416,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +723,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ bool too_old;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid
+ * is ok and when we look it up in the clog. Otherwise an
+ * exception might get thrown on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ xid = get_xid_in_recent_past(xid_with_epoch, &too_old);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (too_old)
+ status = NULL;
+ else if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From ae1b034b9600901afaebc8b67f1a5c233c82e13e Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 4 +-
src/include/utils/builtins.h | 1 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
8 files changed, 185 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8b086f..fe3325b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 69148a5..8052f61 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -724,6 +724,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index baffa38..a95a50f 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1228,6 +1228,7 @@ extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
extern Datum txid_status(PG_FUNCTION_ARGS);
+extern Datum txid_convert_if_recent(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 015dae3..3e3075d 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index bd6decf..cb22007 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 29 August 2016 at 15:53, Craig Ringer <craig@2ndquadrant.com> wrote:
Said better approach attached in revised series. Thanks.
Here's another minor update to the txid_status() and
txid_convert_if_recent() patches. The only change is moving
get_xid_in_recent_past from src/backend/utils/adt/txid.c to
src/backend/access/transam/xlog.c to permit its use by other code.
Specifically, I think it'll be needed for logical decoding on standby.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 44f1ddfcf43bb83ce1478c0290fbbaa916fdc22e Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 31 +++++++++++++++++
src/backend/access/transam/clog.c | 23 -------------
src/backend/access/transam/xlog.c | 61 ++++++++++++++++++++++++++++++++++
src/backend/utils/adt/txid.c | 58 ++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 +++++++++++++
src/include/access/xlog.h | 2 ++
src/include/catalog/pg_proc.h | 2 ++
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 ++++++++++++++++++++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++++++++++++
10 files changed, 284 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5148095..d8b086f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0b991bb..e625922 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -11638,3 +11638,64 @@ XLogRequestWalReceiverReply(void)
{
doRequestWalReceiverReply = true;
}
+
+/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns permanent XIDs
+ * unchanged. Otherwise returns the 32-bit xid and sets the too_old
+ * param to true if status for this xid cannot be reliably determined.
+ * It's only safe to use the returned xid for most purposes if too_old
+ * is false on return.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+TransactionId
+get_xid_in_recent_past(txid xid_with_epoch, bool *too_old)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ TxidEpoch now_epoch;
+
+ load_xid_epoch(&now_epoch);
+
+ *too_old = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch.epoch
+ || (xid_epoch == now_epoch.epoch && xid > now_epoch.last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch.epoch
+ || (xid_epoch + 1 == now_epoch.epoch && xid < now_epoch.last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ *too_old = true;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ *too_old = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch.last_xid));
+ }
+
+ return xid;
+}
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..3d94a1a 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -354,6 +355,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +662,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ bool too_old;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid
+ * is ok and when we look it up in the clog. Otherwise an
+ * exception might get thrown on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ xid = get_xid_in_recent_past(xid_with_epoch, &too_old);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (too_old)
+ status = NULL;
+ else if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..f383b03 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -273,6 +273,8 @@ extern void XLogRequestWalReceiverReply(void);
extern void assign_max_wal_size(int newval, void *extra);
extern void assign_checkpoint_completion_target(double newval, void *extra);
+extern TransactionId get_xid_in_recent_past(txid xid_with_epoch, bool *too_old);
+
/*
* Starting/stopping a base backup
*/
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From fe65578fec1bfc74cacdf4941286046677fb2f4b Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 4 +-
src/include/utils/builtins.h | 1 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
8 files changed, 185 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8b086f..fe3325b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 3d94a1a..14aae61 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -663,6 +663,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index baffa38..a95a50f 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1228,6 +1228,7 @@ extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
extern Datum txid_status(PG_FUNCTION_ARGS);
+extern Datum txid_convert_if_recent(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 015dae3..3e3075d 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index bd6decf..cb22007 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 1 September 2016 at 13:08, Craig Ringer <craig@2ndquadrant.com> wrote:
On 29 August 2016 at 15:53, Craig Ringer <craig@2ndquadrant.com> wrote:
Said better approach attached in revised series. Thanks.
Here's another minor update to the txid_status() and
txid_convert_if_recent() patches. The only change is moving
get_xid_in_recent_past from src/backend/utils/adt/txid.c to
src/backend/access/transam/xlog.c to permit its use by other code.
Specifically, I think it'll be needed for logical decoding on standby.
Ahem, wrong patches. Attached correctly now.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 1e81a5a7c8a450925919903ab68b70926af9d69d Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 31 +++++++++++++++++
src/backend/access/transam/clog.c | 23 -------------
src/backend/access/transam/xlog.c | 62 ++++++++++++++++++++++++++++++++++
src/backend/utils/adt/txid.c | 58 ++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 +++++++++++++
src/include/access/xlog.h | 2 ++
src/include/catalog/pg_proc.h | 2 ++
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 ++++++++++++++++++++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++++++++++++
10 files changed, 285 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5148095..d8b086f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0b991bb..8336d5a 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -11638,3 +11638,65 @@ XLogRequestWalReceiverReply(void)
{
doRequestWalReceiverReply = true;
}
+
+/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns permanent XIDs
+ * unchanged. Otherwise returns the 32-bit xid and sets the too_old
+ * param to true if status for this xid cannot be reliably determined.
+ * It's only safe to use the returned xid for most purposes if too_old
+ * is false on return.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+TransactionId
+get_xid_in_recent_past(uint64 xid_with_epoch, bool *too_old)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch, &now_epoch_last_xid);
+
+ *too_old = false;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ return xid;
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ *too_old = true;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ *too_old = true;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ return xid;
+}
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..3d94a1a 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -354,6 +355,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +662,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ bool too_old;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid
+ * is ok and when we look it up in the clog. Otherwise an
+ * exception might get thrown on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ xid = get_xid_in_recent_past(xid_with_epoch, &too_old);
+
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (too_old)
+ status = NULL;
+ else if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..9379ba1 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -273,6 +273,8 @@ extern void XLogRequestWalReceiverReply(void);
extern void assign_max_wal_size(int newval, void *extra);
extern void assign_checkpoint_completion_target(double newval, void *extra);
+extern TransactionId get_xid_in_recent_past(uint64 xid_with_epoch, bool *too_old);
+
/*
* Starting/stopping a base backup
*/
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From fb4457240082d65e264e389139e9c0d0f00816dc Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 12 ++++
src/include/catalog/pg_proc.h | 4 +-
src/include/utils/builtins.h | 1 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
8 files changed, 185 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8b086f..fe3325b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 3d94a1a..14aae61 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -663,6 +663,18 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ bool wraparound;
+ TransactionId xid = get_xid_in_recent_past(PG_GETARG_INT64(0), &wraparound);
+
+ if (wraparound)
+ PG_RETURN_NULL();
+ else
+ return TransactionIdGetDatum(xid);
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index baffa38..a95a50f 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1228,6 +1228,7 @@ extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
extern Datum txid_status(PG_FUNCTION_ARGS);
+extern Datum txid_convert_if_recent(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 015dae3..3e3075d 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index bd6decf..cb22007 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
Here's another update to these patches. While working on support for
logical decoding on standbys I noticed that I needed something like
get_xid_in_recent_past(...) there too. So I've moved it to xlog.c as
TransactionIdInRecentPast too and flipped its arguments to be more
convenient. No other changes.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 4f7b79a94593004fb62b549cb9c04686e5b6ebb2 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
---
doc/src/sgml/func.sgml | 31 +++++++++++++++++
src/backend/access/transam/clog.c | 23 -------------
src/backend/access/transam/xlog.c | 69 ++++++++++++++++++++++++++++++++++++++
src/backend/utils/adt/txid.c | 61 +++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 +++++++++++++
src/include/access/xlog.h | 2 ++
src/include/catalog/pg_proc.h | 2 ++
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++++++++++++
10 files changed, 295 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5148095..d8b086f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 0b991bb..8e0c2f9 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -11638,3 +11638,72 @@ XLogRequestWalReceiverReply(void)
{
doRequestWalReceiverReply = true;
}
+
+/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns true if the xid is
+ * within the xid wraparound threshold and clog truncation threshold
+ * or if the xid is a permanent xid. Returns false if the xid is past
+ * the clog truncation threshold or wraparound threshold. Sets
+ * extracted_xid to the 32-bit xid.
+ *
+ * It's only safe to use the extracted_xid for most purposes if the
+ * function returns true, otherwise the clog could be truncated away
+ * or it might be an xid from a prior epoch.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+ bool result;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ result = true;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ result = false;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ result = false;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ return result;
+}
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..4b33e89 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -354,6 +355,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +662,60 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ bool too_old;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid
+ * is ok and when we look it up in the clog. Otherwise an
+ * exception might get thrown on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 14b7f7f..5484424 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -273,6 +273,8 @@ extern void XLogRequestWalReceiverReply(void);
extern void assign_max_wal_size(int newval, void *extra);
extern void assign_checkpoint_completion_target(double newval, void *extra);
+extern bool TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid);
+
/*
* Starting/stopping a base backup
*/
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From 408cfe5e177bb7b930b1d52e9807ff4adac84542 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 11 ++++
src/include/catalog/pg_proc.h | 4 +-
src/include/utils/builtins.h | 1 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
8 files changed, 184 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8b086f..fe3325b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 4b33e89..258014e 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -663,6 +663,17 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ TransactionId xid;
+
+ if (TransactionIdInRecentPast(PG_GETARG_INT64(0), &xid))
+ return TransactionIdGetDatum(xid);
+ else
+ PG_RETURN_NULL();
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index baffa38..a95a50f 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1228,6 +1228,7 @@ extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
extern Datum txid_status(PG_FUNCTION_ARGS);
+extern Datum txid_convert_if_recent(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 015dae3..3e3075d 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index bd6decf..cb22007 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 2 September 2016 at 13:16, Craig Ringer <craig@2ndquadrant.com> wrote:
So I've moved it to xlog.c...
I'm pretty sure it shouldn't live in xlog.c, but there may be some
good reason I can't see yet.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 2 Sep. 2016 8:30 pm, "Simon Riggs" <simon@2ndquadrant.com> wrote:
On 2 September 2016 at 13:16, Craig Ringer <craig@2ndquadrant.com> wrote:
So I've moved it to xlog.c...
I'm pretty sure it shouldn't live in xlog.c, but there may be some
good reason I can't see yet.
Ugh. Yes. transam.c would be rather saner.
Only for the helper to determine if an xid is recent though; txid_ status
stays in adt/xact.c where it belongs along with txid_current() etc.
Show quoted text
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 2 September 2016 at 20:38, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 Sep. 2016 8:30 pm, "Simon Riggs" <simon@2ndquadrant.com> wrote:
On 2 September 2016 at 13:16, Craig Ringer <craig@2ndquadrant.com> wrote:
So I've moved it to xlog.c...
I'm pretty sure it shouldn't live in xlog.c, but there may be some
good reason I can't see yet.Ugh. Yes. transam.c would be rather saner.
Only for the helper to determine if an xid is recent though; txid_ status
stays in adt/xact.c where it belongs along with txid_current() etc.
Fixed, moved to transam.c.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 6a07c8eb98c87d55d61af86f94c7f2de87645922 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other functions
that need similar logic in future.
---
doc/src/sgml/func.sgml | 31 ++++++++++++++++
src/backend/access/transam/clog.c | 23 ------------
src/backend/access/transam/transam.c | 69 ++++++++++++++++++++++++++++++++++++
src/backend/utils/adt/txid.c | 61 +++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 ++++++++++++
src/include/access/transam.h | 2 ++
src/include/catalog/pg_proc.h | 2 ++
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 ++++++++++++++++++++
10 files changed, 295 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5148095..d8b086f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/access/transam/transam.c b/src/backend/access/transam/transam.c
index 1eba49a..f565189 100644
--- a/src/backend/access/transam/transam.c
+++ b/src/backend/access/transam/transam.c
@@ -423,3 +423,72 @@ TransactionIdGetCommitLSN(TransactionId xid)
return result;
}
+
+/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns true if the xid is
+ * within the xid wraparound threshold and clog truncation threshold
+ * or if the xid is a permanent xid. Returns false if the xid is past
+ * the clog truncation threshold or wraparound threshold. Sets
+ * extracted_xid to the 32-bit xid.
+ *
+ * It's only safe to use the extracted_xid for most purposes if the
+ * function returns true, otherwise the clog could be truncated away
+ * or it might be an xid from a prior epoch.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+ bool result;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ result = true;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ result = false;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ result = false;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ return result;
+}
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..4b33e89 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -354,6 +355,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +662,60 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ bool too_old;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid
+ * is ok and when we look it up in the clog. Otherwise an
+ * exception might get thrown on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 969eff9..f159fae 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -167,6 +167,8 @@ extern bool TransactionIdFollowsOrEquals(TransactionId id1, TransactionId id2);
extern TransactionId TransactionIdLatest(TransactionId mainxid,
int nxids, const TransactionId *xids);
extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
+extern bool TransactionIdInRecentPast(uint64 xid_with_epoch,
+ TransactionId *extracted_xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From 027de6a30dc2684229a12d4deff903048c75de5b Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 11 ++++
src/include/catalog/pg_proc.h | 4 +-
src/include/utils/builtins.h | 1 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
8 files changed, 184 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8b086f..fe3325b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 4b33e89..258014e 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -663,6 +663,17 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ TransactionId xid;
+
+ if (TransactionIdInRecentPast(PG_GETARG_INT64(0), &xid))
+ return TransactionIdGetDatum(xid);
+ else
+ PG_RETURN_NULL();
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index baffa38..a95a50f 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1228,6 +1228,7 @@ extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
extern Datum txid_status(PG_FUNCTION_ARGS);
+extern Datum txid_convert_if_recent(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 015dae3..3e3075d 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index bd6decf..cb22007 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 2 September 2016 at 21:01, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 September 2016 at 20:38, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 Sep. 2016 8:30 pm, "Simon Riggs" <simon@2ndquadrant.com> wrote:
On 2 September 2016 at 13:16, Craig Ringer <craig@2ndquadrant.com> wrote:
So I've moved it to xlog.c...
I'm pretty sure it shouldn't live in xlog.c, but there may be some
good reason I can't see yet.Ugh. Yes. transam.c would be rather saner.
Only for the helper to determine if an xid is recent though; txid_ status
stays in adt/xact.c where it belongs along with txid_current() etc.Fixed, moved to transam.c.
Missed that this causes an undefined reference to GetNextXidAndEpoch()
which is in xlog.c. I knew there was a reason I put it there. So most
recent patch is wrong, use the prior one.
GetNextXidAndEpoch() needs to be in xlog.c because it uses XLogCtl's
shmem copy of checkPoint.nextXidEpoch. So either transam.c needs to
#include xlog.h (which seems a bit backwards) or
TransactionIdInRecentPast() should go in xlog.c.
I don't like either really. Opinion? I'm sure we'll want this
functionality in other places as part of dealing with the problems
discussed upthread with 'xid' exposed to users.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 02/09/16 15:46, Craig Ringer wrote:
On 2 September 2016 at 21:01, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 September 2016 at 20:38, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 Sep. 2016 8:30 pm, "Simon Riggs" <simon@2ndquadrant.com> wrote:
On 2 September 2016 at 13:16, Craig Ringer <craig@2ndquadrant.com> wrote:
So I've moved it to xlog.c...
I'm pretty sure it shouldn't live in xlog.c, but there may be some
good reason I can't see yet.Ugh. Yes. transam.c would be rather saner.
Only for the helper to determine if an xid is recent though; txid_ status
stays in adt/xact.c where it belongs along with txid_current() etc.Fixed, moved to transam.c.
Missed that this causes an undefined reference to GetNextXidAndEpoch()
which is in xlog.c. I knew there was a reason I put it there. So most
recent patch is wrong, use the prior one.GetNextXidAndEpoch() needs to be in xlog.c because it uses XLogCtl's
shmem copy of checkPoint.nextXidEpoch. So either transam.c needs to
#include xlog.h (which seems a bit backwards) or
TransactionIdInRecentPast() should go in xlog.c.I don't like either really. Opinion? I'm sure we'll want this
functionality in other places as part of dealing with the problems
discussed upthread with 'xid' exposed to users.
You could put it to txid.c where all the other txid stuff is in?
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 2 September 2016 at 23:29, Petr Jelinek <petr@2ndquadrant.com> wrote:
You could put it to txid.c where all the other txid stuff is in?
Yeah, even though it's in adt/ I think it'll do.
I thought I'd need get_xid_in_recent_past() for catalog_xmin hot
standby feedback, but upon closer examination the needed logic isn't
the same anymore. txid_status() wants to ensure clog lookups are safe
and limit by oldest xid, wheras the walsender doesn't actually care
about that and is just avoiding wrapped xids.
I'm just going back to how it was, all in adt/txid.c, and making it
static again. We can move it and make it non-static if a need to do so
comes up.
Attached rebased patch updated and vs master.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Install-the-Perl-TAP-tests.patchtext/x-patch; charset=US-ASCII; name=0001-Install-the-Perl-TAP-tests.patchDownload
From 462a0ab51935b45d17820b83b8e9f6abd4ad2904 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Tue, 13 Sep 2016 11:06:58 +0800
Subject: [PATCH 1/3] Install the Perl TAP tests
---
src/Makefile | 3 ++-
src/test/Makefile | 2 +-
src/test/perl/GNUmakefile | 39 +++++++++++++++++++++++++++++++++++++++
3 files changed, 42 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/GNUmakefile
diff --git a/src/Makefile b/src/Makefile
index b526be7..977f80b 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -26,7 +26,8 @@ SUBDIRS = \
bin \
pl \
makefiles \
- test/regress
+ test/regress \
+ test/perl
# There are too many interdependencies between the subdirectories, so
# don't attempt parallel make here.
diff --git a/src/test/Makefile b/src/test/Makefile
index 7f7754f..6b40cf5 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,7 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = regress isolation modules recovery
+SUBDIRS = perl regress isolation modules recovery
# We don't build or execute examples/, locale/, or thread/ by default,
# but we do want "make clean" etc to recurse into them. Likewise for ssl/,
diff --git a/src/test/perl/GNUmakefile b/src/test/perl/GNUmakefile
new file mode 100644
index 0000000..3c5dc70
--- /dev/null
+++ b/src/test/perl/GNUmakefile
@@ -0,0 +1,39 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/perl
+#
+# Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/perl/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/perl
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(pgxsdir)/$(subdir)'
+
+ifeq ($(enable_tap_tests),yes)
+
+install: all installdirs
+ $(INSTALL_DATA) $(srcdir)/TestLib.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/TestLib.pm'
+ $(INSTALL_DATA) $(srcdir)/SimpleTee.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/SimpleTee.pm'
+ $(INSTALL_DATA) $(srcdir)/RecursiveCopy.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/RecursiveCopy.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgresNode.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgresNode.pm'
+
+uninstall:
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/TestLib.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/SimpleTee.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/RecursiveCopy.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgresNode.pm'
+
+else
+
+install: ;
+
+uninstall: ;
+
+endif
--
2.5.5
0002-Add-install-rules-for-isolation-tester.patchtext/x-patch; charset=US-ASCII; name=0002-Add-install-rules-for-isolation-tester.patchDownload
From 86aff40374e05fec0160cbab0d1879bd01c6f411 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Tue, 13 Sep 2016 11:00:41 +0800
Subject: [PATCH 2/3] Add install rules for isolation tester
Allow 'make install' for the isolation tester to work so it can be
used from PGXS extensions.
---
src/Makefile | 1 +
src/test/isolation/Makefile | 11 +++++++++++
2 files changed, 12 insertions(+)
diff --git a/src/Makefile b/src/Makefile
index 977f80b..d4aa06b 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -27,6 +27,7 @@ SUBDIRS = \
pl \
makefiles \
test/regress \
+ test/isolation \
test/perl
# There are too many interdependencies between the subdirectories, so
diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefile
index 3d272d5..e111bf0 100644
--- a/src/test/isolation/Makefile
+++ b/src/test/isolation/Makefile
@@ -66,3 +66,14 @@ installcheck-prepared-txns: all temp-install
check-prepared-txns: all temp-install
./pg_isolation_regress --temp-instance=./tmp_check $(TEMP_CONF) $(EXTRA_REGRESS_OPTS) --inputdir=$(srcdir) --schedule=$(srcdir)/isolation_schedule prepared-transactions
+
+install: all installdirs
+ $(INSTALL_PROGRAM) isolationtester$(X) '$(DESTDIR)$(pgxsdir)/$(subdir)/isolationtester$(X)'
+ $(INSTALL_PROGRAM) pg_isolation_regress$(X) '$(DESTDIR)$(pgxsdir)/$(subdir)/pg_isolation_regress$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(pgxsdir)/$(subdir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/isolationtester$(X)'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/pg_isolation_regress$(X)'
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Thu, Sep 15, 2016 at 8:52 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 September 2016 at 23:29, Petr Jelinek <petr@2ndquadrant.com> wrote:
You could put it to txid.c where all the other txid stuff is in?
Yeah, even though it's in adt/ I think it'll do.
I thought I'd need get_xid_in_recent_past() for catalog_xmin hot
standby feedback, but upon closer examination the needed logic isn't
the same anymore. txid_status() wants to ensure clog lookups are safe
and limit by oldest xid, wheras the walsender doesn't actually care
about that and is just avoiding wrapped xids.I'm just going back to how it was, all in adt/txid.c, and making it
static again. We can move it and make it non-static if a need to do so
comes up.Attached rebased patch updated and vs master.
You appear to have attached the wrong patch set.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 16 September 2016 at 21:28, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Sep 15, 2016 at 8:52 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 September 2016 at 23:29, Petr Jelinek <petr@2ndquadrant.com> wrote:
You could put it to txid.c where all the other txid stuff is in?
Yeah, even though it's in adt/ I think it'll do.
I thought I'd need get_xid_in_recent_past() for catalog_xmin hot
standby feedback, but upon closer examination the needed logic isn't
the same anymore. txid_status() wants to ensure clog lookups are safe
and limit by oldest xid, wheras the walsender doesn't actually care
about that and is just avoiding wrapped xids.I'm just going back to how it was, all in adt/txid.c, and making it
static again. We can move it and make it non-static if a need to do so
comes up.Attached rebased patch updated and vs master.
You appear to have attached the wrong patch set.
Whoops, multitasking fail.
Sorry for the late response, hospitals are "fun".
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 3d3cfd16dfc167f3cf4b8790e1625a2084dbce6b Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:44:15 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an appliation is disconnected while a COMMIT request is in flight,
the backend crashes mid-commit, etc, then an application may not be
sure whether or not a commit completed successfully or was rolled
back. While two-phase commit solves this it does so at a considerable
overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a
a commit based on an xid-with-epoch as returned by txid_current()
or similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other functions
that need similar logic in future.
---
doc/src/sgml/func.sgml | 31 +++++++++
src/backend/access/transam/clog.c | 23 -------
src/backend/utils/adt/txid.c | 130 +++++++++++++++++++++++++++++++++++++
src/include/access/clog.h | 23 +++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++
8 files changed, 293 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 47fcb30..3123232 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..1a6e26d 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -41,29 +41,6 @@
#include "miscadmin.h"
#include "pg_trace.h"
-/*
- * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
- * everywhere else in Postgres.
- *
- * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
- * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
- * and CLOG segment numbering at
- * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
- * explicit notice of that fact in this module, except when comparing segment
- * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
- */
-
-/* We need two bits per xact, so four xacts fit in a byte */
-#define CLOG_BITS_PER_XACT 2
-#define CLOG_XACTS_PER_BYTE 4
-#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
-#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
-
-#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
-#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
-#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
-
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP 32 /* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE (CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..52395ff 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,75 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns true if the xid is
+ * within the xid wraparound threshold and clog truncation threshold
+ * or if the xid is a permanent xid. Returns false if the xid is past
+ * the clog truncation threshold or wraparound threshold. Sets
+ * extracted_xid to the 32-bit xid.
+ *
+ * It's only safe to use the extracted_xid for most purposes if the
+ * function returns true, otherwise the clog could be truncated away
+ * or it might be an xid from a prior epoch.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+ bool result;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ result = true;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ result = false;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ result = false;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ return result;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +425,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +732,59 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid
+ * is ok and when we look it up in the clog. Otherwise an
+ * exception might get thrown on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..a763dfb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,29 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+/*
+ * Defines for CLOG page sizes. A page is the same BLCKSZ as is used
+ * everywhere else in Postgres.
+ *
+ * Note: because TransactionIds are 32 bits and wrap around at 0xFFFFFFFF,
+ * CLOG page numbering also wraps around at 0xFFFFFFFF/CLOG_XACTS_PER_PAGE,
+ * and CLOG segment numbering at
+ * 0xFFFFFFFF/CLOG_XACTS_PER_PAGE/SLRU_PAGES_PER_SEGMENT. We need take no
+ * explicit notice of that fact in this module, except when comparing segment
+ * and page numbers in TruncateCLOG (see CLOGPagePrecedes).
+ */
+
+/* We need two bits per xact, so four xacts fit in a byte */
+#define CLOG_BITS_PER_XACT 2
+#define CLOG_XACTS_PER_BYTE 4
+#define CLOG_XACTS_PER_PAGE (BLCKSZ * CLOG_XACTS_PER_BYTE)
+#define CLOG_XACT_BITMASK ((1 << CLOG_BITS_PER_XACT) - 1)
+
+#define TransactionIdToPage(xid) ((xid) / (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToPgIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_PAGE)
+#define TransactionIdToByte(xid) (TransactionIdToPgIndex(xid) / CLOG_XACTS_PER_BYTE)
+#define TransactionIdToBIndex(xid) ((xid) % (TransactionId) CLOG_XACTS_PER_BYTE)
+
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchtext/x-patch; charset=US-ASCII; name=0002-Add-txid_convert_if_recent-to-get-the-32-bit-xid-fro.patchDownload
From ea7e16c45991664deae628bd7b41ea8d96f05a12 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 19 Aug 2016 14:49:52 +0800
Subject: [PATCH 2/3] Add txid_convert_if_recent() to get the 32-bit xid from a
bigint xid
txid_current() returns an epoch-extended 64-bit xid as a bigint, but
many PostgreSQL functions take and many views report the narrow 32-bit
'xid' type that's subject to wrap-around. To compare these apps must
currently bit-shift the 64-bit xid down and they have no way to check
the epoch.
Add a function that returns the downshifted xid if it's in the current
epoch, or null if the xid is too far in the past and cannot be
compared with any 'xid' value in the current server epoch.
---
doc/src/sgml/func.sgml | 17 ++++-
src/backend/utils/adt/txid.c | 11 ++++
src/include/catalog/pg_proc.h | 4 +-
src/include/utils/builtins.h | 1 +
src/test/regress/expected/alter_table.out | 4 +-
src/test/regress/expected/txid.out | 100 ++++++++++++++++++++++++++++++
src/test/regress/sql/alter_table.sql | 4 +-
src/test/regress/sql/txid.sql | 51 +++++++++++++++
8 files changed, 184 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 3123232..ed0f627 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17198,6 +17198,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
<row>
+ <entry><literal><function>txid_convert_if_recent(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>xid</type></entry>
+ <entry>return the 32-bit <type>xid</> for a 64-bit transaction ID if it isn't wrapped around, otherwise return null</entry>
+ </row>
+ <row>
<entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
<entry><type>txid_status</type></entry>
<entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
@@ -17210,9 +17215,15 @@ SELECT collation for ('foo' COLLATE "de_DE");
The internal transaction ID type (<type>xid</>) is 32 bits wide and
wraps around every 4 billion transactions. However, these functions
export a 64-bit format that is extended with an <quote>epoch</> counter
- so it will not wrap around during the life of an installation.
- The data type used by these functions, <type>txid_snapshot</type>,
- stores information about transaction ID
+ so it will not wrap around during the life of an installation. For that
+ reason you cannot cast a bigint transaction ID directly to <type>xid</>
+ and must use <function>txid_convert_if_recent(bigint)</function> instead of
+ casting to <type>xid</>.
+ </para>
+
+ <para>
+ The data type used by the xid snapshot functions,
+ <type>txid_snapshot</type>, stores information about transaction ID
visibility at a particular moment in time. Its components are
described in <xref linkend="functions-txid-snapshot-parts">.
</para>
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 52395ff..6287eff 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -733,6 +733,17 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
}
}
+Datum
+txid_convert_if_recent(PG_FUNCTION_ARGS)
+{
+ TransactionId xid;
+
+ if (TransactionIdInRecentPast(PG_GETARG_INT64(0), &xid))
+ return TransactionIdGetDatum(xid);
+ else
+ PG_RETURN_NULL();
+}
+
/*
* Report the status of a recent transaction ID, or null for wrapped,
* truncated away or otherwise too old XIDs.
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0ad870c..59fa907 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,8 +4928,10 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
-DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DATA(insert OID = 3347 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
DESCR("commit status of transaction");
+DATA(insert OID = 3344 ( txid_convert_if_recent PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 28 "20" _null_ _null_ _null_ _null_ _null_ txid_convert_if_recent _null_ _null_ _null_ ));
+DESCR("get the xid from a bigint transaction id if not wrapped around");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index baffa38..a95a50f 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1228,6 +1228,7 @@ extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
extern Datum txid_status(PG_FUNCTION_ARGS);
+extern Datum txid_convert_if_recent(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 3232cda..3bdbb87 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2029,7 +2029,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -2192,7 +2192,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 015dae3..3e3075d 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -263,6 +263,63 @@ SELECT txid_current() AS rolledback \gset
ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+ ?column?
+----------
+ t
+(1 row)
+
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+ txid_convert_if_recent
+------------------------
+ 0
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+ txid_convert_if_recent
+------------------------
+ 1
+(1 row)
+
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+ txid_convert_if_recent
+------------------------
+ 2
+(1 row)
+
SELECT txid_status(:committed) AS committed;
committed
-----------
@@ -300,6 +357,49 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
(1 row)
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+NOTICE: got expected xid out of range error
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid
+-----------------
+
+(1 row)
+
+ROLLBACK;
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 72e65d4..124d71f 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -1335,7 +1335,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname != 'my_locks'
@@ -1422,7 +1422,7 @@ from pg_locks l join pg_class c on l.relation = c.oid
where virtualtransaction = (
select virtualtransaction
from pg_locks
- where transactionid = txid_current()::integer)
+ where transactionid is not distinct from txid_convert_if_recent(txid_current()) )
and locktype = 'relation'
and relnamespace != (select oid from pg_namespace where nspname = 'pg_catalog')
and c.relname = 'my_locks'
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index bd6decf..cb22007 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -72,6 +72,19 @@ ROLLBACK;
BEGIN;
SELECT txid_current() AS inprogress \gset
+-- We can reasonably assume we haven't hit the first xid
+-- wraparound here, so:
+SELECT txid_convert_if_recent(:committed) = :'committed'::xid;
+SELECT txid_convert_if_recent(:rolledback) = :'rolledback'::xid;
+SELECT txid_convert_if_recent(:inprogress) = :'inprogress'::xid;
+SELECT txid_convert_if_recent(0) = '0'::xid; -- InvalidTransactionId
+SELECT txid_convert_if_recent(1) = '1'::xid; -- BootstrapTransactionId
+SELECT txid_convert_if_recent(2) = '2'::xid; -- FrozenTransactionId
+-- we ignore epoch for the fixed xids
+SELECT txid_convert_if_recent(BIGINT '1' << 32);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 1);
+SELECT txid_convert_if_recent((BIGINT '1' << 32) + 2);
+
SELECT txid_status(:committed) AS committed;
SELECT txid_status(:rolledback) AS rolledback;
SELECT txid_status(:inprogress) AS inprogress;
@@ -81,6 +94,44 @@ SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will alway
COMMIT;
+-- Check xids in the future
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent(txid_current() + (BIGINT '1' << 32));
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+DO
+$$
+BEGIN
+ PERFORM txid_convert_if_recent((BIGINT '1' << 32) - 1);
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'got expected xid out of range error';
+END;
+$$;
+
+BEGIN;
+CREATE FUNCTION test_future_xid(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_convert_if_recent($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid(:inprogress + 100000);
+ROLLBACK;
+
BEGIN;
CREATE FUNCTION test_future_xid_status(bigint)
RETURNS void
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Mon, Sep 19, 2016 at 9:54 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
You appear to have attached the wrong patch set.
Whoops, multitasking fail.
Sorry for the late response, hospitals are "fun".
I did some cleanup of 0001 (see attached) and was all set to commit it
when I realized what I think is a problem: holding XidGenLock doesn't
seem to help with the race condition between this function and CLOG
truncation, because vac_truncate_clog() updates the shared memory
limits AFTER performing the truncation. If the order of those
operations were reversed, we'd be fine, because it would get stuck
trying to update the shared memory limits and wouldn't be able to
truncate until it did - and conversely, if it updated the shared
memory limits before we examined them, that would be OK, too, because
we'd be sure not to consult the pages that are about to be truncated.
As it is, though, I don't see that there's any real interlock here.
(BTW, the stuff you moved from clog.c to clog.h doesn't actually need
to be moved; one of the changes I made here was to undo that.)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
txid-status-rmh.patchinvalid/octet-stream; name=txid-status-rmh.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 47fcb30..c51dca5 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..e40f6bc 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -28,6 +28,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +94,63 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestXid is protected by XidGenLock, but we don't
+ * acquire that lock here. Instead, we require the caller to acquire it,
+ * because the caller is presumably going to look up the returned XID.
+ * If we took and released the lock within this function, a CLOG
+ * truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(XidGenLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +412,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +719,60 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must hold XidGenLock here to prevent oldestXid advancing and
+ * triggering clog truncation between when we check that the xid is ok and
+ * when we look it up in the clog. Otherwise an exception might get thrown
+ * on clog access.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 20 September 2016 at 22:46, Robert Haas <robertmhaas@gmail.com> wrote:
I did some cleanup of 0001 (see attached) and was all set to commit it
when I realized what I think is a problem: holding XidGenLock doesn't
seem to help with the race condition between this function and CLOG
truncation, because vac_truncate_clog() updates the shared memory
limits AFTER performing the truncation.
Thanks ... and drat.
If the order of those
operations were reversed, we'd be fine, because it would get stuck
trying to update the shared memory limits and wouldn't be able to
truncate until it did - and conversely, if it updated the shared
memory limits before we examined them, that would be OK, too, because
we'd be sure not to consult the pages that are about to be truncated.
I'm hesitant to mess with something that fundamental for what I was
hoping was a low-impact feature, albeit one that seems to be trying
hard not to be at every turn. It looks pretty reasonable to update
oldestXid before clog truncation but I don't want to be wrong and
create some obscure race or crash recovery issue related to wraparound
and clog truncation. It could well be a problem if we're very close to
wraparound.
So far nothing has had any reason to care about this, since there's no
way to attempt to access an older xid in a normally functioning
system. The commit timestamp lookup code doesn't care whether the xid
is still in clog or not and nothing else does lookups based on xids
supplied by the user. If anything else did or does in future it will
have the same problem as txid_status.
We've already ruled out releasing the slru LWLock in SlruReportIOError
then PG_TRY / PG_CATCH()ing clog access errors in txid_status() per my
original approach to this issue.
So I see a few options now:
* Do nothing. Permit this race to exist, document the error, and
expect apps to cope. I'm pretty tempted to go for exactly this since
it pushes the cost onto users of the feature and doesn't make a mess
elsewhere. People who use this will typically be invoking it as a
standalone toplevel function anyway, so it's mostly just a bit of
noise in the error logs if you lose the race - and we have plenty of
other sources of that already.
* Rather than calling TransactionIdDidCommit / TransactionIdDidAbort,
call clog.c's TransactionIdGetStatus with a new missing_ok flag. That
sets a bool* missing param added to SimpleLruReadPage_ReadOnly(...)
and in turn to SimpleLruReadPage(...) that's set instead of calling
SlruReportIOError(...). This seems rather intrusive and will add
little-used params and paths to fairly hot slru.c code so I'm not
keen.
* Take CLogControlLock LW_SHARED in txid_status() to prevent
truncation before reading oldestXid. We'd need a way to pass an
"already locked" state through TransactionIdGetStatus(...) to
SimpleLruReadPage_ReadOnly(...), which isn't great since again it's
pretty hot code.
* Don't release the slru LWLock in SlruReportIOError; instead release
CLogControlLock from txid_status on clog access failure. As before
means PG_TRY / PG_CATCH without PG_RE_THROW, but it means the only
place it affects is callers of txid_status(...). For added safety,
restrict txid_status() to being called in a toplevel virtual xact so
we know we'll finish up promptly. It's a horrible layering violation
having txid_status(...) release the clog control lock though, and
seems risky.
The only non-horrible one of those is IMO to just let the caller see
an error if they lose the race. It's a function that's intended for
use when you're dealing with indeterminate transaction state after a
server or application error anyway, so it's part of an error path
already. So I say we just document the behaviour. Because slru.c
doesn't release its LWLock on error we also need to ensure
txid_status(...) is also only called from a toplevel xact so the user
doesn't attempt to wrap it in plpgsql BEGIN ... EXCEPTION block and it
causes the xact to abort.
Will follow up with just that.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Wed, Sep 21, 2016 at 3:40 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
The only non-horrible one of those is IMO to just let the caller see
an error if they lose the race. It's a function that's intended for
use when you're dealing with indeterminate transaction state after a
server or application error anyway, so it's part of an error path
already. So I say we just document the behaviour.
I am not keen on that idea. The errors we're likely to be exposing
are going to look like low-level internal failures, which might scare
some DBA. Even if they don't, I think it's playing with fire. The
system is designed on the assumption that nobody will try to look up
an XID that's too old, and if we start to violate that assumption I
think we're undermining the design integrity of the system in a way
we'll likely come to regret. To put that more plainly, when the code
is written with the assumption that X will never happen, it's usually
a bad idea to casually add code that does X.
Because slru.c
doesn't release its LWLock on error we also need to ensure
txid_status(...) is also only called from a toplevel xact so the user
doesn't attempt to wrap it in plpgsql BEGIN ... EXCEPTION block and it
causes the xact to abort.
I think this is muddled, because an error aborts the transaction, and
AbortTransaction() and AbortSubTransaction() start with
LWLockReleaseAll().
It might not be too hard to add a second copy of oldestXid in shared
memory that is updated before truncation rather than afterward... but
yeah, like you, I'm not finding this nearly as straightforward as
might have been hoped.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 21 September 2016 at 22:16, Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Sep 21, 2016 at 3:40 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
The only non-horrible one of those is IMO to just let the caller see
an error if they lose the race. It's a function that's intended for
use when you're dealing with indeterminate transaction state after a
server or application error anyway, so it's part of an error path
already. So I say we just document the behaviour.I am not keen on that idea. The errors we're likely to be exposing
are going to look like low-level internal failures, which might scare
some DBA. Even if they don't, I think it's playing with fire. The
system is designed on the assumption that nobody will try to look up
an XID that's too old, and if we start to violate that assumption I
think we're undermining the design integrity of the system in a way
we'll likely come to regret. To put that more plainly, when the code
is written with the assumption that X will never happen, it's usually
a bad idea to casually add code that does X.
Fair point.
[snip]
It might not be too hard to add a second copy of oldestXid in shared
memory that is updated before truncation rather than afterward... but
yeah, like you, I'm not finding this nearly as straightforward as
might have been hoped.
Yeah.
I suspect that'll be the way to go, to add another copy that's updated
before clog truncation. It just seems ... unclean. Like it shouldn't
be necessary, something else isn't right. But it's probably the lowest
pain option.
I'm going to take a step back on this and see if I can spot an alternative.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 27 September 2016 at 09:23, Craig Ringer <craig@2ndquadrant.com> wrote:
On 21 September 2016 at 22:16, Robert Haas <robertmhaas@gmail.com> wrote:
It might not be too hard to add a second copy of oldestXid in shared
memory that is updated before truncation rather than afterward... but
yeah, like you, I'm not finding this nearly as straightforward as
might have been hoped.Yeah.
I suspect that'll be the way to go, to add another copy that's updated
before clog truncation. It just seems ... unclean. Like it shouldn't
be necessary, something else isn't right. But it's probably the lowest
pain option.
OK, so summarizing the problem:
slru.c and clog.c have no soft-failure entrypoints where we can look
it up and fail gracefully if the xid isn't in the clog.
vac_truncate_clog() calls TruncateCLOG to chop SLRU pages from the
clog before it takes XidGenLock to advance oldestXid. So we cannot use
oldestXid protected by XidGenLock as an interlock against concurrent
clog truncation to prevent SLRU access errors looking up an xid, and
there's no other candidate lock.
This hasn't been an issue before because nobody looks up arbitrary
user-supplied XIDs in clog, we only look up things that're protected
by datfrozenxid etc. But the whole point of txid_status is to look up
user-supplied xids.
We can't just take ClogControlLock from txid_status() to block
truncation because clog.c expects to own that lock, and takes it (via
slru.c) in TransactionIdGetStatus, with no way to pass an
already_locked state. We'd self-deadlock.
Adding a second copy of oldestXid - say pendingOldestXid - won't
actually help us unless we also take some suitable LWLock before
updating it, otherwise truncation can continue after we look at
pendingOldestXid but before we do the clog lookup. That means an extra
LWLock for each clog truncation, but compared to the I/O done during
clog truncation and the cost of the SlruScanDirectory() pre-check done
by TruncateCLOG it's nothing.
We could take XidGenLock twice, once to update this new
pendingOldestXid field and once to update oldestXid, but that's a
highly contested lock I'd rather not mess with even on a path that's
not hit much.
Instead, I've added a new LWLock, ClogTruncationLock, for that
purpose. vac_truncate_clog() takes it if it decides to attempt clog
truncation. This lock is held throughout the whole process of clog
truncation and oldestXid advance, so there's no need for a new
pendingOldestXid field in ShmemVariableCache. We just take the lock
then look at oldestXid, knowing that it's guaranteed to correspond to
an existing clog page that won't go away while we're looking.
ClogTruncationLock is utterly uncontested so it's going to have no
meaningful impact compared to all the work we do scanning the clog to
decide whether we're even going to try truncating it, etc.
(BTW, it seems like a pity that lwlocknames.txt doesn't have comments
on each lock. We could have
src/backend/storage/lmgr/generate-lwlocknames.pl transform the #
comments into /* comments */ on the generated header. Thoughts?)
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 0fcfc84226afa4901277a43ee78ec34a30322e8f Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Wed, 21 Dec 2016 15:37:29 +0800
Subject: [PATCH 1/3] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you. This
hasn't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog is truncated before we advance oldestXid so
taking XidGenLock is insufficient, and there's no way to look up a
SLRU with soft-failure. So we introduce ClogTruncationLock to guard
against concurrent clog truncation.
---
doc/src/sgml/func.sgml | 31 +++++++
src/backend/access/transam/clog.c | 2 +
src/backend/access/transam/commit_ts.c | 2 +
src/backend/access/transam/multixact.c | 1 +
src/backend/commands/vacuum.c | 13 +++
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/utils/adt/txid.c | 141 +++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
11 files changed, 300 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 47fcb30..3123232 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..b35438e 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -644,6 +644,8 @@ TruncateCLOG(TransactionId oldestXact)
{
int cutoffPage;
+ Assert(LWLockHeldByMe(ClogTruncationLock));
+
/*
* The cutoff point is the start of the segment containing oldestXact. We
* pass the *page* containing oldestXact to SimpleLruTruncate.
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index a8d275f..6d71a15 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -805,6 +805,8 @@ TruncateCommitTs(TransactionId oldestXact)
{
int cutoffPage;
+ Assert(LWLockHeldByMe(ClogTruncationLock));
+
/*
* The cutoff point is the start of the segment containing oldestXact. We
* pass the *page* containing oldestXact to SimpleLruTruncate.
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index e9588a7..a2875e6 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -2937,6 +2937,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
Assert(!RecoveryInProgress());
Assert(MultiXactState->finishedStartup);
+ Assert(LWLockHeldByMe(ClogTruncationLock));
/*
* We can only allow one truncation to happen at once. Otherwise parts of
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 58bbf55..ddc486b 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1153,6 +1153,17 @@ vac_truncate_clog(TransactionId frozenXID,
return;
/*
+ * Because we advance oldestXid only after truncating the clog, holding
+ * XidGenLock is insufficient to be sure that a given xid will still exist
+ * in clog when we go to look up its state. Code that may look up arbitrary
+ * and possibly old xids should take ClogTruncationLock in shared mode to
+ * guard against concurrent clog truncation. If such code also takes
+ * XidGenLock it must be taken after ClogTruncationLog to guard against
+ * deadlocks.
+ */
+ LWLockAcquire(ClogTruncationLock, LW_EXCLUSIVE);
+
+ /*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
TruncateCLOG(frozenXID);
@@ -1168,6 +1179,8 @@ vac_truncate_clog(TransactionId frozenXID,
SetTransactionIdLimit(frozenXID, oldestxid_datoid);
SetMultiXactIdLimit(minMulti, minmulti_datoid);
AdvanceOldestCommitTsXid(frozenXID);
+
+ LWLockRelease(ClogTruncationLock);
}
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index f8996cd..7aebeeb 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -47,3 +47,4 @@ CommitTsLock 39
ReplicationOriginLock 40
MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
+ClogTruncationLock 43
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..7af0abb 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,77 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns true if the xid is
+ * within the xid wraparound threshold and clog truncation threshold
+ * or if the xid is a permanent xid. Returns false if the xid is past
+ * the clog truncation threshold or wraparound threshold. Sets
+ * extracted_xid to the 32-bit xid.
+ *
+ * It's only safe to use the extracted_xid for most purposes if the
+ * function returns true, otherwise the clog could be truncated away
+ * or it might be an xid from a prior epoch.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+ bool result;
+
+ Assert(LWLockHeldByMe(ClogTruncationLock));
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ result = true;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ result = false;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ result = false;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ return result;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +427,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +734,68 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We use oldestXid to determine whether the xid we're examining still has
+ * commit information retained. We must ensure the clog for the XIDs we're
+ * examining doesn't get truncated away while we're looking, otherwise we'll
+ * fail with confusing SLRU access errors. See vac_truncate_clog(..).
+ */
+ LWLockAcquire(ClogTruncationLock, LW_SHARED);
+ /*
+ * We should also hold XidGenLock to prevent oldestXid advancing, which can
+ * happen whether or not actual clog truncation is attempted. We don't want
+ * to try to look up state for an xid that becomes in-the-future after
+ * we check if it's in the recent past.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ LWLockRelease(ClogTruncationLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+ LWLockRelease(ClogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Wed, Dec 21, 2016 at 3:02 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
Instead, I've added a new LWLock, ClogTruncationLock, for that
purpose. vac_truncate_clog() takes it if it decides to attempt clog
truncation. This lock is held throughout the whole process of clog
truncation and oldestXid advance, so there's no need for a new
pendingOldestXid field in ShmemVariableCache. We just take the lock
then look at oldestXid, knowing that it's guaranteed to correspond to
an existing clog page that won't go away while we're looking.
ClogTruncationLock is utterly uncontested so it's going to have no
meaningful impact compared to all the work we do scanning the clog to
decide whether we're even going to try truncating it, etc.
That makes everything that happens between when we acquire that lock
and when we release it non-interruptible, which seems undesirable. I
think that extra copy of oldestXid is a nicer approach.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 December 2016 at 00:30, Robert Haas <robertmhaas@gmail.com> wrote:
That makes everything that happens between when we acquire that lock
and when we release it non-interruptible, which seems undesirable. I
think that extra copy of oldestXid is a nicer approach.
That's a side-effect I didn't realise. Given that, yes, I agree.
Since we don't truncate clog much, do you think it's reasonable to
just take XidGenLock again before we proceed? I'm reluctant to add
another acquisition of a frequently contested lock for something 99.9%
of the codebase won't care about, so I think it's probably better to
add a new LWLock, and I'll resubmit on that basis, but figure it's
worth asking.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 December 2016 at 07:49, Craig Ringer <craig@2ndquadrant.com> wrote:
On 22 December 2016 at 00:30, Robert Haas <robertmhaas@gmail.com> wrote:
That makes everything that happens between when we acquire that lock
and when we release it non-interruptible, which seems undesirable. I
think that extra copy of oldestXid is a nicer approach.That's a side-effect I didn't realise. Given that, yes, I agree.
Since we don't truncate clog much, do you think it's reasonable to
just take XidGenLock again before we proceed? I'm reluctant to add
another acquisition of a frequently contested lock for something 99.9%
of the codebase won't care about, so I think it's probably better to
add a new LWLock, and I'll resubmit on that basis, but figure it's
worth asking.
Updated.
If you think it's better to just take XidGenLock again, let me know.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From b10ee9a6abb2cc6867b4b78e91b819fe33ae0119 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Wed, 21 Dec 2016 15:37:29 +0800
Subject: [PATCH] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you. This
hasn't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog is truncated before we advance oldestXid so
taking XidGenLock is insufficient, and there's no way to look up a
SLRU with soft-failure. So we introduce ClogTruncationLock to guard
against concurrent clog truncation.
---
doc/src/sgml/func.sgml | 31 +++++++
src/backend/access/transam/varsup.c | 24 ++++++
src/backend/commands/vacuum.c | 3 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/utils/adt/txid.c | 141 +++++++++++++++++++++++++++++++
src/include/access/transam.h | 6 ++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
10 files changed, 315 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 47fcb30..3123232 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,28 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and/or database server crashed or
+ lost connection while a <literal>COMMIT</literal> command was in progress.
+ The status of a transaction will be reported as one of:
+ <itemizedlist>
+ <listitem><para><literal>'in progress'</></></>
+ <listitem><para><literal>'committed'</></></>
+ <listitem><para><literal>'aborted'</></></>
+ <listitem><para><literal>NULL</> if xid too old</></>
+ </itemizedlist>
+ PostgreSQL discards the commit status transactions after no references to
+ the transaction survive in other active transactions, tables, replication
+ slots, etc. This means that the status of older transactions cannot be
+ determined. <function>txid_status(bigint)</> returns <literal>NULL</> if a
+ transaction is too old to look up. Prepared transactions are reported as
+ <literal>in progress</>; applications must check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine if the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 2f7e645..120e1ce 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -258,6 +258,30 @@ ReadNewTransactionId(void)
return xid;
}
+
+/*
+ * Because vac_truncate_clog calls SetTransactionIdLimit to advance oldestXid
+ * only after truncating the clog, holding XidGenLock is insufficient to be
+ * sure that a given xid will still exist in clog when we go to look up its
+ * state.
+ *
+ * Here we advance pendingOldestXid before we truncate the clog, under
+ * ClogTrunationLock, so interested code can ensure that it never looks up
+ * clog/committs/multixact entries that may be about to be removed. It
+ * is guaranteed safe to look up any arbitrary xid >= pendingOldestXid
+ * while holding ClogTruncationLock.
+ *
+ * Code that takes both XidGenLock and ClogTruncationLock should take
+ * ClogTruncationLock first to prevent deadlock risk.
+ */
+void
+SetPendingTransactionIdLimit(TransactionId oldest_xid)
+{
+ LWLockAcquire(ClogTruncationLock, LW_EXCLUSIVE);
+ ShmemVariableCache->pendingOldestXid = oldest_xid;
+ LWLockRelease(ClogTruncationLock);
+}
+
/*
* Determine the last safe XID to allocate given the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 58bbf55..f2c3d87 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1152,6 +1152,9 @@ vac_truncate_clog(TransactionId frozenXID,
if (bogus)
return;
+ /* warn that we're about to truncate clog and advance oldestXid */
+ SetPendingTransactionIdLimit(frozenXID);
+
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index f8996cd..7aebeeb 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -47,3 +47,4 @@ CommitTsLock 39
ReplicationOriginLock 40
MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
+ClogTruncationLock 43
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..7af0abb 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,77 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns true if the xid is
+ * within the xid wraparound threshold and clog truncation threshold
+ * or if the xid is a permanent xid. Returns false if the xid is past
+ * the clog truncation threshold or wraparound threshold. Sets
+ * extracted_xid to the 32-bit xid.
+ *
+ * It's only safe to use the extracted_xid for most purposes if the
+ * function returns true, otherwise the clog could be truncated away
+ * or it might be an xid from a prior epoch.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+ bool result;
+
+ Assert(LWLockHeldByMe(ClogTruncationLock));
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ result = true;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ result = false;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ result = false;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ return result;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +427,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +734,68 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We use oldestXid to determine whether the xid we're examining still has
+ * commit information retained. We must ensure the clog for the XIDs we're
+ * examining doesn't get truncated away while we're looking, otherwise we'll
+ * fail with confusing SLRU access errors. See vac_truncate_clog(..).
+ */
+ LWLockAcquire(ClogTruncationLock, LW_SHARED);
+ /*
+ * We should also hold XidGenLock to prevent oldestXid advancing, which can
+ * happen whether or not actual clog truncation is attempted. We don't want
+ * to try to look up state for an xid that becomes in-the-future after
+ * we check if it's in the recent past.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ LWLockRelease(ClogTruncationLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ /*
+ * can't test TransactionIdIsInProgress here or we race
+ * with concurrent commit/abort. There's no point anyway,
+ * since it might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+ LWLockRelease(ClogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 969eff9..7302c5d 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -134,6 +134,11 @@ typedef struct VariableCacheData
*/
TransactionId latestCompletedXid; /* newest XID that has committed or
* aborted */
+
+ /*
+ * These fields are protected by ClogTruncationLock
+ */
+ TransactionId pendingOldestXid; /* oldest xid it's safe to look up in clog */
} VariableCacheData;
typedef VariableCacheData *VariableCache;
@@ -171,6 +176,7 @@ extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
+extern void SetPendingTransactionIdLimit(TransactionId oldest_xid);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
Oid oldest_datoid);
extern bool ForceTransactionIdLimitUpdate(void);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 December 2016 at 09:55, Craig Ringer <craig@2ndquadrant.com> wrote:
Updated.
If you think it's better to just take XidGenLock again, let me know.
Here's a further update that merges in Robert's changes from the patch
posted upthread.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From bc0fea43200ae861e2b8562f5e9f81fa73d207f9 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Thu, 22 Dec 2016 13:07:00 +0800
Subject: [PATCH] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you. This
hasn't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog is truncated before we advance oldestXid so
taking XidGenLock is insufficient, and there's no way to look up a
SLRU with soft-failure. So we introduce a new pendingOldestXid member
of ShmemVariableCache and advance it under the new ClogTruncationLock
LWLock before beginning clog truncation.
Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 +++++
src/backend/access/transam/varsup.c | 24 ++++
src/backend/commands/vacuum.c | 3 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/utils/adt/txid.c | 197 +++++++++++++++++++++++++++++++
src/include/access/transam.h | 6 +
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++
src/test/regress/sql/txid.sql | 38 ++++++
10 files changed, 367 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 47fcb30..c51dca5 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17143,6 +17143,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17193,6 +17197,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17263,6 +17272,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 2f7e645..120e1ce 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -258,6 +258,30 @@ ReadNewTransactionId(void)
return xid;
}
+
+/*
+ * Because vac_truncate_clog calls SetTransactionIdLimit to advance oldestXid
+ * only after truncating the clog, holding XidGenLock is insufficient to be
+ * sure that a given xid will still exist in clog when we go to look up its
+ * state.
+ *
+ * Here we advance pendingOldestXid before we truncate the clog, under
+ * ClogTrunationLock, so interested code can ensure that it never looks up
+ * clog/committs/multixact entries that may be about to be removed. It
+ * is guaranteed safe to look up any arbitrary xid >= pendingOldestXid
+ * while holding ClogTruncationLock.
+ *
+ * Code that takes both XidGenLock and ClogTruncationLock should take
+ * ClogTruncationLock first to prevent deadlock risk.
+ */
+void
+SetPendingTransactionIdLimit(TransactionId oldest_xid)
+{
+ LWLockAcquire(ClogTruncationLock, LW_EXCLUSIVE);
+ ShmemVariableCache->pendingOldestXid = oldest_xid;
+ LWLockRelease(ClogTruncationLock);
+}
+
/*
* Determine the last safe XID to allocate given the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 58bbf55..f2c3d87 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1152,6 +1152,9 @@ vac_truncate_clog(TransactionId frozenXID,
if (bogus)
return;
+ /* warn that we're about to truncate clog and advance oldestXid */
+ SetPendingTransactionIdLimit(frozenXID);
+
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index f8996cd..7aebeeb 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -47,3 +47,4 @@ CommitTsLock 39
ReplicationOriginLock 40
MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
+ClogTruncationLock 43
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..53a6f61 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,133 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+<<<<<<< Updated upstream
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+=======
+ * Helper to get a TransactionId from a 64-bit txid with wraparound
+ * detection.
+ *
+ * ERRORs if the txid is in the future. Returns true if the xid is
+ * within the xid wraparound threshold and clog truncation threshold
+ * or if the xid is a permanent xid. Returns false if the xid is past
+ * the clog truncation threshold or wraparound threshold. Sets
+ * extracted_xid to the 32-bit xid.
+ *
+ * It's only safe to use the extracted_xid for most purposes if the
+ * function returns true, otherwise the clog could be truncated away
+ * or it might be an xid from a prior epoch.
+ *
+ * XIDs older than ShmemVariableCache->oldestXid are treated as too
+ * old to look up because the clog could've been truncated away - even
+ * if they're still far from the xid wraparound theshold. The caller
+ * should have at least a share lock on XidGenLock to prevent
+ * oldestXid from advancing between our oldestXid check and subsequent
+ * lookups of transaction status using the returned xid. Failure to do
+ * so risks ERRORs on clog access but nothing worse.
+>>>>>>> Stashed changes
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+<<<<<<< Updated upstream
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestXid is protected by XidGenLock, but we don't
+ * acquire that lock here. Instead, we require the caller to acquire it,
+ * because the caller is presumably going to look up the returned XID.
+ * If we took and released the lock within this function, a CLOG
+ * truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(XidGenLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ return false;
+
+ return true;
+=======
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+ bool result;
+
+ Assert(LWLockHeldByMe(ClogTruncationLock));
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ result = true;
+
+ if (!TransactionIdIsNormal(xid))
+ {
+ /* must be a permanent XID, ignore the epoch and return unchanged */
+ }
+ else if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID "UINT64_FORMAT" is in the future",
+ xid_with_epoch)));
+ }
+ else if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid))
+ {
+ /* xid is wrapped, too far in the past */
+ result = false;
+ }
+ else if (TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ {
+ /* xid isn't wrapped, but clog could've been truncated away */
+ result = false;
+ }
+ else
+ {
+ Assert(TransactionIdPrecedesOrEquals(xid, now_epoch_last_xid));
+ }
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ return result;
+>>>>>>> Stashed changes
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +483,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +790,68 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We use oldestXid to determine whether the xid we're examining still has
+ * commit information retained. We must ensure the clog for the XIDs we're
+ * examining doesn't get truncated away while we're looking, otherwise we'll
+ * fail with confusing SLRU access errors. See vac_truncate_clog(..).
+ */
+ LWLockAcquire(ClogTruncationLock, LW_SHARED);
+ /*
+ * We should also hold XidGenLock to prevent oldestXid advancing, which can
+ * happen whether or not actual clog truncation is attempted. We don't want
+ * to try to look up state for an xid that becomes in-the-future after
+ * we check if it's in the recent past.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+
+ LWLockRelease(XidGenLock);
+ LWLockRelease(ClogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 969eff9..7302c5d 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -134,6 +134,11 @@ typedef struct VariableCacheData
*/
TransactionId latestCompletedXid; /* newest XID that has committed or
* aborted */
+
+ /*
+ * These fields are protected by ClogTruncationLock
+ */
+ TransactionId pendingOldestXid; /* oldest xid it's safe to look up in clog */
} VariableCacheData;
typedef VariableCacheData *VariableCache;
@@ -171,6 +176,7 @@ extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
+extern void SetPendingTransactionIdLimit(TransactionId oldest_xid);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
Oid oldest_datoid);
extern bool ForceTransactionIdLimitUpdate(void);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..0ad870c 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4928,6 +4928,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..baffa38 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Thu, Dec 22, 2016 at 12:12 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 22 December 2016 at 09:55, Craig Ringer <craig@2ndquadrant.com> wrote:
Updated.
If you think it's better to just take XidGenLock again, let me know.
Here's a further update that merges in Robert's changes from the patch
posted upthread.
This patch contains unresolved merge conflicts. Maybe
SetPendingTransactionIdLimit could happen in TruncateCLOG rather than
the caller. Instead of using an LWLock, how about a spin lock?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 December 2016 at 01:26, Robert Haas <robertmhaas@gmail.com> wrote:
This patch contains unresolved merge conflicts.
Ah, it conflicts with fe0a0b59, the PostmasterRandom changes. I
thought I'd rebased more recently than that.
Maybe SetPendingTransactionIdLimit could happen in TruncateCLOG rather than
the caller.
I've spent a while looking at how committs and multixact handle the
race hazard of the gap between clog truncation and shmem state change
- and the related race on standby, where the hazard is the gap between
replay of CLOG_TRUNCATE records and replay of the next checkpoint
containing updated limits.
pg_xact_commit_timestamp() and the underlying
TransactionIdGetCommitTsData are supposed to accept arbitrary xids,
like txid_status(), so they should handle this.
However, as far as I can tell, committs has the same issues presented
by txid_status(). We only update ShmemVariableCache->oldestCommitTsXid
_after_ we truncate the committs SLRU, no lock is held over the
duration, and no other lock-protected var is set before truncation. We
don't record the new limit values in COMMIT_TS_TRUNCATE records so
they only becomes known to the standby at replay of the next
checkpoint.
multixact instead runs a single critical section to write xlog, set
shmem state, and _then_ truncate the SLRUs. It's more complex than
clog or commit_ts since it has two inter-related SLRUs, though.
So: I think we should be setting ShmemVariableCache->oldestXid (under
XidGenLock) from TruncateCLOG, after we write xlog but _before_
truncation. Then the rest of SetTransactionIdLimit(...) proceeds as
before. I don't _think_ we need the critical section since we only
have the one SLRU to worry about. We should also add oldestXid to
CLOG_TRUNCATE xlog records so it is up to date on standbys.
It's a little annoying to take XidGenLock twice - once for oldestXid,
once for the other xid limits - but we can just delay advancing
oldestXid entirely until we've got a clog page to truncate away, in
which case there's a big enough cost that it doesn't matter. We have
to wait to advance the other limits until after we truncate clog to
prevent new (wrapped) xids trying to set values in clog for the
before-wrap xids we're about to truncate away.
On the upside, the need for pendingOldestXid, which always felt hacky,
goes away.
While we're at it, lets do the same thing for commit_ts. Advance its
limit before truncation, not after, and record that limit in its xlog
records for redo. There's no need for any special dancing around
there.
Instead of using an LWLock, how about a spin lock?
I wrote an explanation of how that wouldn't work, but then I found the
problem with standby, so it no longer matters.
I'll have to follow up with a patch soon, as it's Toddler o'Clock.
(It's interesting that most of what I'm doing at the moment is
wrestling with resource retention limits and how they're often quite
implicit. All the catalog_xmin stuff for logical decoding on standby
has parallels to this, for example, though not enough to create useful
overlap of functionality.)
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 December 2016 at 18:00, Craig Ringer <craig@2ndquadrant.com> wrote:
I'll have to follow up with a patch soon, as it's Toddler o'Clock.
Here we go.
This patch advances oldestXid, under XidGenLock, _before_ truncating clog.
txid_status() holds XidGenLock from when it tests oldestXid until it's
done looking up clog, thus eliminating the race.
CLOG_TRUNCATE records now contain the oldestXid, so they can advance
oldestXid on a standby, or when we've truncated clog since the most
recent checkpoint on the master during recovery. It's advanced under
XidGenLock during redo to protect against this race on standby.
As outlined in my prior mail I think this is the right approach. I
don't like taking XidGenLock twice, but we don't advance datfrozenxid
much so it's not a big concern. While a separate ClogTruncationLock
could be added like in my earlier patch, oldestXid is currently under
XidGenLock and I'd rather not change that.
The biggest change here is that oldestXid is advanced separately to
the vac limits in the rest of ShmemVariableCache. As far as I can tell
we don't prevent two manual VACUUMs on different DBs from trying to
concurrently run vac_truncate_clog, so this has to be safe against two
invocations racing each other. Rather than try to lock out such
concurrency, the patch ensures that oldestXid can never go backwards.
It doesn't really matter if the vac limits go backwards, it's no worse
than what can already happen in the current code.
We cannot advance the vacuum limits before we truncate the clog away,
in case someone tries to access a very new xid (if we're near
wraparound)
I'm pretty sure that commit timestamps suffer from the same flaw as
Robert identified upthread with clog. This patch fixes the clog race,
but not the similar one in commit timestamps. Unlike the clog race
with txid_status(), the commit timestamps one is already potentially
user-visible since we allow arbitrary xids to be looked up for commit
timestamps. I'll address that separately.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From a61d9b81c82f3241e2fc47caf261b6f7bedf82d9 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Thu, 22 Dec 2016 13:07:00 +0800
Subject: [PATCH] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you. This
hasn't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog was truncated before we advance oldestXid so
taking XidGenLock was insufficient. There's no way to look up a
SLRU with soft-failure. To address this, increase oldestXid under XidGenLock
before we trunate clog rather than after, so concurrent access is safe.
Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++++
src/backend/access/rmgrdesc/clogdesc.c | 12 +++-
src/backend/access/transam/clog.c | 33 +++++++---
src/backend/access/transam/varsup.c | 38 ++++++++++-
src/backend/access/transam/xlog.c | 17 +++--
src/backend/commands/vacuum.c | 13 ++++
src/backend/utils/adt/txid.c | 116 +++++++++++++++++++++++++++++++++
src/include/access/clog.h | 5 ++
src/include/access/transam.h | 2 +
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 1 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++
13 files changed, 355 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 10e3186..8b28dc6 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17182,6 +17182,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17232,6 +17236,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17302,6 +17311,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c
index 41ea254..44b150d 100644
--- a/src/backend/access/rmgrdesc/clogdesc.c
+++ b/src/backend/access/rmgrdesc/clogdesc.c
@@ -23,12 +23,20 @@ clog_desc(StringInfo buf, XLogReaderState *record)
char *rec = XLogRecGetData(record);
uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
- if (info == CLOG_ZEROPAGE || info == CLOG_TRUNCATE)
+ if (info == CLOG_ZEROPAGE)
{
int pageno;
memcpy(&pageno, rec, sizeof(int));
- appendStringInfo(buf, "%d", pageno);
+ appendStringInfo(buf, "page %d", pageno);
+ }
+ else if (info == CLOG_TRUNCATE)
+ {
+ xl_clog_truncate xlrec;
+
+ memcpy(&xlrec, rec, sizeof(xl_clog_truncate));
+ appendStringInfo(buf, "page %d; oldestXact %u",
+ xlrec.pageno, xlrec.oldestXact);
}
}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 2634476..9c9f43b 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -83,7 +83,7 @@ static SlruCtlData ClogCtlData;
static int ZeroCLOGPage(int pageno, bool writeXlog);
static bool CLOGPagePrecedes(int page1, int page2);
static void WriteZeroPageXlogRec(int pageno);
-static void WriteTruncateXlogRec(int pageno);
+static void WriteTruncateXlogRec(int pageno, TransactionId oldestXact);
static void TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status,
XLogRecPtr lsn, int pageno);
@@ -654,8 +654,17 @@ TruncateCLOG(TransactionId oldestXact)
if (!SlruScanDirectory(ClogCtl, SlruScanDirCbReportPresence, &cutoffPage))
return; /* nothing to remove */
- /* Write XLOG record and flush XLOG to disk */
- WriteTruncateXlogRec(cutoffPage);
+ /* vac_truncate_clog already advanced oldestXid */
+ Assert(TransactionIdPrecedesOrEquals(oldestXact,
+ ShmemVariableCache->oldestXid));
+
+ /*
+ * Write XLOG record and flush XLOG to disk. We record the oldest xid we're
+ * keeping information about here so we can ensure that it's always ahead
+ * of clog truncation in case we crash, and so a standby finds out the new
+ * valid xid before the next checkpoint.
+ */
+ WriteTruncateXlogRec(cutoffPage, oldestXact);
/* Now we can remove the old CLOG segment(s) */
SimpleLruTruncate(ClogCtl, cutoffPage);
@@ -704,12 +713,16 @@ WriteZeroPageXlogRec(int pageno)
* in TruncateCLOG().
*/
static void
-WriteTruncateXlogRec(int pageno)
+WriteTruncateXlogRec(int pageno, TransactionId oldestXact)
{
XLogRecPtr recptr;
+ xl_clog_truncate xlrec;
+
+ xlrec.pageno = pageno;
+ xlrec.oldestXact = oldestXact;
XLogBeginInsert();
- XLogRegisterData((char *) (&pageno), sizeof(int));
+ XLogRegisterData((char *) (&xlrec), sizeof(xl_clog_truncate));
recptr = XLogInsert(RM_CLOG_ID, CLOG_TRUNCATE);
XLogFlush(recptr);
}
@@ -742,17 +755,19 @@ clog_redo(XLogReaderState *record)
}
else if (info == CLOG_TRUNCATE)
{
- int pageno;
+ xl_clog_truncate xlrec;
- memcpy(&pageno, XLogRecGetData(record), sizeof(int));
+ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_clog_truncate));
/*
* During XLOG replay, latest_page_number isn't set up yet; insert a
* suitable value to bypass the sanity test in SimpleLruTruncate.
*/
- ClogCtl->shared->latest_page_number = pageno;
+ ClogCtl->shared->latest_page_number = xlrec.pageno;
+
+ AdvanceOldestXid(xlrec.oldestXact);
- SimpleLruTruncate(ClogCtl, pageno);
+ SimpleLruTruncate(ClogCtl, xlrec.pageno);
}
else
elog(PANIC, "clog_redo: unknown op code %u", info);
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 2f7e645..ca67572 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -259,6 +259,27 @@ ReadNewTransactionId(void)
}
/*
+ * Advance the cluster-wide oldestXid.
+ *
+ * We must ensure that this never goes backwards, otherwise the xid limits set
+ * in SetTransactionIdLimit(...) could be insufficiently conservative if two
+ * vacuums race, with the lower oldestXmin winning then the higher xid limits
+ * winning.
+ *
+ * The reverse is safe and just means we fail to advance our xid limits until
+ * the next vacuum.
+ */
+void
+AdvanceOldestXid(TransactionId oldest_datfrozenxid)
+{
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+ if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
+ oldest_datfrozenxid))
+ ShmemVariableCache->oldestXid = oldest_datfrozenxid;
+ LWLockRelease(XidGenLock);
+}
+
+/*
* Determine the last safe XID to allocate given the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* of our cluster), and the OID of the (or a) database with that value.
@@ -330,9 +351,22 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
if (xidVacLimit < FirstNormalTransactionId)
xidVacLimit += FirstNormalTransactionId;
- /* Grab lock for just long enough to set the new limit values */
+ /*
+ * Grab lock for just long enough to set the new limit values.
+ *
+ * If we're called by vac_truncate_clog, a concurrent vacuum of another
+ * database might've advanced oldestXid between when our caller advanced it
+ * and when we're called to advance the vacuum limits. This is harmless;
+ * our limits will be based on the lower oldestXmin and thus more
+ * conservative.
+ *
+ * It's unsafe to proceed if we calculated limits based on a greater
+ * oldestXmin than is currently in effect. All calls to
+ * SetTransactionIdLimit must be preceded by a call to AdvanceOldestXid.
+ */
LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
- ShmemVariableCache->oldestXid = oldest_datfrozenxid;
+ Assert(TransactionIdFollowsOrEquals(ShmemVariableCache->oldestXid,
+ oldest_datfrozenxid));
ShmemVariableCache->xidVacLimit = xidVacLimit;
ShmemVariableCache->xidWarnLimit = xidWarnLimit;
ShmemVariableCache->xidStopLimit = xidStopLimit;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f8ffa5c..115a115 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4851,6 +4851,7 @@ BootStrapXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(InvalidTransactionId, InvalidTransactionId);
@@ -6445,6 +6446,7 @@ StartupXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(checkPoint.oldestCommitTsXid,
@@ -9430,6 +9432,10 @@ xlog_redo(XLogReaderState *record)
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
+ /*
+ * No need to call AdvanceOldestXid, startup or an earlier clog trunate
+ * record will have already advanced it. Just advance the limits.
+ */
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
/*
@@ -9527,10 +9533,13 @@ xlog_redo(XLogReaderState *record)
*/
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
- if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
- checkPoint.oldestXid))
- SetTransactionIdLimit(checkPoint.oldestXid,
- checkPoint.oldestXidDB);
+ /*
+ * We don't need to AdvanceOldestXid here; StartupXLOG or a clog
+ * truncation record will ensure it's up to date, and we just update
+ * the corresponding xid limits here.
+ */
+ SetTransactionIdLimit(checkPoint.oldestXid,
+ checkPoint.oldestXidDB);
/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
ControlFile->checkPointCopy.nextXidEpoch = checkPoint.nextXidEpoch;
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index b1be2f7..d3fa9e5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1153,6 +1153,19 @@ vac_truncate_clog(TransactionId frozenXID,
return;
/*
+ * Advance oldestXid _before_ truncating clog, so concurrent xact status
+ * lookups can ensure they don't attempt to access truncated-away clog.
+ *
+ * We must do this even if we find we can't actually truncate away any clog
+ * pages, since we'll advance the xid limits and need oldestXmin to be
+ * consistent with the new limits.
+ *
+ * Losing this on crash before a checkpoint is harmless unless we truncated
+ * clog, in which case redo of the clog truncation will re-apply it.
+ */
+ AdvanceOldestXid(frozenXID);
+
+ /*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
TruncateCLOG(frozenXID);
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 276075e..ac48d1b 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,63 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestXid is protected by XidGenLock, but we don't
+ * acquire that lock here. Instead, we require the caller to acquire it,
+ * because the caller is presumably going to look up the returned XID.
+ * If we took and released the lock within this function, a CLOG
+ * truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(XidGenLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +413,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +720,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * Ensure clog isn't removed between when we check whether the current xid
+ * is valid and when we look its status up.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 06c069a..c7f2d0b 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,11 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+typedef struct xl_clog_truncate
+{
+ int pageno;
+ TransactionId oldestXact;
+} xl_clog_truncate;
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 969eff9..1bd553e 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -171,6 +171,8 @@ extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
+extern void SetPendingTransactionIdLimit(TransactionId oldest_xid);
+extern void AdvanceOldestXid(TransactionId oldest_datfrozenxid);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
Oid oldest_datoid);
extern bool ForceTransactionIdLimitUpdate(void);
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index cd7b909..fd269bf 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4929,6 +4929,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3346 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 7ed1623..e25b514 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1227,6 +1227,7 @@ extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
/* uuid.c */
extern Datum uuid_in(PG_FUNCTION_ARGS);
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 28 December 2016 at 18:00, Craig Ringer <craig@2ndquadrant.com> wrote:
On 23 December 2016 at 18:00, Craig Ringer <craig@2ndquadrant.com> wrote:
I'll have to follow up with a patch soon, as it's Toddler o'Clock.
Here we go.
This patch advances oldestXid, under XidGenLock, _before_ truncating clog.
txid_status() holds XidGenLock from when it tests oldestXid until it's
done looking up clog, thus eliminating the race.CLOG_TRUNCATE records now contain the oldestXid, so they can advance
oldestXid on a standby, or when we've truncated clog since the most
recent checkpoint on the master during recovery. It's advanced under
XidGenLock during redo to protect against this race on standby.As outlined in my prior mail I think this is the right approach. I
don't like taking XidGenLock twice, but we don't advance datfrozenxid
much so it's not a big concern. While a separate ClogTruncationLock
could be added like in my earlier patch, oldestXid is currently under
XidGenLock and I'd rather not change that.The biggest change here is that oldestXid is advanced separately to
the vac limits in the rest of ShmemVariableCache. As far as I can tell
we don't prevent two manual VACUUMs on different DBs from trying to
concurrently run vac_truncate_clog, so this has to be safe against two
invocations racing each other. Rather than try to lock out such
concurrency, the patch ensures that oldestXid can never go backwards.
It doesn't really matter if the vac limits go backwards, it's no worse
than what can already happen in the current code.We cannot advance the vacuum limits before we truncate the clog away,
in case someone tries to access a very new xid (if we're near
wraparound)I'm pretty sure that commit timestamps suffer from the same flaw as
Robert identified upthread with clog. This patch fixes the clog race,
but not the similar one in commit timestamps. Unlike the clog race
with txid_status(), the commit timestamps one is already potentially
user-visible since we allow arbitrary xids to be looked up for commit
timestamps. I'll address that separately.
Rebased patch attached. I've split the clog changes out from
txid_status() its self.
There is relevant discussion on the commit timestamp truncation fix
thread where the similar fix for commit_ts got committed.
/messages/by-id/979ff13d-0b8e-4937-01e8-2925c0adc306@2ndquadrant.com
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Fix-race-between-clog-truncation-and-lookup.patchtext/x-patch; charset=US-ASCII; name=0001-Fix-race-between-clog-truncation-and-lookup.patchDownload
From f8e5e89145fee7afa9c629c80a3d578fc31e21b4 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:25:30 +0800
Subject: [PATCH 1/2] Fix race between clog truncation and lookup
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you.
This hasn 't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog was truncated before we advanced oldestXid under
XidGenLock, so holding XidGenLock during a clog lookup was insufficient to
prevent the race. There's no way to look up a SLRU with soft-failure;
attempting a lookup produces an I/O error. There's also no safe way to trap and
swallow the SLRU lookup error due mainly to locking issues.
To address this, increase oldestXid under XidGenLock before we trunate clog
rather than after, so concurrent lookups of arbitrary XIDs are safe if they are
done under XidGenLock. The rest of the xid limits are still advanced after clog
truncation to ensure there's no chance of a new xid trying to access an
about-to-be-truncated clog page. In practice this can't happen anyway since we
use only half the xid space at any time, but additional guards against future
change are warranted with something this crucial.
This race also exists in a worse form on standby servers. On a standby we only
advance oldestXid when we replay the next checkpoint, so there's a much larger
window between clog truncation and subsequent updating of the limit. Fix this
by recording the oldest xid in clog truncation records and applying the
oldestXid under XidGenLock before replaying the clog truncation.
Note that There's no need to take XidGenLock for normal clog lookups protected
by datfrozenxid, only if accepting arbitrary XIDs that might not be protected by
vacuum thresholds.
---
src/backend/access/rmgrdesc/clogdesc.c | 12 +++++++++--
src/backend/access/transam/clog.c | 33 +++++++++++++++++++++--------
src/backend/access/transam/varsup.c | 38 ++++++++++++++++++++++++++++++++--
src/backend/access/transam/xlog.c | 17 +++++++++++----
src/backend/commands/vacuum.c | 13 ++++++++++++
src/include/access/clog.h | 5 +++++
src/include/access/transam.h | 2 ++
7 files changed, 103 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c
index 352de48..ef268c5 100644
--- a/src/backend/access/rmgrdesc/clogdesc.c
+++ b/src/backend/access/rmgrdesc/clogdesc.c
@@ -23,12 +23,20 @@ clog_desc(StringInfo buf, XLogReaderState *record)
char *rec = XLogRecGetData(record);
uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
- if (info == CLOG_ZEROPAGE || info == CLOG_TRUNCATE)
+ if (info == CLOG_ZEROPAGE)
{
int pageno;
memcpy(&pageno, rec, sizeof(int));
- appendStringInfo(buf, "%d", pageno);
+ appendStringInfo(buf, "page %d", pageno);
+ }
+ else if (info == CLOG_TRUNCATE)
+ {
+ xl_clog_truncate xlrec;
+
+ memcpy(&xlrec, rec, sizeof(xl_clog_truncate));
+ appendStringInfo(buf, "page %d; oldestXact %u",
+ xlrec.pageno, xlrec.oldestXact);
}
}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 1a43819..37ad88c 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -83,7 +83,7 @@ static SlruCtlData ClogCtlData;
static int ZeroCLOGPage(int pageno, bool writeXlog);
static bool CLOGPagePrecedes(int page1, int page2);
static void WriteZeroPageXlogRec(int pageno);
-static void WriteTruncateXlogRec(int pageno);
+static void WriteTruncateXlogRec(int pageno, TransactionId oldestXact);
static void TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status,
XLogRecPtr lsn, int pageno);
@@ -654,8 +654,17 @@ TruncateCLOG(TransactionId oldestXact)
if (!SlruScanDirectory(ClogCtl, SlruScanDirCbReportPresence, &cutoffPage))
return; /* nothing to remove */
- /* Write XLOG record and flush XLOG to disk */
- WriteTruncateXlogRec(cutoffPage);
+ /* vac_truncate_clog already advanced oldestXid */
+ Assert(TransactionIdPrecedesOrEquals(oldestXact,
+ ShmemVariableCache->oldestXid));
+
+ /*
+ * Write XLOG record and flush XLOG to disk. We record the oldest xid we're
+ * keeping information about here so we can ensure that it's always ahead
+ * of clog truncation in case we crash, and so a standby finds out the new
+ * valid xid before the next checkpoint.
+ */
+ WriteTruncateXlogRec(cutoffPage, oldestXact);
/* Now we can remove the old CLOG segment(s) */
SimpleLruTruncate(ClogCtl, cutoffPage);
@@ -704,12 +713,16 @@ WriteZeroPageXlogRec(int pageno)
* in TruncateCLOG().
*/
static void
-WriteTruncateXlogRec(int pageno)
+WriteTruncateXlogRec(int pageno, TransactionId oldestXact)
{
XLogRecPtr recptr;
+ xl_clog_truncate xlrec;
+
+ xlrec.pageno = pageno;
+ xlrec.oldestXact = oldestXact;
XLogBeginInsert();
- XLogRegisterData((char *) (&pageno), sizeof(int));
+ XLogRegisterData((char *) (&xlrec), sizeof(xl_clog_truncate));
recptr = XLogInsert(RM_CLOG_ID, CLOG_TRUNCATE);
XLogFlush(recptr);
}
@@ -742,17 +755,19 @@ clog_redo(XLogReaderState *record)
}
else if (info == CLOG_TRUNCATE)
{
- int pageno;
+ xl_clog_truncate xlrec;
- memcpy(&pageno, XLogRecGetData(record), sizeof(int));
+ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_clog_truncate));
/*
* During XLOG replay, latest_page_number isn't set up yet; insert a
* suitable value to bypass the sanity test in SimpleLruTruncate.
*/
- ClogCtl->shared->latest_page_number = pageno;
+ ClogCtl->shared->latest_page_number = xlrec.pageno;
+
+ AdvanceOldestXid(xlrec.oldestXact);
- SimpleLruTruncate(ClogCtl, pageno);
+ SimpleLruTruncate(ClogCtl, xlrec.pageno);
}
else
elog(PANIC, "clog_redo: unknown op code %u", info);
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index fc084c5..809805a 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -259,6 +259,27 @@ ReadNewTransactionId(void)
}
/*
+ * Advance the cluster-wide oldestXid.
+ *
+ * We must ensure that this never goes backwards, otherwise the xid limits set
+ * in SetTransactionIdLimit(...) could be insufficiently conservative if two
+ * vacuums race, with the lower oldestXmin winning then the higher xid limits
+ * winning.
+ *
+ * The reverse is safe and just means we fail to advance our xid limits until
+ * the next vacuum.
+ */
+void
+AdvanceOldestXid(TransactionId oldest_datfrozenxid)
+{
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+ if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
+ oldest_datfrozenxid))
+ ShmemVariableCache->oldestXid = oldest_datfrozenxid;
+ LWLockRelease(XidGenLock);
+}
+
+/*
* Determine the last safe XID to allocate given the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* of our cluster), and the OID of the (or a) database with that value.
@@ -330,9 +351,22 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
if (xidVacLimit < FirstNormalTransactionId)
xidVacLimit += FirstNormalTransactionId;
- /* Grab lock for just long enough to set the new limit values */
+ /*
+ * Grab lock for just long enough to set the new limit values.
+ *
+ * If we're called by vac_truncate_clog, a concurrent vacuum of another
+ * database might've advanced oldestXid between when our caller advanced it
+ * and when we're called to advance the vacuum limits. This is harmless;
+ * our limits will be based on the lower oldestXmin and thus more
+ * conservative.
+ *
+ * It's unsafe to proceed if we calculated limits based on a greater
+ * oldestXmin than is currently in effect. All calls to
+ * SetTransactionIdLimit must be preceded by a call to AdvanceOldestXid.
+ */
LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
- ShmemVariableCache->oldestXid = oldest_datfrozenxid;
+ Assert(TransactionIdFollowsOrEquals(ShmemVariableCache->oldestXid,
+ oldest_datfrozenxid));
ShmemVariableCache->xidVacLimit = xidVacLimit;
ShmemVariableCache->xidWarnLimit = xidWarnLimit;
ShmemVariableCache->xidStopLimit = xidStopLimit;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2f5d603..ddf65e6 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4877,6 +4877,7 @@ BootStrapXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(InvalidTransactionId, InvalidTransactionId);
@@ -6471,6 +6472,7 @@ StartupXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(checkPoint.oldestCommitTsXid,
@@ -9456,6 +9458,10 @@ xlog_redo(XLogReaderState *record)
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
+ /*
+ * No need to call AdvanceOldestXid, startup or an earlier clog trunate
+ * record will have already advanced it. Just advance the limits.
+ */
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
/*
@@ -9553,10 +9559,13 @@ xlog_redo(XLogReaderState *record)
*/
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
- if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
- checkPoint.oldestXid))
- SetTransactionIdLimit(checkPoint.oldestXid,
- checkPoint.oldestXidDB);
+ /*
+ * We don't need to AdvanceOldestXid here; StartupXLOG or a clog
+ * truncation record will ensure it's up to date, and we just update
+ * the corresponding xid limits here.
+ */
+ SetTransactionIdLimit(checkPoint.oldestXid,
+ checkPoint.oldestXidDB);
/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
ControlFile->checkPointCopy.nextXidEpoch = checkPoint.nextXidEpoch;
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 812fb4a..055650b 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1162,6 +1162,19 @@ vac_truncate_clog(TransactionId frozenXID,
AdvanceOldestCommitTsXid(frozenXID);
/*
+ * Advance oldestXid before truncating clog, so concurrent xact status
+ * lookups can ensure they don't attempt to access truncated-away clog.
+ *
+ * We must do this even if we find we can't actually truncate away any clog
+ * pages, since we'll advance the xid limits and need oldestXmin to be
+ * consistent with the new limits.
+ *
+ * Losing this on crash before a checkpoint is harmless unless we truncated
+ * clog, in which case redo of the clog truncation will re-apply it.
+ */
+ AdvanceOldestXid(frozenXID);
+
+ /*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
TruncateCLOG(frozenXID);
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 2894bd5..4e0a3f2 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,11 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+typedef struct xl_clog_truncate
+{
+ int pageno;
+ TransactionId oldestXact;
+} xl_clog_truncate;
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 522c104..ad0ef79 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -171,6 +171,8 @@ extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
+extern void SetPendingTransactionIdLimit(TransactionId oldest_xid);
+extern void AdvanceOldestXid(TransactionId oldest_datfrozenxid);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
Oid oldest_datoid);
extern bool ForceTransactionIdLimitUpdate(void);
--
2.5.5
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 725bca72e5d6ade0dbd29dfcf566a25be97de903 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH 2/2] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++
src/backend/utils/adt/txid.c | 116 ++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 164 +++++++++++++++++++++++++++++++++++++
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
6 files changed, 415 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b214218..0095915 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17190,6 +17190,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17240,6 +17244,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17310,6 +17319,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..8f86916 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,63 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestXid is protected by XidGenLock, but we don't
+ * acquire that lock here. Instead, we require the caller to acquire it,
+ * because the caller is presumably going to look up the returned XID.
+ * If we took and released the lock within this function, a CLOG
+ * truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(XidGenLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +413,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +720,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * Ensure clog isn't removed between when we check whether the current xid
+ * is valid and when we look its status up.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index ab12761..912cb5a 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4936,6 +4936,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 5bdca82..26f76ca 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -119,5 +119,169 @@ extern int32 type_maximum_size(Oid type_oid, int32 typemod);
/* quote.c */
extern char *quote_literal_cstr(const char *rawstr);
+extern Datum quote_nullable(PG_FUNCTION_ARGS);
+
+/* guc.c */
+extern Datum show_config_by_name(PG_FUNCTION_ARGS);
+extern Datum show_config_by_name_missing_ok(PG_FUNCTION_ARGS);
+extern Datum set_config_by_name(PG_FUNCTION_ARGS);
+extern Datum show_all_settings(PG_FUNCTION_ARGS);
+extern Datum show_all_file_settings(PG_FUNCTION_ARGS);
+
+/* pg_config.c */
+extern Datum pg_config(PG_FUNCTION_ARGS);
+
+/* pg_controldata.c */
+extern Datum pg_control_checkpoint(PG_FUNCTION_ARGS);
+extern Datum pg_control_system(PG_FUNCTION_ARGS);
+extern Datum pg_control_init(PG_FUNCTION_ARGS);
+extern Datum pg_control_recovery(PG_FUNCTION_ARGS);
+
+/* rls.c */
+extern Datum row_security_active(PG_FUNCTION_ARGS);
+extern Datum row_security_active_name(PG_FUNCTION_ARGS);
+
+/* lockfuncs.c */
+extern Datum pg_lock_status(PG_FUNCTION_ARGS);
+extern Datum pg_blocking_pids(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_all(PG_FUNCTION_ARGS);
+
+/* txid.c */
+extern Datum txid_snapshot_in(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_out(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_recv(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_send(PG_FUNCTION_ARGS);
+extern Datum txid_current(PG_FUNCTION_ARGS);
+extern Datum txid_current_if_assigned(PG_FUNCTION_ARGS);
+extern Datum txid_current_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
+extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
+
+/* uuid.c */
+extern Datum uuid_in(PG_FUNCTION_ARGS);
+extern Datum uuid_out(PG_FUNCTION_ARGS);
+extern Datum uuid_send(PG_FUNCTION_ARGS);
+extern Datum uuid_recv(PG_FUNCTION_ARGS);
+extern Datum uuid_lt(PG_FUNCTION_ARGS);
+extern Datum uuid_le(PG_FUNCTION_ARGS);
+extern Datum uuid_eq(PG_FUNCTION_ARGS);
+extern Datum uuid_ge(PG_FUNCTION_ARGS);
+extern Datum uuid_gt(PG_FUNCTION_ARGS);
+extern Datum uuid_ne(PG_FUNCTION_ARGS);
+extern Datum uuid_cmp(PG_FUNCTION_ARGS);
+extern Datum uuid_sortsupport(PG_FUNCTION_ARGS);
+extern Datum uuid_hash(PG_FUNCTION_ARGS);
+
+/* windowfuncs.c */
+extern Datum window_row_number(PG_FUNCTION_ARGS);
+extern Datum window_rank(PG_FUNCTION_ARGS);
+extern Datum window_dense_rank(PG_FUNCTION_ARGS);
+extern Datum window_percent_rank(PG_FUNCTION_ARGS);
+extern Datum window_cume_dist(PG_FUNCTION_ARGS);
+extern Datum window_ntile(PG_FUNCTION_ARGS);
+extern Datum window_lag(PG_FUNCTION_ARGS);
+extern Datum window_lag_with_offset(PG_FUNCTION_ARGS);
+extern Datum window_lag_with_offset_and_default(PG_FUNCTION_ARGS);
+extern Datum window_lead(PG_FUNCTION_ARGS);
+extern Datum window_lead_with_offset(PG_FUNCTION_ARGS);
+extern Datum window_lead_with_offset_and_default(PG_FUNCTION_ARGS);
+extern Datum window_first_value(PG_FUNCTION_ARGS);
+extern Datum window_last_value(PG_FUNCTION_ARGS);
+extern Datum window_nth_value(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgquadtreeproc.c */
+extern Datum spg_quad_config(PG_FUNCTION_ARGS);
+extern Datum spg_quad_choose(PG_FUNCTION_ARGS);
+extern Datum spg_quad_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_quad_inner_consistent(PG_FUNCTION_ARGS);
+extern Datum spg_quad_leaf_consistent(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgkdtreeproc.c */
+extern Datum spg_kd_config(PG_FUNCTION_ARGS);
+extern Datum spg_kd_choose(PG_FUNCTION_ARGS);
+extern Datum spg_kd_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_kd_inner_consistent(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgtextproc.c */
+extern Datum spg_text_config(PG_FUNCTION_ARGS);
+extern Datum spg_text_choose(PG_FUNCTION_ARGS);
+extern Datum spg_text_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_text_inner_consistent(PG_FUNCTION_ARGS);
+extern Datum spg_text_leaf_consistent(PG_FUNCTION_ARGS);
+
+/* access/gin/ginarrayproc.c */
+extern Datum ginarrayextract(PG_FUNCTION_ARGS);
+extern Datum ginarrayextract_2args(PG_FUNCTION_ARGS);
+extern Datum ginqueryarrayextract(PG_FUNCTION_ARGS);
+extern Datum ginarrayconsistent(PG_FUNCTION_ARGS);
+extern Datum ginarraytriconsistent(PG_FUNCTION_ARGS);
+
+/* access/tablesample/bernoulli.c */
+extern Datum tsm_bernoulli_handler(PG_FUNCTION_ARGS);
+
+/* access/tablesample/system.c */
+extern Datum tsm_system_handler(PG_FUNCTION_ARGS);
+
+/* access/transam/twophase.c */
+extern Datum pg_prepared_xact(PG_FUNCTION_ARGS);
+
+/* access/transam/multixact.c */
+extern Datum pg_get_multixact_members(PG_FUNCTION_ARGS);
+
+/* access/transam/committs.c */
+extern Datum pg_xact_commit_timestamp(PG_FUNCTION_ARGS);
+extern Datum pg_last_committed_xact(PG_FUNCTION_ARGS);
+
+/* catalogs/dependency.c */
+extern Datum pg_describe_object(PG_FUNCTION_ARGS);
+extern Datum pg_identify_object(PG_FUNCTION_ARGS);
+extern Datum pg_identify_object_as_address(PG_FUNCTION_ARGS);
+
+/* catalog/objectaddress.c */
+extern Datum pg_get_object_address(PG_FUNCTION_ARGS);
+
+/* commands/constraint.c */
+extern Datum unique_key_recheck(PG_FUNCTION_ARGS);
+
+/* commands/event_trigger.c */
+extern Datum pg_event_trigger_dropped_objects(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_table_rewrite_oid(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_table_rewrite_reason(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS);
+
+/* commands/extension.c */
+extern Datum pg_available_extensions(PG_FUNCTION_ARGS);
+extern Datum pg_available_extension_versions(PG_FUNCTION_ARGS);
+extern Datum pg_extension_update_paths(PG_FUNCTION_ARGS);
+extern Datum pg_extension_config_dump(PG_FUNCTION_ARGS);
+
+/* commands/prepare.c */
+extern Datum pg_prepared_statement(PG_FUNCTION_ARGS);
+
+/* utils/mmgr/portalmem.c */
+extern Datum pg_cursor(PG_FUNCTION_ARGS);
#endif /* BUILTINS_H */
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Mon, Jan 23, 2017 at 1:32 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
Rebased patch attached. I've split the clog changes out from
txid_status() its self.
I think it's fairly surprising that TruncateCLOG() has responsibility
for writing the xlog record that protects advancing
ShmemVariableCache->oldestXid, but not the responsibility for actually
advancing that value. In other words, I think the AdvanceOldestXid()
call in vac_truncate_clog() should be moved into TruncateClog().
(Similarly, one wonders why AdvanceOldestCommitTsXid() isn't the
responsibility of TruncateCommitTs().)
I think it is not correct to advance oldestXid but not oldestXidDB.
Otherwise, GetNewTransactionId() might complain about the wrong
database.
The way that SetTransactionIdLimit() now works looks a bit dangerous.
xidWrapLimit, xidStopLimit, and xidWarnLimit are computed based on the
passed-in oldestXid value and written straight into shared memory.
But the shared memory copy of oldestXid could have a different value.
I'm not sure if that breaks anything, but it certainly weakens any
confidence callers might have had that all those values are consistent
with each other.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 24 January 2017 at 23:49, Robert Haas <robertmhaas@gmail.com> wrote:
I think it's fairly surprising that TruncateCLOG() has responsibility
for writing the xlog record that protects advancing
ShmemVariableCache->oldestXid, but not the responsibility for actually
advancing that value. In other words, I think the AdvanceOldestXid()
call in vac_truncate_clog() should be moved into TruncateClog().
(Similarly, one wonders why AdvanceOldestCommitTsXid() isn't the
responsibility of TruncateCommitTs().)
Agreed, and done in attached.
I haven't done the same for commit_ts but will do so on the thread for
the commit_ts race fix if we go ahead with this change here.
I think it is not correct to advance oldestXid but not oldestXidDB.
Otherwise, GetNewTransactionId() might complain about the wrong
database.
That's a clear oversight. Fixed in attached.
The way that SetTransactionIdLimit() now works looks a bit dangerous.
xidWrapLimit, xidStopLimit, and xidWarnLimit are computed based on the
passed-in oldestXid value and written straight into shared memory.
But the shared memory copy of oldestXid could have a different value.
I'm not sure if that breaks anything, but it certainly weakens any
confidence callers might have had that all those values are consistent
with each other.
This was my main hesitation with the whole thing too.
It's necessary to advance oldestXmin before we xlog the advance and
truncate clog, and necessary to advance the vacuum limits only
afterwards.
I thought it sensible to be conservative about the xid limit to
minimise the change from current behaviour, i.e. advance only up to
xid limits calculated from the oldestXid determined by the currently
vac_truncate_clog. But the current one is actually wrong; in the
(unlikely if it's possible at all) case where two vac_truncate_clog()s
A and B run with ordering A(advanceOldestXmin) B(advanceOldestXmin)
A(truncate) B(truncate) B(advanceLimits) A(advanceLimits), A's advance
of the limits would clobber b's. Not too bad, but it'd delay
advancing the limits until the next vacuum, and some xid could get
allocated before the limits go backwards.
It's safer to take XidGenLock for slightly longer in order to capture
the oldestXid from shmem and calculate using it. That ensures we never
have any risk of going backwards. The attached updated patch does so.
BTW, I find it quite amusing that this was intended to be a small,
unintrusive patch, and now it's messing with xid limits and clog
truncation. I'm almost tempted to say we should commit it with the
(tiny) race with clog truncation in place. We certainly haven't had
any complaints about the same race from people using commit_ts. But
the race window on standby is quite large and I'd rather not knowingly
introduce new bugs.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Fix-race-between-clog-truncation-and-lookup.patchtext/x-patch; charset=US-ASCII; name=0001-Fix-race-between-clog-truncation-and-lookup.patchDownload
From ca892924e859cdac455fd175e1361c5e08ebee47 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:25:30 +0800
Subject: [PATCH 1/2] Fix race between clog truncation and lookup
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you.
This hasn 't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog was truncated before we advanced oldestXid under
XidGenLock, so holding XidGenLock during a clog lookup was insufficient to
prevent the race. There's no way to look up a SLRU with soft-failure;
attempting a lookup produces an I/O error. There's also no safe way to trap and
swallow the SLRU lookup error due mainly to locking issues.
To address this, increase oldestXid under XidGenLock before we trunate clog
rather than after, so concurrent lookups of arbitrary XIDs are safe if they are
done under XidGenLock. The rest of the xid limits are still advanced after clog
truncation to ensure there's no chance of a new xid trying to access an
about-to-be-truncated clog page. In practice this can't happen anyway since we
use only half the xid space at any time, but additional guards against future
change are warranted with something this crucial.
This race also exists in a worse form on standby servers. On a standby we only
advance oldestXid when we replay the next checkpoint, so there's a much larger
window between clog truncation and subsequent updating of the limit. Fix this
by recording the oldest xid in clog truncation records and applying the
oldestXid under XidGenLock before replaying the clog truncation.
Note that There's no need to take XidGenLock for normal clog lookups protected
by datfrozenxid, only if accepting arbitrary XIDs that might not be protected by
vacuum thresholds.
---
src/backend/access/rmgrdesc/clogdesc.c | 12 ++++++--
src/backend/access/transam/clog.c | 54 +++++++++++++++++++++++++++-------
src/backend/access/transam/varsup.c | 49 +++++++++++++++++++++++-------
src/backend/access/transam/xlog.c | 22 +++++++++-----
src/backend/commands/vacuum.c | 4 +--
src/include/access/clog.h | 7 ++++-
src/include/access/transam.h | 4 +--
7 files changed, 118 insertions(+), 34 deletions(-)
diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c
index 352de48..ef268c5 100644
--- a/src/backend/access/rmgrdesc/clogdesc.c
+++ b/src/backend/access/rmgrdesc/clogdesc.c
@@ -23,12 +23,20 @@ clog_desc(StringInfo buf, XLogReaderState *record)
char *rec = XLogRecGetData(record);
uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
- if (info == CLOG_ZEROPAGE || info == CLOG_TRUNCATE)
+ if (info == CLOG_ZEROPAGE)
{
int pageno;
memcpy(&pageno, rec, sizeof(int));
- appendStringInfo(buf, "%d", pageno);
+ appendStringInfo(buf, "page %d", pageno);
+ }
+ else if (info == CLOG_TRUNCATE)
+ {
+ xl_clog_truncate xlrec;
+
+ memcpy(&xlrec, rec, sizeof(xl_clog_truncate));
+ appendStringInfo(buf, "page %d; oldestXact %u",
+ xlrec.pageno, xlrec.oldestXact);
}
}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 1a43819..a0b48a7 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -83,7 +83,7 @@ static SlruCtlData ClogCtlData;
static int ZeroCLOGPage(int pageno, bool writeXlog);
static bool CLOGPagePrecedes(int page1, int page2);
static void WriteZeroPageXlogRec(int pageno);
-static void WriteTruncateXlogRec(int pageno);
+static void WriteTruncateXlogRec(int pageno, TransactionId oldestXact);
static void TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status,
XLogRecPtr lsn, int pageno);
@@ -640,11 +640,24 @@ ExtendCLOG(TransactionId newestXact)
* the XLOG flush unless we have confirmed that there is a removable segment.
*/
void
-TruncateCLOG(TransactionId oldestXact)
+TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid)
{
int cutoffPage;
/*
+ * Advance oldestXid before truncating clog, so concurrent xact status
+ * lookups can ensure they don't attempt to access truncated-away clog.
+ *
+ * We must do this even if we find we can't actually truncate away any clog
+ * pages, since we'll advance the xid limits and need oldestXmin to be
+ * consistent with the new limits.
+ *
+ * Losing this on crash before a checkpoint is harmless unless we truncated
+ * clog, in which case redo of the clog truncation will re-apply it.
+ */
+ AdvanceOldestXid(oldestXact, oldestxid_datoid);
+
+ /*
* The cutoff point is the start of the segment containing oldestXact. We
* pass the *page* containing oldestXact to SimpleLruTruncate.
*/
@@ -654,8 +667,17 @@ TruncateCLOG(TransactionId oldestXact)
if (!SlruScanDirectory(ClogCtl, SlruScanDirCbReportPresence, &cutoffPage))
return; /* nothing to remove */
- /* Write XLOG record and flush XLOG to disk */
- WriteTruncateXlogRec(cutoffPage);
+ /* vac_truncate_clog already advanced oldestXid */
+ Assert(TransactionIdPrecedesOrEquals(oldestXact,
+ ShmemVariableCache->oldestXid));
+
+ /*
+ * Write XLOG record and flush XLOG to disk. We record the oldest xid we're
+ * keeping information about here so we can ensure that it's always ahead
+ * of clog truncation in case we crash, and so a standby finds out the new
+ * valid xid before the next checkpoint.
+ */
+ WriteTruncateXlogRec(cutoffPage, oldestXact);
/* Now we can remove the old CLOG segment(s) */
SimpleLruTruncate(ClogCtl, cutoffPage);
@@ -704,12 +726,16 @@ WriteZeroPageXlogRec(int pageno)
* in TruncateCLOG().
*/
static void
-WriteTruncateXlogRec(int pageno)
+WriteTruncateXlogRec(int pageno, TransactionId oldestXact)
{
XLogRecPtr recptr;
+ xl_clog_truncate xlrec;
+
+ xlrec.pageno = pageno;
+ xlrec.oldestXact = oldestXact;
XLogBeginInsert();
- XLogRegisterData((char *) (&pageno), sizeof(int));
+ XLogRegisterData((char *) (&xlrec), sizeof(xl_clog_truncate));
recptr = XLogInsert(RM_CLOG_ID, CLOG_TRUNCATE);
XLogFlush(recptr);
}
@@ -742,17 +768,25 @@ clog_redo(XLogReaderState *record)
}
else if (info == CLOG_TRUNCATE)
{
- int pageno;
+ xl_clog_truncate xlrec;
- memcpy(&pageno, XLogRecGetData(record), sizeof(int));
+ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_clog_truncate));
/*
* During XLOG replay, latest_page_number isn't set up yet; insert a
* suitable value to bypass the sanity test in SimpleLruTruncate.
*/
- ClogCtl->shared->latest_page_number = pageno;
+ ClogCtl->shared->latest_page_number = xlrec.pageno;
+
+ /*
+ * Make sure we know the oldest valid xid in clog on the standby.
+ *
+ * We don't bother keeping track of oldestXidDB, as we don't emit
+ * vacuum-related errors on a standby.
+ */
+ AdvanceOldestXid(xlrec.oldestXact, InvalidOid);
- SimpleLruTruncate(ClogCtl, pageno);
+ SimpleLruTruncate(ClogCtl, xlrec.pageno);
}
else
elog(PANIC, "clog_redo: unknown op code %u", info);
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index fc084c5..9aacb0f 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -259,19 +259,52 @@ ReadNewTransactionId(void)
}
/*
- * Determine the last safe XID to allocate given the currently oldest
+ * Advance the cluster-wide oldestXid.
+ *
+ * We must ensure that this never goes backwards, otherwise the xid limits set
+ * in SetTransactionIdLimit(...) could be insufficiently conservative if two
+ * vacuums race, with the lower oldestXmin winning then the higher xid limits
+ * winning.
+ */
+void
+AdvanceOldestXid(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
+{
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+ if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
+ oldest_datfrozenxid))
+ {
+ ShmemVariableCache->oldestXid = oldest_datfrozenxid;
+ ShmemVariableCache->oldestXidDB = oldest_datoid;
+ }
+ LWLockRelease(XidGenLock);
+}
+
+/*
+ * Determine the last safe XID to allocate using the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* of our cluster), and the OID of the (or a) database with that value.
*/
void
-SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
+SetTransactionIdLimit(void)
{
TransactionId xidVacLimit;
TransactionId xidWarnLimit;
TransactionId xidStopLimit;
TransactionId xidWrapLimit;
TransactionId curXid;
+ Oid oldest_datoid;
+ TransactionId oldest_datfrozenxid;
+
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+ /*
+ * We fetch oldest_datfrozenxid and oldest_datoid from shmem to ensure that
+ * if some other vacuum has advanced oldestXid since we did, we use the
+ * latest values and the limits never go backwards.
+ */
+ oldest_datfrozenxid = ShmemVariableCache->oldestXid;
+ oldest_datoid = ShmemVariableCache->oldestXidDB;
+
Assert(TransactionIdIsNormal(oldest_datfrozenxid));
/*
@@ -284,6 +317,7 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1);
if (xidWrapLimit < FirstNormalTransactionId)
xidWrapLimit += FirstNormalTransactionId;
+ ShmemVariableCache->xidWrapLimit = xidWrapLimit;
/*
* We'll refuse to continue assigning XIDs in interactive mode once we get
@@ -296,6 +330,7 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidStopLimit = xidWrapLimit - 1000000;
if (xidStopLimit < FirstNormalTransactionId)
xidStopLimit -= FirstNormalTransactionId;
+ ShmemVariableCache->xidStopLimit = xidStopLimit;
/*
* We'll start complaining loudly when we get within 10M transactions of
@@ -310,6 +345,7 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidWarnLimit = xidStopLimit - 10000000;
if (xidWarnLimit < FirstNormalTransactionId)
xidWarnLimit -= FirstNormalTransactionId;
+ ShmemVariableCache->xidWarnLimit = xidWarnLimit;
/*
* We'll start trying to force autovacuums when oldest_datfrozenxid gets
@@ -329,15 +365,8 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidVacLimit = oldest_datfrozenxid + autovacuum_freeze_max_age;
if (xidVacLimit < FirstNormalTransactionId)
xidVacLimit += FirstNormalTransactionId;
-
- /* Grab lock for just long enough to set the new limit values */
- LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
- ShmemVariableCache->oldestXid = oldest_datfrozenxid;
ShmemVariableCache->xidVacLimit = xidVacLimit;
- ShmemVariableCache->xidWarnLimit = xidWarnLimit;
- ShmemVariableCache->xidStopLimit = xidStopLimit;
- ShmemVariableCache->xidWrapLimit = xidWrapLimit;
- ShmemVariableCache->oldestXidDB = oldest_datoid;
+
curXid = ShmemVariableCache->nextXid;
LWLockRelease(XidGenLock);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2f5d603..823f4bd 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4877,7 +4877,8 @@ BootStrapXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
- SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ AdvanceOldestXid(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ SetTransactionIdLimit();
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(InvalidTransactionId, InvalidTransactionId);
@@ -6471,7 +6472,8 @@ StartupXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
- SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ AdvanceOldestXid(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ SetTransactionIdLimit();
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(checkPoint.oldestCommitTsXid,
checkPoint.newestCommitTsXid);
@@ -9456,7 +9458,11 @@ xlog_redo(XLogReaderState *record)
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
- SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ /*
+ * No need to call AdvanceOldestXid, startup or an earlier clog trunate
+ * record will have already advanced it. Just advance the limits.
+ */
+ SetTransactionIdLimit();
/*
* If we see a shutdown checkpoint while waiting for an end-of-backup
@@ -9553,10 +9559,12 @@ xlog_redo(XLogReaderState *record)
*/
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
- if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
- checkPoint.oldestXid))
- SetTransactionIdLimit(checkPoint.oldestXid,
- checkPoint.oldestXidDB);
+ /*
+ * We don't need to AdvanceOldestXid here; StartupXLOG or a clog
+ * truncation record will ensure it's up to date, and we just update
+ * the limits here based on what's already in shmem.
+ */
+ SetTransactionIdLimit();
/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
ControlFile->checkPointCopy.nextXidEpoch = checkPoint.nextXidEpoch;
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 812fb4a..5ec307f 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1164,7 +1164,7 @@ vac_truncate_clog(TransactionId frozenXID,
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
- TruncateCLOG(frozenXID);
+ TruncateCLOG(frozenXID, oldestxid_datoid);
TruncateCommitTs(frozenXID);
TruncateMultiXact(minMulti, minmulti_datoid);
@@ -1174,7 +1174,7 @@ vac_truncate_clog(TransactionId frozenXID,
* for an(other) autovac cycle if needed. XXX should we avoid possibly
* signalling twice?
*/
- SetTransactionIdLimit(frozenXID, oldestxid_datoid);
+ SetTransactionIdLimit();
SetMultiXactIdLimit(minMulti, minmulti_datoid);
}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 2894bd5..167cadb 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,11 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+typedef struct xl_clog_truncate
+{
+ int pageno;
+ TransactionId oldestXact;
+} xl_clog_truncate;
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
@@ -42,7 +47,7 @@ extern void TrimCLOG(void);
extern void ShutdownCLOG(void);
extern void CheckPointCLOG(void);
extern void ExtendCLOG(TransactionId newestXact);
-extern void TruncateCLOG(TransactionId oldestXact);
+extern void TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid);
/* XLOG stuff */
#define CLOG_ZEROPAGE 0x00
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 522c104..95cdef1 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -171,8 +171,8 @@ extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
-extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
- Oid oldest_datoid);
+extern void AdvanceOldestXid(TransactionId oldest_datfrozenxid, Oid oldest_datoid);
+extern void SetTransactionIdLimit(void);
extern bool ForceTransactionIdLimitUpdate(void);
extern Oid GetNewObjectId(void);
--
2.5.5
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From fbbc2481b3dea0d780bd3ef4d096055c5327b768 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH 2/2] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++
src/backend/utils/adt/txid.c | 116 ++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 164 +++++++++++++++++++++++++++++++++++++
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
6 files changed, 415 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b214218..0095915 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17190,6 +17190,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17240,6 +17244,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17310,6 +17319,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..8f86916 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,63 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestXid is protected by XidGenLock, but we don't
+ * acquire that lock here. Instead, we require the caller to acquire it,
+ * because the caller is presumably going to look up the returned XID.
+ * If we took and released the lock within this function, a CLOG
+ * truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(XidGenLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +413,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +720,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * Ensure clog isn't removed between when we check whether the current xid
+ * is valid and when we look its status up.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index ab12761..912cb5a 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4936,6 +4936,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 5bdca82..26f76ca 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -119,5 +119,169 @@ extern int32 type_maximum_size(Oid type_oid, int32 typemod);
/* quote.c */
extern char *quote_literal_cstr(const char *rawstr);
+extern Datum quote_nullable(PG_FUNCTION_ARGS);
+
+/* guc.c */
+extern Datum show_config_by_name(PG_FUNCTION_ARGS);
+extern Datum show_config_by_name_missing_ok(PG_FUNCTION_ARGS);
+extern Datum set_config_by_name(PG_FUNCTION_ARGS);
+extern Datum show_all_settings(PG_FUNCTION_ARGS);
+extern Datum show_all_file_settings(PG_FUNCTION_ARGS);
+
+/* pg_config.c */
+extern Datum pg_config(PG_FUNCTION_ARGS);
+
+/* pg_controldata.c */
+extern Datum pg_control_checkpoint(PG_FUNCTION_ARGS);
+extern Datum pg_control_system(PG_FUNCTION_ARGS);
+extern Datum pg_control_init(PG_FUNCTION_ARGS);
+extern Datum pg_control_recovery(PG_FUNCTION_ARGS);
+
+/* rls.c */
+extern Datum row_security_active(PG_FUNCTION_ARGS);
+extern Datum row_security_active_name(PG_FUNCTION_ARGS);
+
+/* lockfuncs.c */
+extern Datum pg_lock_status(PG_FUNCTION_ARGS);
+extern Datum pg_blocking_pids(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_all(PG_FUNCTION_ARGS);
+
+/* txid.c */
+extern Datum txid_snapshot_in(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_out(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_recv(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_send(PG_FUNCTION_ARGS);
+extern Datum txid_current(PG_FUNCTION_ARGS);
+extern Datum txid_current_if_assigned(PG_FUNCTION_ARGS);
+extern Datum txid_current_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
+extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
+
+/* uuid.c */
+extern Datum uuid_in(PG_FUNCTION_ARGS);
+extern Datum uuid_out(PG_FUNCTION_ARGS);
+extern Datum uuid_send(PG_FUNCTION_ARGS);
+extern Datum uuid_recv(PG_FUNCTION_ARGS);
+extern Datum uuid_lt(PG_FUNCTION_ARGS);
+extern Datum uuid_le(PG_FUNCTION_ARGS);
+extern Datum uuid_eq(PG_FUNCTION_ARGS);
+extern Datum uuid_ge(PG_FUNCTION_ARGS);
+extern Datum uuid_gt(PG_FUNCTION_ARGS);
+extern Datum uuid_ne(PG_FUNCTION_ARGS);
+extern Datum uuid_cmp(PG_FUNCTION_ARGS);
+extern Datum uuid_sortsupport(PG_FUNCTION_ARGS);
+extern Datum uuid_hash(PG_FUNCTION_ARGS);
+
+/* windowfuncs.c */
+extern Datum window_row_number(PG_FUNCTION_ARGS);
+extern Datum window_rank(PG_FUNCTION_ARGS);
+extern Datum window_dense_rank(PG_FUNCTION_ARGS);
+extern Datum window_percent_rank(PG_FUNCTION_ARGS);
+extern Datum window_cume_dist(PG_FUNCTION_ARGS);
+extern Datum window_ntile(PG_FUNCTION_ARGS);
+extern Datum window_lag(PG_FUNCTION_ARGS);
+extern Datum window_lag_with_offset(PG_FUNCTION_ARGS);
+extern Datum window_lag_with_offset_and_default(PG_FUNCTION_ARGS);
+extern Datum window_lead(PG_FUNCTION_ARGS);
+extern Datum window_lead_with_offset(PG_FUNCTION_ARGS);
+extern Datum window_lead_with_offset_and_default(PG_FUNCTION_ARGS);
+extern Datum window_first_value(PG_FUNCTION_ARGS);
+extern Datum window_last_value(PG_FUNCTION_ARGS);
+extern Datum window_nth_value(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgquadtreeproc.c */
+extern Datum spg_quad_config(PG_FUNCTION_ARGS);
+extern Datum spg_quad_choose(PG_FUNCTION_ARGS);
+extern Datum spg_quad_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_quad_inner_consistent(PG_FUNCTION_ARGS);
+extern Datum spg_quad_leaf_consistent(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgkdtreeproc.c */
+extern Datum spg_kd_config(PG_FUNCTION_ARGS);
+extern Datum spg_kd_choose(PG_FUNCTION_ARGS);
+extern Datum spg_kd_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_kd_inner_consistent(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgtextproc.c */
+extern Datum spg_text_config(PG_FUNCTION_ARGS);
+extern Datum spg_text_choose(PG_FUNCTION_ARGS);
+extern Datum spg_text_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_text_inner_consistent(PG_FUNCTION_ARGS);
+extern Datum spg_text_leaf_consistent(PG_FUNCTION_ARGS);
+
+/* access/gin/ginarrayproc.c */
+extern Datum ginarrayextract(PG_FUNCTION_ARGS);
+extern Datum ginarrayextract_2args(PG_FUNCTION_ARGS);
+extern Datum ginqueryarrayextract(PG_FUNCTION_ARGS);
+extern Datum ginarrayconsistent(PG_FUNCTION_ARGS);
+extern Datum ginarraytriconsistent(PG_FUNCTION_ARGS);
+
+/* access/tablesample/bernoulli.c */
+extern Datum tsm_bernoulli_handler(PG_FUNCTION_ARGS);
+
+/* access/tablesample/system.c */
+extern Datum tsm_system_handler(PG_FUNCTION_ARGS);
+
+/* access/transam/twophase.c */
+extern Datum pg_prepared_xact(PG_FUNCTION_ARGS);
+
+/* access/transam/multixact.c */
+extern Datum pg_get_multixact_members(PG_FUNCTION_ARGS);
+
+/* access/transam/committs.c */
+extern Datum pg_xact_commit_timestamp(PG_FUNCTION_ARGS);
+extern Datum pg_last_committed_xact(PG_FUNCTION_ARGS);
+
+/* catalogs/dependency.c */
+extern Datum pg_describe_object(PG_FUNCTION_ARGS);
+extern Datum pg_identify_object(PG_FUNCTION_ARGS);
+extern Datum pg_identify_object_as_address(PG_FUNCTION_ARGS);
+
+/* catalog/objectaddress.c */
+extern Datum pg_get_object_address(PG_FUNCTION_ARGS);
+
+/* commands/constraint.c */
+extern Datum unique_key_recheck(PG_FUNCTION_ARGS);
+
+/* commands/event_trigger.c */
+extern Datum pg_event_trigger_dropped_objects(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_table_rewrite_oid(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_table_rewrite_reason(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS);
+
+/* commands/extension.c */
+extern Datum pg_available_extensions(PG_FUNCTION_ARGS);
+extern Datum pg_available_extension_versions(PG_FUNCTION_ARGS);
+extern Datum pg_extension_update_paths(PG_FUNCTION_ARGS);
+extern Datum pg_extension_config_dump(PG_FUNCTION_ARGS);
+
+/* commands/prepare.c */
+extern Datum pg_prepared_statement(PG_FUNCTION_ARGS);
+
+/* utils/mmgr/portalmem.c */
+extern Datum pg_cursor(PG_FUNCTION_ARGS);
#endif /* BUILTINS_H */
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 25 January 2017 at 13:44, Craig Ringer <craig@2ndquadrant.com> wrote:
On 24 January 2017 at 23:49, Robert Haas <robertmhaas@gmail.com> wrote:
I think it's fairly surprising that TruncateCLOG() has responsibility
for writing the xlog record that protects advancing
ShmemVariableCache->oldestXid, but not the responsibility for actually
advancing that value. In other words, I think the AdvanceOldestXid()
call in vac_truncate_clog() should be moved into TruncateClog().
(Similarly, one wonders why AdvanceOldestCommitTsXid() isn't the
responsibility of TruncateCommitTs().)Agreed, and done in attached.
I haven't done the same for commit_ts but will do so on the thread for
the commit_ts race fix if we go ahead with this change here.I think it is not correct to advance oldestXid but not oldestXidDB.
Otherwise, GetNewTransactionId() might complain about the wrong
database.That's a clear oversight. Fixed in attached.
New attached also records it in xlog and applies it to the standby.
If we want to save the 4 bytes per xmin advance (probably not worth
caring) we can instead skip setting it on the standby, in which case
it'll be potentially wrong until the next checkpoint. I'd rather make
sure it stays correct.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Fix-race-between-clog-truncation-and-lookup.patchtext/x-patch; charset=US-ASCII; name=0001-Fix-race-between-clog-truncation-and-lookup.patchDownload
From ce69d2824f89f675a25e098b3980a304baca7ade Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:25:30 +0800
Subject: [PATCH 1/2] Fix race between clog truncation and lookup
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you.
This hasn 't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog was truncated before we advanced oldestXid under
XidGenLock, so holding XidGenLock during a clog lookup was insufficient to
prevent the race. There's no way to look up a SLRU with soft-failure;
attempting a lookup produces an I/O error. There's also no safe way to trap and
swallow the SLRU lookup error due mainly to locking issues.
To address this, increase oldestXid under XidGenLock before we trunate clog
rather than after, so concurrent lookups of arbitrary XIDs are safe if they are
done under XidGenLock. The rest of the xid limits are still advanced after clog
truncation to ensure there's no chance of a new xid trying to access an
about-to-be-truncated clog page. In practice this can't happen anyway since we
use only half the xid space at any time, but additional guards against future
change are warranted with something this crucial.
This race also exists in a worse form on standby servers. On a standby we only
advance oldestXid when we replay the next checkpoint, so there's a much larger
window between clog truncation and subsequent updating of the limit. Fix this
by recording the oldest xid in clog truncation records and applying the
oldestXid under XidGenLock before replaying the clog truncation.
Note that There's no need to take XidGenLock for normal clog lookups protected
by datfrozenxid, only if accepting arbitrary XIDs that might not be protected by
vacuum thresholds.
---
src/backend/access/rmgrdesc/clogdesc.c | 12 ++++++--
src/backend/access/transam/clog.c | 50 +++++++++++++++++++++++++++-------
src/backend/access/transam/varsup.c | 49 ++++++++++++++++++++++++++-------
src/backend/access/transam/xlog.c | 22 ++++++++++-----
src/backend/commands/vacuum.c | 4 +--
src/include/access/clog.h | 8 +++++-
src/include/access/transam.h | 4 +--
7 files changed, 115 insertions(+), 34 deletions(-)
diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c
index 352de48..ef268c5 100644
--- a/src/backend/access/rmgrdesc/clogdesc.c
+++ b/src/backend/access/rmgrdesc/clogdesc.c
@@ -23,12 +23,20 @@ clog_desc(StringInfo buf, XLogReaderState *record)
char *rec = XLogRecGetData(record);
uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
- if (info == CLOG_ZEROPAGE || info == CLOG_TRUNCATE)
+ if (info == CLOG_ZEROPAGE)
{
int pageno;
memcpy(&pageno, rec, sizeof(int));
- appendStringInfo(buf, "%d", pageno);
+ appendStringInfo(buf, "page %d", pageno);
+ }
+ else if (info == CLOG_TRUNCATE)
+ {
+ xl_clog_truncate xlrec;
+
+ memcpy(&xlrec, rec, sizeof(xl_clog_truncate));
+ appendStringInfo(buf, "page %d; oldestXact %u",
+ xlrec.pageno, xlrec.oldestXact);
}
}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 1a43819..cfd1c91 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -83,7 +83,8 @@ static SlruCtlData ClogCtlData;
static int ZeroCLOGPage(int pageno, bool writeXlog);
static bool CLOGPagePrecedes(int page1, int page2);
static void WriteZeroPageXlogRec(int pageno);
-static void WriteTruncateXlogRec(int pageno);
+static void WriteTruncateXlogRec(int pageno, TransactionId oldestXact,
+ Oid oldestXidDb);
static void TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status,
XLogRecPtr lsn, int pageno);
@@ -640,11 +641,24 @@ ExtendCLOG(TransactionId newestXact)
* the XLOG flush unless we have confirmed that there is a removable segment.
*/
void
-TruncateCLOG(TransactionId oldestXact)
+TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid)
{
int cutoffPage;
/*
+ * Advance oldestXid before truncating clog, so concurrent xact status
+ * lookups can ensure they don't attempt to access truncated-away clog.
+ *
+ * We must do this even if we find we can't actually truncate away any clog
+ * pages, since we'll advance the xid limits and need oldestXmin to be
+ * consistent with the new limits.
+ *
+ * Losing this on crash before a checkpoint is harmless unless we truncated
+ * clog, in which case redo of the clog truncation will re-apply it.
+ */
+ AdvanceOldestXid(oldestXact, oldestxid_datoid);
+
+ /*
* The cutoff point is the start of the segment containing oldestXact. We
* pass the *page* containing oldestXact to SimpleLruTruncate.
*/
@@ -654,8 +668,17 @@ TruncateCLOG(TransactionId oldestXact)
if (!SlruScanDirectory(ClogCtl, SlruScanDirCbReportPresence, &cutoffPage))
return; /* nothing to remove */
- /* Write XLOG record and flush XLOG to disk */
- WriteTruncateXlogRec(cutoffPage);
+ /* vac_truncate_clog already advanced oldestXid */
+ Assert(TransactionIdPrecedesOrEquals(oldestXact,
+ ShmemVariableCache->oldestXid));
+
+ /*
+ * Write XLOG record and flush XLOG to disk. We record the oldest xid we're
+ * keeping information about here so we can ensure that it's always ahead
+ * of clog truncation in case we crash, and so a standby finds out the new
+ * valid xid before the next checkpoint.
+ */
+ WriteTruncateXlogRec(cutoffPage, oldestXact, oldestxid_datoid);
/* Now we can remove the old CLOG segment(s) */
SimpleLruTruncate(ClogCtl, cutoffPage);
@@ -704,12 +727,17 @@ WriteZeroPageXlogRec(int pageno)
* in TruncateCLOG().
*/
static void
-WriteTruncateXlogRec(int pageno)
+WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb)
{
XLogRecPtr recptr;
+ xl_clog_truncate xlrec;
+
+ xlrec.pageno = pageno;
+ xlrec.oldestXact = oldestXact;
+ xlrec.oldestXactDb = oldestXactDb;
XLogBeginInsert();
- XLogRegisterData((char *) (&pageno), sizeof(int));
+ XLogRegisterData((char *) (&xlrec), sizeof(xl_clog_truncate));
recptr = XLogInsert(RM_CLOG_ID, CLOG_TRUNCATE);
XLogFlush(recptr);
}
@@ -742,17 +770,19 @@ clog_redo(XLogReaderState *record)
}
else if (info == CLOG_TRUNCATE)
{
- int pageno;
+ xl_clog_truncate xlrec;
- memcpy(&pageno, XLogRecGetData(record), sizeof(int));
+ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_clog_truncate));
/*
* During XLOG replay, latest_page_number isn't set up yet; insert a
* suitable value to bypass the sanity test in SimpleLruTruncate.
*/
- ClogCtl->shared->latest_page_number = pageno;
+ ClogCtl->shared->latest_page_number = xlrec.pageno;
+
+ AdvanceOldestXid(xlrec.oldestXact, xlrec.oldestXactDb);
- SimpleLruTruncate(ClogCtl, pageno);
+ SimpleLruTruncate(ClogCtl, xlrec.pageno);
}
else
elog(PANIC, "clog_redo: unknown op code %u", info);
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index fc084c5..9aacb0f 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -259,19 +259,52 @@ ReadNewTransactionId(void)
}
/*
- * Determine the last safe XID to allocate given the currently oldest
+ * Advance the cluster-wide oldestXid.
+ *
+ * We must ensure that this never goes backwards, otherwise the xid limits set
+ * in SetTransactionIdLimit(...) could be insufficiently conservative if two
+ * vacuums race, with the lower oldestXmin winning then the higher xid limits
+ * winning.
+ */
+void
+AdvanceOldestXid(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
+{
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+ if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
+ oldest_datfrozenxid))
+ {
+ ShmemVariableCache->oldestXid = oldest_datfrozenxid;
+ ShmemVariableCache->oldestXidDB = oldest_datoid;
+ }
+ LWLockRelease(XidGenLock);
+}
+
+/*
+ * Determine the last safe XID to allocate using the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* of our cluster), and the OID of the (or a) database with that value.
*/
void
-SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
+SetTransactionIdLimit(void)
{
TransactionId xidVacLimit;
TransactionId xidWarnLimit;
TransactionId xidStopLimit;
TransactionId xidWrapLimit;
TransactionId curXid;
+ Oid oldest_datoid;
+ TransactionId oldest_datfrozenxid;
+
+ LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+ /*
+ * We fetch oldest_datfrozenxid and oldest_datoid from shmem to ensure that
+ * if some other vacuum has advanced oldestXid since we did, we use the
+ * latest values and the limits never go backwards.
+ */
+ oldest_datfrozenxid = ShmemVariableCache->oldestXid;
+ oldest_datoid = ShmemVariableCache->oldestXidDB;
+
Assert(TransactionIdIsNormal(oldest_datfrozenxid));
/*
@@ -284,6 +317,7 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1);
if (xidWrapLimit < FirstNormalTransactionId)
xidWrapLimit += FirstNormalTransactionId;
+ ShmemVariableCache->xidWrapLimit = xidWrapLimit;
/*
* We'll refuse to continue assigning XIDs in interactive mode once we get
@@ -296,6 +330,7 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidStopLimit = xidWrapLimit - 1000000;
if (xidStopLimit < FirstNormalTransactionId)
xidStopLimit -= FirstNormalTransactionId;
+ ShmemVariableCache->xidStopLimit = xidStopLimit;
/*
* We'll start complaining loudly when we get within 10M transactions of
@@ -310,6 +345,7 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidWarnLimit = xidStopLimit - 10000000;
if (xidWarnLimit < FirstNormalTransactionId)
xidWarnLimit -= FirstNormalTransactionId;
+ ShmemVariableCache->xidWarnLimit = xidWarnLimit;
/*
* We'll start trying to force autovacuums when oldest_datfrozenxid gets
@@ -329,15 +365,8 @@ SetTransactionIdLimit(TransactionId oldest_datfrozenxid, Oid oldest_datoid)
xidVacLimit = oldest_datfrozenxid + autovacuum_freeze_max_age;
if (xidVacLimit < FirstNormalTransactionId)
xidVacLimit += FirstNormalTransactionId;
-
- /* Grab lock for just long enough to set the new limit values */
- LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
- ShmemVariableCache->oldestXid = oldest_datfrozenxid;
ShmemVariableCache->xidVacLimit = xidVacLimit;
- ShmemVariableCache->xidWarnLimit = xidWarnLimit;
- ShmemVariableCache->xidStopLimit = xidStopLimit;
- ShmemVariableCache->xidWrapLimit = xidWrapLimit;
- ShmemVariableCache->oldestXidDB = oldest_datoid;
+
curXid = ShmemVariableCache->nextXid;
LWLockRelease(XidGenLock);
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2f5d603..823f4bd 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4877,7 +4877,8 @@ BootStrapXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
- SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ AdvanceOldestXid(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ SetTransactionIdLimit();
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(InvalidTransactionId, InvalidTransactionId);
@@ -6471,7 +6472,8 @@ StartupXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
- SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ AdvanceOldestXid(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ SetTransactionIdLimit();
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB);
SetCommitTsLimit(checkPoint.oldestCommitTsXid,
checkPoint.newestCommitTsXid);
@@ -9456,7 +9458,11 @@ xlog_redo(XLogReaderState *record)
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
- SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
+ /*
+ * No need to call AdvanceOldestXid, startup or an earlier clog trunate
+ * record will have already advanced it. Just advance the limits.
+ */
+ SetTransactionIdLimit();
/*
* If we see a shutdown checkpoint while waiting for an end-of-backup
@@ -9553,10 +9559,12 @@ xlog_redo(XLogReaderState *record)
*/
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
- if (TransactionIdPrecedes(ShmemVariableCache->oldestXid,
- checkPoint.oldestXid))
- SetTransactionIdLimit(checkPoint.oldestXid,
- checkPoint.oldestXidDB);
+ /*
+ * We don't need to AdvanceOldestXid here; StartupXLOG or a clog
+ * truncation record will ensure it's up to date, and we just update
+ * the limits here based on what's already in shmem.
+ */
+ SetTransactionIdLimit();
/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
ControlFile->checkPointCopy.nextXidEpoch = checkPoint.nextXidEpoch;
ControlFile->checkPointCopy.nextXid = checkPoint.nextXid;
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 812fb4a..5ec307f 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1164,7 +1164,7 @@ vac_truncate_clog(TransactionId frozenXID,
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
- TruncateCLOG(frozenXID);
+ TruncateCLOG(frozenXID, oldestxid_datoid);
TruncateCommitTs(frozenXID);
TruncateMultiXact(minMulti, minmulti_datoid);
@@ -1174,7 +1174,7 @@ vac_truncate_clog(TransactionId frozenXID,
* for an(other) autovac cycle if needed. XXX should we avoid possibly
* signalling twice?
*/
- SetTransactionIdLimit(frozenXID, oldestxid_datoid);
+ SetTransactionIdLimit();
SetMultiXactIdLimit(minMulti, minmulti_datoid);
}
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 2894bd5..60a9e11 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,12 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+typedef struct xl_clog_truncate
+{
+ int pageno;
+ TransactionId oldestXact;
+ Oid oldestXactDb;
+} xl_clog_truncate;
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
@@ -42,7 +48,7 @@ extern void TrimCLOG(void);
extern void ShutdownCLOG(void);
extern void CheckPointCLOG(void);
extern void ExtendCLOG(TransactionId newestXact);
-extern void TruncateCLOG(TransactionId oldestXact);
+extern void TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid);
/* XLOG stuff */
#define CLOG_ZEROPAGE 0x00
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 522c104..95cdef1 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -171,8 +171,8 @@ extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
-extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
- Oid oldest_datoid);
+extern void AdvanceOldestXid(TransactionId oldest_datfrozenxid, Oid oldest_datoid);
+extern void SetTransactionIdLimit(void);
extern bool ForceTransactionIdLimitUpdate(void);
extern Oid GetNewObjectId(void);
--
2.5.5
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From c8f517e63762c9f89eaa3aa80c24e5668e801b8e Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH 2/2] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++
src/backend/utils/adt/txid.c | 116 ++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/include/utils/builtins.h | 164 +++++++++++++++++++++++++++++++++++++
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
6 files changed, 415 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b214218..0095915 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17190,6 +17190,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17240,6 +17244,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17310,6 +17319,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..8f86916 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,63 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestXid is protected by XidGenLock, but we don't
+ * acquire that lock here. Instead, we require the caller to acquire it,
+ * because the caller is presumably going to look up the returned XID.
+ * If we took and released the lock within this function, a CLOG
+ * truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(XidGenLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +413,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +720,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * Ensure clog isn't removed between when we check whether the current xid
+ * is valid and when we look its status up.
+ */
+ LWLockAcquire(XidGenLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(XidGenLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index ab12761..912cb5a 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4936,6 +4936,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 5bdca82..26f76ca 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -119,5 +119,169 @@ extern int32 type_maximum_size(Oid type_oid, int32 typemod);
/* quote.c */
extern char *quote_literal_cstr(const char *rawstr);
+extern Datum quote_nullable(PG_FUNCTION_ARGS);
+
+/* guc.c */
+extern Datum show_config_by_name(PG_FUNCTION_ARGS);
+extern Datum show_config_by_name_missing_ok(PG_FUNCTION_ARGS);
+extern Datum set_config_by_name(PG_FUNCTION_ARGS);
+extern Datum show_all_settings(PG_FUNCTION_ARGS);
+extern Datum show_all_file_settings(PG_FUNCTION_ARGS);
+
+/* pg_config.c */
+extern Datum pg_config(PG_FUNCTION_ARGS);
+
+/* pg_controldata.c */
+extern Datum pg_control_checkpoint(PG_FUNCTION_ARGS);
+extern Datum pg_control_system(PG_FUNCTION_ARGS);
+extern Datum pg_control_init(PG_FUNCTION_ARGS);
+extern Datum pg_control_recovery(PG_FUNCTION_ARGS);
+
+/* rls.c */
+extern Datum row_security_active(PG_FUNCTION_ARGS);
+extern Datum row_security_active_name(PG_FUNCTION_ARGS);
+
+/* lockfuncs.c */
+extern Datum pg_lock_status(PG_FUNCTION_ARGS);
+extern Datum pg_blocking_pids(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_shared_int8(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_xact_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_try_advisory_xact_lock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_shared_int4(PG_FUNCTION_ARGS);
+extern Datum pg_advisory_unlock_all(PG_FUNCTION_ARGS);
+
+/* txid.c */
+extern Datum txid_snapshot_in(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_out(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_recv(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_send(PG_FUNCTION_ARGS);
+extern Datum txid_current(PG_FUNCTION_ARGS);
+extern Datum txid_current_if_assigned(PG_FUNCTION_ARGS);
+extern Datum txid_current_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xmin(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xmax(PG_FUNCTION_ARGS);
+extern Datum txid_snapshot_xip(PG_FUNCTION_ARGS);
+extern Datum txid_visible_in_snapshot(PG_FUNCTION_ARGS);
+extern Datum txid_status(PG_FUNCTION_ARGS);
+
+/* uuid.c */
+extern Datum uuid_in(PG_FUNCTION_ARGS);
+extern Datum uuid_out(PG_FUNCTION_ARGS);
+extern Datum uuid_send(PG_FUNCTION_ARGS);
+extern Datum uuid_recv(PG_FUNCTION_ARGS);
+extern Datum uuid_lt(PG_FUNCTION_ARGS);
+extern Datum uuid_le(PG_FUNCTION_ARGS);
+extern Datum uuid_eq(PG_FUNCTION_ARGS);
+extern Datum uuid_ge(PG_FUNCTION_ARGS);
+extern Datum uuid_gt(PG_FUNCTION_ARGS);
+extern Datum uuid_ne(PG_FUNCTION_ARGS);
+extern Datum uuid_cmp(PG_FUNCTION_ARGS);
+extern Datum uuid_sortsupport(PG_FUNCTION_ARGS);
+extern Datum uuid_hash(PG_FUNCTION_ARGS);
+
+/* windowfuncs.c */
+extern Datum window_row_number(PG_FUNCTION_ARGS);
+extern Datum window_rank(PG_FUNCTION_ARGS);
+extern Datum window_dense_rank(PG_FUNCTION_ARGS);
+extern Datum window_percent_rank(PG_FUNCTION_ARGS);
+extern Datum window_cume_dist(PG_FUNCTION_ARGS);
+extern Datum window_ntile(PG_FUNCTION_ARGS);
+extern Datum window_lag(PG_FUNCTION_ARGS);
+extern Datum window_lag_with_offset(PG_FUNCTION_ARGS);
+extern Datum window_lag_with_offset_and_default(PG_FUNCTION_ARGS);
+extern Datum window_lead(PG_FUNCTION_ARGS);
+extern Datum window_lead_with_offset(PG_FUNCTION_ARGS);
+extern Datum window_lead_with_offset_and_default(PG_FUNCTION_ARGS);
+extern Datum window_first_value(PG_FUNCTION_ARGS);
+extern Datum window_last_value(PG_FUNCTION_ARGS);
+extern Datum window_nth_value(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgquadtreeproc.c */
+extern Datum spg_quad_config(PG_FUNCTION_ARGS);
+extern Datum spg_quad_choose(PG_FUNCTION_ARGS);
+extern Datum spg_quad_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_quad_inner_consistent(PG_FUNCTION_ARGS);
+extern Datum spg_quad_leaf_consistent(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgkdtreeproc.c */
+extern Datum spg_kd_config(PG_FUNCTION_ARGS);
+extern Datum spg_kd_choose(PG_FUNCTION_ARGS);
+extern Datum spg_kd_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_kd_inner_consistent(PG_FUNCTION_ARGS);
+
+/* access/spgist/spgtextproc.c */
+extern Datum spg_text_config(PG_FUNCTION_ARGS);
+extern Datum spg_text_choose(PG_FUNCTION_ARGS);
+extern Datum spg_text_picksplit(PG_FUNCTION_ARGS);
+extern Datum spg_text_inner_consistent(PG_FUNCTION_ARGS);
+extern Datum spg_text_leaf_consistent(PG_FUNCTION_ARGS);
+
+/* access/gin/ginarrayproc.c */
+extern Datum ginarrayextract(PG_FUNCTION_ARGS);
+extern Datum ginarrayextract_2args(PG_FUNCTION_ARGS);
+extern Datum ginqueryarrayextract(PG_FUNCTION_ARGS);
+extern Datum ginarrayconsistent(PG_FUNCTION_ARGS);
+extern Datum ginarraytriconsistent(PG_FUNCTION_ARGS);
+
+/* access/tablesample/bernoulli.c */
+extern Datum tsm_bernoulli_handler(PG_FUNCTION_ARGS);
+
+/* access/tablesample/system.c */
+extern Datum tsm_system_handler(PG_FUNCTION_ARGS);
+
+/* access/transam/twophase.c */
+extern Datum pg_prepared_xact(PG_FUNCTION_ARGS);
+
+/* access/transam/multixact.c */
+extern Datum pg_get_multixact_members(PG_FUNCTION_ARGS);
+
+/* access/transam/committs.c */
+extern Datum pg_xact_commit_timestamp(PG_FUNCTION_ARGS);
+extern Datum pg_last_committed_xact(PG_FUNCTION_ARGS);
+
+/* catalogs/dependency.c */
+extern Datum pg_describe_object(PG_FUNCTION_ARGS);
+extern Datum pg_identify_object(PG_FUNCTION_ARGS);
+extern Datum pg_identify_object_as_address(PG_FUNCTION_ARGS);
+
+/* catalog/objectaddress.c */
+extern Datum pg_get_object_address(PG_FUNCTION_ARGS);
+
+/* commands/constraint.c */
+extern Datum unique_key_recheck(PG_FUNCTION_ARGS);
+
+/* commands/event_trigger.c */
+extern Datum pg_event_trigger_dropped_objects(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_table_rewrite_oid(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_table_rewrite_reason(PG_FUNCTION_ARGS);
+extern Datum pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS);
+
+/* commands/extension.c */
+extern Datum pg_available_extensions(PG_FUNCTION_ARGS);
+extern Datum pg_available_extension_versions(PG_FUNCTION_ARGS);
+extern Datum pg_extension_update_paths(PG_FUNCTION_ARGS);
+extern Datum pg_extension_config_dump(PG_FUNCTION_ARGS);
+
+/* commands/prepare.c */
+extern Datum pg_prepared_statement(PG_FUNCTION_ARGS);
+
+/* utils/mmgr/portalmem.c */
+extern Datum pg_cursor(PG_FUNCTION_ARGS);
#endif /* BUILTINS_H */
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Wed, Jan 25, 2017 at 4:02 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
If we want to save the 4 bytes per xmin advance (probably not worth
caring) we can instead skip setting it on the standby, in which case
it'll be potentially wrong until the next checkpoint. I'd rather make
sure it stays correct.
Those patches still apply and no reviews have come in yet, so moved to
CF 2017-03.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Wed, Jan 25, 2017 at 12:44 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
The way that SetTransactionIdLimit() now works looks a bit dangerous.
xidWrapLimit, xidStopLimit, and xidWarnLimit are computed based on the
passed-in oldestXid value and written straight into shared memory.
But the shared memory copy of oldestXid could have a different value.
I'm not sure if that breaks anything, but it certainly weakens any
confidence callers might have had that all those values are consistent
with each other.This was my main hesitation with the whole thing too.
It's necessary to advance oldestXmin before we xlog the advance and
truncate clog, and necessary to advance the vacuum limits only
afterwards.
Well, that's why I tried to advocate that their should be two separate
XID limits in shared memory: leave what's there now the way it is, and
then add a new limit for "don't try to look up XIDs before this point:
splat". I still think that'd be less risky.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 10 March 2017 at 02:55, Robert Haas <robertmhaas@gmail.com> wrote:
Well, that's why I tried to advocate that their should be two separate
XID limits in shared memory: leave what's there now the way it is, and
then add a new limit for "don't try to look up XIDs before this point:
splat". I still think that'd be less risky.
I'm coming back to this "cold" after an extended break, so I hope I
get the details right.
TL;DR: doing it that way duplicates a bunch of stuff and is ugly
without offering significant benefit over just fixing the original.
I started out with the approach you suggested, but it turns out to be
less helpful than you'd think. Simply advancing a new lower limit
field before beginning truncation is no help; there's then a race
where the lower-limit field can be advanced after we test it but
before we actually do the SLRU read from clog. To guard against this
it's necessary for SLRU truncation to take an exclusive lock during
which it advances the lower bound. That way a concurrent reader can
take the lock in shared mode before reading the lower bound and hold
it until it finishes the clog read, knowing that vacuum cannot then
truncate the data out from under it because it'll block trying to
advance the lower limit.
A spinlock isn't suitable for this. While we can take the lock only
briefly to update the limit field in vac_truncate_clog, in
txid_status() we have to hold it from when we test the boundary
through to when we finish the SLRU clog lookup, and that lookup does
I/O and might sleep. If we release it after testing the lower bound
but before the SLRU lookup our race comes back since vacuum can jump
in and truncate it out from under us. So now we need a new LWLock used
only for vac_truncate_clog before advancing the clog truncation.
I implemented just that - a new ClogTruncateLog in the lwlocks array
and a new field in ShmemVariableCache for the lower xid bound, IIRC.
Other than requiring an extra lwlock acquisition for vac_truncate_clog
it works fine ... for the master.
But it doesn't fix the much bigger race on the standby. We only emit
WAL for xid limits after we truncate clog, and the clog truncation
record doesn't record the new limit.
So now we need a new, somewhat redundant, xlog record and redo
function for this lower clog bound pointer. Which, really, is only
actually tracking a slightly more up to date version of oldestXid.
At that point I was just papering around a race that should just be
fixed at its source instead. Advance oldestXid before truncating clog,
and write a clog truncation record that includes the new oldestXid.
So... I can go back to the old approach and just add the new xlog
record and redo method, new lwlock, new shmemvariablecache field, etc,
if you're really concerned this approach is risky. But I'd rather fix
the original problem instead.
It might be helpful if I separate out the patch that touches oldestXid
from the rest for separate review, so I'll update here with a 2-patch
series instead of a squashed single patch soon.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Fri, Mar 10, 2017 at 2:00 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 10 March 2017 at 02:55, Robert Haas <robertmhaas@gmail.com> wrote:
Well, that's why I tried to advocate that their should be two separate
XID limits in shared memory: leave what's there now the way it is, and
then add a new limit for "don't try to look up XIDs before this point:
splat". I still think that'd be less risky.I'm coming back to this "cold" after an extended break, so I hope I
get the details right.
Yeah, sorry I've been away from this for a while.
TL;DR: doing it that way duplicates a bunch of stuff and is ugly
without offering significant benefit over just fixing the original.I started out with the approach you suggested, but it turns out to be
less helpful than you'd think. Simply advancing a new lower limit
field before beginning truncation is no help; there's then a race
where the lower-limit field can be advanced after we test it but
before we actually do the SLRU read from clog. To guard against this
it's necessary for SLRU truncation to take an exclusive lock during
which it advances the lower bound. That way a concurrent reader can
take the lock in shared mode before reading the lower bound and hold
it until it finishes the clog read, knowing that vacuum cannot then
truncate the data out from under it because it'll block trying to
advance the lower limit.
That's a good point which I hadn't fully considered. On the other
hand, there really are two separate notions of the "oldest" XID.
There's the oldest XID that we can safely look up, and then there's
the oldest XID that we can't reuse. These two are the same when no
truncation is in progress, but when a truncation is in progress then
they're different: the oldest XID that's safe to look up is the first
one after whatever we're truncating away, but the oldest XID that we
can't reuse is the newest one preceding the stuff that we're busy
truncating. IOW, when truncation is happening, there's a portion of
the XID space whose clog files are being removed - and the XIDs that
are in that range aren't safe to look up any more, but are also not
available for reuse to prevent wraparound. Right now, all of the
relevant fields in VariableCacheData are based on the ready-for-reuse
concept, and I don't think that switching some but not all of them to
be based on the safe-to-look-up concept necessarily qualifies as an
improvement. It's different, but I'm not sure it's better.
What if we approached this problem from the other end? Suppose we use
a heavyweight lock on, say, transaction ID 1 to represent the right to
truncate CLOG. We grab this lock in exclusive mode before beginning
to truncate, and in shared mode while looking up XIDs.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 11 March 2017 at 05:09, Robert Haas <robertmhaas@gmail.com> wrote:
On the other
hand, there really are two separate notions of the "oldest" XID.
There's the oldest XID that we can safely look up, and then there's
the oldest XID that we can't reuse. These two are the same when no
truncation is in progress, but when a truncation is in progress then
they're different: the oldest XID that's safe to look up is the first
one after whatever we're truncating away, but the oldest XID that we
can't reuse is the newest one preceding the stuff that we're busy
truncating.
Right.
My view here is that the oldest xid we cannot reuse is already guarded
by xidWrapLimit, which we advance after clog truncation. Whether as
this advances at the same time as or after we advance oldestXid and
truncate clog doesn't actually matter, we must just ensure that it
never advances _before_.
So tracking a second copy of oldestXid whose only purpose is to
recalculate xidWrapLimit serves no real purpose. It's redundant except
during vac_truncate_clog, during which time local state is sufficient
*if* we add oldestXid to the clog truncation xlog record, which we
must do anyway because:
Any number of locking hoop-jumping schemes fail to solve the problem
of outdated oldestXid information on standbys. Right now we truncate
clog and xlog the truncation before we write the new oldestXid limit
to xlog. In fact, we don't write the new xid limit to xlog until the
next checkpoint. So the standby has a huge window where its idea of
oldestXid is completely wrong, and unless we at least add the new
oldestXid to the clog truncation xlog record we can't fix that.
We only get away with this now because there's no way to look up an
arbitrary xid's status.
No locking scheme on the master can solve this, because the locks on
the master do not affect the standby or vice versa.
Therefore, we _must_ advance oldestXid (or a copy of it used only for
"oldest xid still in clog) before truncating clog.
If we're going to do that we might as well just make sure the
standby's xid limits are updated correctly when we truncate clog
rather than doing it lazily at checkpoints. Advance oldestXid before
truncating clog away, and record the new xid in the clog truncation
xlog record. On redo after master crash, and on standbys, we're
guaranteed to re-do the whole clog truncation operation - advance
oldestXid, truncate clog, advance xidWrapLimit etc - and everything
stays consistent.
I'll extract this part of the patch so it can be looked at separately,
it'll be clearer that way.
I think of it as slightly contracting then slightly expanding the xid
range window during clog truncation. Advance the oldest xid slightly
before the xidWrapLimit, so temporarily the range of xids is narrower
than 2^31. xlog it first so we ensure it's all redone on crash and on
standby. Because no lock is held throughout all of vac_truncate_clog,
make sure the ordering of the different phases between concurrent
vac_truncate_xlog runs doesn't matter.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 11 March 2017 at 14:32, Craig Ringer <craig@2ndquadrant.com> wrote:
I'll extract this part of the patch so it can be looked at separately,
it'll be clearer that way.
Apparently I thought that last time too since I already posted it
split up. Ahem. Working on too many different things at once.
Last-posted patches apply fine, no need for an update.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Sat, Mar 11, 2017 at 1:32 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 11 March 2017 at 05:09, Robert Haas <robertmhaas@gmail.com> wrote:
On the other
hand, there really are two separate notions of the "oldest" XID.
There's the oldest XID that we can safely look up, and then there's
the oldest XID that we can't reuse. These two are the same when no
truncation is in progress, but when a truncation is in progress then
they're different: the oldest XID that's safe to look up is the first
one after whatever we're truncating away, but the oldest XID that we
can't reuse is the newest one preceding the stuff that we're busy
truncating.Right.
My view here is that the oldest xid we cannot reuse is already guarded
by xidWrapLimit, which we advance after clog truncation. Whether as
this advances at the same time as or after we advance oldestXid and
truncate clog doesn't actually matter, we must just ensure that it
never advances _before_.So tracking a second copy of oldestXid whose only purpose is to
recalculate xidWrapLimit serves no real purpose.
Hmm, so what this patch is doing changed quite a bit between January
23rd and January 25th. In the January 23rd version, oldestXid and
oldestXidDB are changed to track the oldest XID that we can safely
look up, and the remaining related fields are still relative to the
oldest XID that can be reused. That seems, as I said before, scary.
But in the January 25th version, *all* of the related fields have been
changed to track the oldest XID that we can safely look up, because
SetTransactionIdLimit() now uses the values set by AdvanceOldestXid()
to compute all of the other values, which seems flat-out incorrect.
AdvanceOldestXid() is called to advance that limit before clog
truncation happens, and if somebody then calls SetTransactionIdLimit()
before clog truncation is complete, we'll advanced those derived
limits prematurely.
For example, suppose vacuum #1 comes along, advances the limits,
truncates clog, and then gets descheduled. Now vacuum #2 comes along,
advances the limits further, and then gets descheduled. Now vacuum #1
wakes up and calls SetTransactionIdLimit() and prematurely advances
xidWrapLimit. Oops.
The way you put it is that having a second copy of oldestXid whose
only purpose is to recompute xidWrapLimit is pointless, but the way
I'd say is that you're trying to make one variable do two jobs.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 14 March 2017 at 05:43, Robert Haas <robertmhaas@gmail.com> wrote:
For example, suppose vacuum #1 comes along, advances the limits,
truncates clog, and then gets descheduled. Now vacuum #2 comes along,
advances the limits further, and then gets descheduled. Now vacuum #1
wakes up and calls SetTransactionIdLimit() and prematurely advances
xidWrapLimit. Oops.
Mm, right. And without a lock held from when oldestXid advances
through to completion of clog truncation, then taking the same lock in
SetTransactionIdLimit, there's not really a way around it.
I'm embarrassed not to have seen that.
Doing things the other way around, per the earlier patch, can cause
SetTransactionIdLimit to not to advance as far as it should.
OK, I'm convinced, a new field is safer, even if it's redundant most
of the time.
I'll introduce a new LWLock, ClogTruncationLock, which will be held
from when we advance the new clogOldestXid field through to when clog
truncation completes.
Most of the rest can stay the same. In particular, the expanded xlog
record for clog truncation will still be required.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Mon, Mar 13, 2017 at 10:22 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
I'll introduce a new LWLock, ClogTruncationLock, which will be held
from when we advance the new clogOldestXid field through to when clog
truncation completes.
+1.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 14 March 2017 at 19:57, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Mar 13, 2017 at 10:22 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
I'll introduce a new LWLock, ClogTruncationLock, which will be held
from when we advance the new clogOldestXid field through to when clog
truncation completes.+1.
OK, here's the revised patch. Relevant comments added where needed.
It still changes the xlog record for clog truncation to carry the new
xid, but I've left advancing oldestXid until the checkpoint as is
currently the case, and only advanced oldestClogXid when we replay
clog truncation.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Fix-race-between-clog-truncation-and-lookup.patchtext/x-patch; charset=US-ASCII; name=0001-Fix-race-between-clog-truncation-and-lookup.patchDownload
From f2280911c8ba9d4ce75d72948aef2acd4821f613 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:25:30 +0800
Subject: [PATCH 1/3] Fix race between clog truncation and lookup
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you.
This hasn 't been a problem because anything looking up xids in clog knows
they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog is truncated before we advanced oldestXid under
XidGenLock, so holding XidGenLock during a clog lookup is insufficient to
prevent the race. There's no way to look up a SLRU with soft-failure;
attempting a lookup produces an I/O error. There's also no safe way to trap and
swallow the SLRU lookup error due mainly to locking issues.
To address this, introduce a copy of oldestXid, oldestClogXid, that is advanced
before clog truncation under ClogTruncationLock. Lookups of arbitrary XIDs
must take and hold ClogTruncationLock to prevent concurrent advance of the
minimum valid xid in clog.
This race also exists in a worse form on standby servers. On a standby we only
advance oldestXid when we replay the next checkpoint, so there's a much larger
window between clog truncation and subsequent updating of the limit. Fix this
by recording the oldest xid in clog truncation records and applying the update
to oldestClogXid under ClogTruncationLock before replaying the clog truncation.
Note that there's no need to take ClogTruncationLock for normal clog lookups
protected by datfrozenxid, only if accepting arbitrary XIDs that might not be
protected by vacuum thresholds.
---
src/backend/access/rmgrdesc/clogdesc.c | 12 +++++++--
src/backend/access/transam/clog.c | 46 +++++++++++++++++++++++++-------
src/backend/access/transam/transam.c | 10 +++++--
src/backend/access/transam/varsup.c | 23 +++++++++++++++-
src/backend/access/transam/xlog.c | 11 ++++++++
src/backend/commands/vacuum.c | 2 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/include/access/clog.h | 8 +++++-
src/include/access/transam.h | 7 +++++
9 files changed, 103 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c
index 352de48..ef268c5 100644
--- a/src/backend/access/rmgrdesc/clogdesc.c
+++ b/src/backend/access/rmgrdesc/clogdesc.c
@@ -23,12 +23,20 @@ clog_desc(StringInfo buf, XLogReaderState *record)
char *rec = XLogRecGetData(record);
uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
- if (info == CLOG_ZEROPAGE || info == CLOG_TRUNCATE)
+ if (info == CLOG_ZEROPAGE)
{
int pageno;
memcpy(&pageno, rec, sizeof(int));
- appendStringInfo(buf, "%d", pageno);
+ appendStringInfo(buf, "page %d", pageno);
+ }
+ else if (info == CLOG_TRUNCATE)
+ {
+ xl_clog_truncate xlrec;
+
+ memcpy(&xlrec, rec, sizeof(xl_clog_truncate));
+ appendStringInfo(buf, "page %d; oldestXact %u",
+ xlrec.pageno, xlrec.oldestXact);
}
}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 5b1d13d..2d33510 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -83,7 +83,8 @@ static SlruCtlData ClogCtlData;
static int ZeroCLOGPage(int pageno, bool writeXlog);
static bool CLOGPagePrecedes(int page1, int page2);
static void WriteZeroPageXlogRec(int pageno);
-static void WriteTruncateXlogRec(int pageno);
+static void WriteTruncateXlogRec(int pageno, TransactionId oldestXact,
+ Oid oldestXidDb);
static void TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status,
XLogRecPtr lsn, int pageno);
@@ -640,7 +641,7 @@ ExtendCLOG(TransactionId newestXact)
* the XLOG flush unless we have confirmed that there is a removable segment.
*/
void
-TruncateCLOG(TransactionId oldestXact)
+TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid)
{
int cutoffPage;
@@ -654,8 +655,26 @@ TruncateCLOG(TransactionId oldestXact)
if (!SlruScanDirectory(ClogCtl, SlruScanDirCbReportPresence, &cutoffPage))
return; /* nothing to remove */
- /* Write XLOG record and flush XLOG to disk */
- WriteTruncateXlogRec(cutoffPage);
+ /*
+ * Advance oldestClogXid before truncating clog, so concurrent xact status
+ * lookups can ensure they don't attempt to access truncated-away clog.
+ *
+ * It's only necessary to do this if we will actually truncate away clog
+ * pages.
+ */
+ AdvanceOldestClogXid(oldestXact);
+
+ /* vac_truncate_clog already advanced oldestXid */
+ Assert(TransactionIdPrecedesOrEquals(oldestXact,
+ ShmemVariableCache->oldestXid));
+
+ /*
+ * Write XLOG record and flush XLOG to disk. We record the oldest xid we're
+ * keeping information about here so we can ensure that it's always ahead
+ * of clog truncation in case we crash, and so a standby finds out the new
+ * valid xid before the next checkpoint.
+ */
+ WriteTruncateXlogRec(cutoffPage, oldestXact, oldestxid_datoid);
/* Now we can remove the old CLOG segment(s) */
SimpleLruTruncate(ClogCtl, cutoffPage);
@@ -704,12 +723,17 @@ WriteZeroPageXlogRec(int pageno)
* in TruncateCLOG().
*/
static void
-WriteTruncateXlogRec(int pageno)
+WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb)
{
XLogRecPtr recptr;
+ xl_clog_truncate xlrec;
+
+ xlrec.pageno = pageno;
+ xlrec.oldestXact = oldestXact;
+ xlrec.oldestXactDb = oldestXactDb;
XLogBeginInsert();
- XLogRegisterData((char *) (&pageno), sizeof(int));
+ XLogRegisterData((char *) (&xlrec), sizeof(xl_clog_truncate));
recptr = XLogInsert(RM_CLOG_ID, CLOG_TRUNCATE);
XLogFlush(recptr);
}
@@ -742,17 +766,19 @@ clog_redo(XLogReaderState *record)
}
else if (info == CLOG_TRUNCATE)
{
- int pageno;
+ xl_clog_truncate xlrec;
- memcpy(&pageno, XLogRecGetData(record), sizeof(int));
+ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_clog_truncate));
/*
* During XLOG replay, latest_page_number isn't set up yet; insert a
* suitable value to bypass the sanity test in SimpleLruTruncate.
*/
- ClogCtl->shared->latest_page_number = pageno;
+ ClogCtl->shared->latest_page_number = xlrec.pageno;
- SimpleLruTruncate(ClogCtl, pageno);
+ AdvanceOldestClogXid(xlrec.oldestXact);
+
+ SimpleLruTruncate(ClogCtl, xlrec.pageno);
}
else
elog(PANIC, "clog_redo: unknown op code %u", info);
diff --git a/src/backend/access/transam/transam.c b/src/backend/access/transam/transam.c
index b91a259..2b36b86 100644
--- a/src/backend/access/transam/transam.c
+++ b/src/backend/access/transam/transam.c
@@ -119,7 +119,10 @@ TransactionLogFetch(TransactionId transactionId)
* True iff transaction associated with the identifier did commit.
*
* Note:
- * Assumes transaction identifier is valid.
+ * Assumes transaction identifier is valid and exists in clog.
+ *
+ * Callers looking up arbitrary xids should take ClogTruncationLock
+ * and test oldestClogXid.
*/
bool /* true if given transaction committed */
TransactionIdDidCommit(TransactionId transactionId)
@@ -175,7 +178,10 @@ TransactionIdDidCommit(TransactionId transactionId)
* True iff transaction associated with the identifier did abort.
*
* Note:
- * Assumes transaction identifier is valid.
+ * Assumes transaction identifier is valid and exists in clog.
+ *
+ * Callers looking up arbitrary xids should take ClogTruncationLock
+ * and test oldestClogXid.
*/
bool /* true if given transaction aborted */
TransactionIdDidAbort(TransactionId transactionId)
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 42fc351..104b8eb 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -259,7 +259,28 @@ ReadNewTransactionId(void)
}
/*
- * Determine the last safe XID to allocate given the currently oldest
+ * Advance the cluster-wide value for the oldest valid clog entry.
+ *
+ * We must acquire ClogTruncationLock to advance the oldestClogXid. It's not
+ * necessary to hold the lock during the actual clog truncation, only when we
+ * advance the limit, as code looking up arbitrary xids is required to hold
+ * ClogTruncationLock from when it tests oldestClogXid through to when it
+ * completes the clog lookup.
+ */
+void
+AdvanceOldestClogXid(TransactionId oldest_datfrozenxid)
+{
+ LWLockAcquire(ClogTruncationLock, LW_EXCLUSIVE);
+ if (TransactionIdPrecedes(ShmemVariableCache->oldestClogXid,
+ oldest_datfrozenxid))
+ {
+ ShmemVariableCache->oldestClogXid = oldest_datfrozenxid;
+ }
+ LWLockRelease(ClogTruncationLock);
+}
+
+/*
+ * Determine the last safe XID to allocate using the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* of our cluster), and the OID of the (or a) database with that value.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 9480377..fbdff55 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -5016,6 +5016,7 @@ BootStrapXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestClogXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB, true);
SetCommitTsLimit(InvalidTransactionId, InvalidTransactionId);
@@ -6622,6 +6623,7 @@ StartupXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestClogXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB, true);
SetCommitTsLimit(checkPoint.oldestCommitTsXid,
@@ -8687,6 +8689,11 @@ CreateCheckPoint(int flags)
/*
* Get the other info we need for the checkpoint record.
+ *
+ * We don't need to save oldestClogXid in the checkpoint, it only matters
+ * for the short period in which clog is being truncated, and if we crash
+ * during that we'll redo the clog truncation and fix up oldestClogXid
+ * there.
*/
LWLockAcquire(XidGenLock, LW_SHARED);
checkPoint.nextXid = ShmemVariableCache->nextXid;
@@ -9616,6 +9623,10 @@ xlog_redo(XLogReaderState *record)
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
+ /*
+ * No need to set oldestClogXid here as well; it'll be set when we
+ * redo an xl_clog_truncate if it changed since initialization.
+ */
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
/*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ff633fa..c4a0f89 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1194,7 +1194,7 @@ vac_truncate_clog(TransactionId frozenXID,
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
- TruncateCLOG(frozenXID);
+ TruncateCLOG(frozenXID, oldestxid_datoid);
TruncateCommitTs(frozenXID);
TruncateMultiXact(minMulti, minmulti_datoid);
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index cd8b08f..a556ec3 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
BackendRandomLock 43
LogicalRepWorkerLock 44
+ClogTruncationLock 45
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 2894bd5..60a9e11 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,12 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+typedef struct xl_clog_truncate
+{
+ int pageno;
+ TransactionId oldestXact;
+ Oid oldestXactDb;
+} xl_clog_truncate;
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
@@ -42,7 +48,7 @@ extern void TrimCLOG(void);
extern void ShutdownCLOG(void);
extern void CheckPointCLOG(void);
extern void ExtendCLOG(TransactionId newestXact);
-extern void TruncateCLOG(TransactionId oldestXact);
+extern void TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid);
/* XLOG stuff */
#define CLOG_ZEROPAGE 0x00
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 522c104..5e3ca33 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -134,6 +134,12 @@ typedef struct VariableCacheData
*/
TransactionId latestCompletedXid; /* newest XID that has committed or
* aborted */
+
+ /*
+ * These fields are protected by ClogTruncationLock
+ */
+ TransactionId oldestClogXid; /* oldest it's safe to look up in clog */
+
} VariableCacheData;
typedef VariableCacheData *VariableCache;
@@ -173,6 +179,7 @@ extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
Oid oldest_datoid);
+extern void AdvanceOldestClogXid(TransactionId oldest_datfrozenxid);
extern bool ForceTransactionIdLimitUpdate(void);
extern Oid GetNewObjectId(void);
--
2.5.5
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 35b2061b903e17897b2aaa5ee576c7db5a0e5d0f Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH 2/3] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 +++++++++
src/backend/utils/adt/txid.c | 120 +++++++++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 ++++++++++++
5 files changed, 255 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9408a25..c8639b8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17523,6 +17523,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17573,6 +17577,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17643,6 +17652,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..0564b77 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,67 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ *
+ * The caller must hold ClogTruncationLock since it's dealing with arbitrary
+ * XIDs, and must continue to hold it until it's done with any clog lookups
+ * relating to those XIDs.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestClogXid is protected by ClogTruncationLock,
+ * but we don't acquire that lock here. Instead, we require the caller to
+ * acquire it, because the caller is presumably going to look up the
+ * returned XID. If we took and released the lock within this function, a
+ * CLOG truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(ClogTruncationLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestClogXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestClogXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +417,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +724,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must protect against concurrent truncation of clog entries to avoid
+ * an I/O error on SLRU lookup.
+ */
+ LWLockAcquire(ClogTruncationLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(ClogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 836d6ff..39bd295 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4975,6 +4975,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Mon, Mar 20, 2017 at 1:38 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 14 March 2017 at 19:57, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Mar 13, 2017 at 10:22 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
I'll introduce a new LWLock, ClogTruncationLock, which will be held
from when we advance the new clogOldestXid field through to when clog
truncation completes.+1.
OK, here's the revised patch. Relevant comments added where needed.
It still changes the xlog record for clog truncation to carry the new
xid, but I've left advancing oldestXid until the checkpoint as is
currently the case, and only advanced oldestClogXid when we replay
clog truncation.
/me smacks forehead. Actually, it should be CLogTruncationLock, with
a capital L, for consistency with CLogControlLock.
The new lock needs to be added to the table in monitoring.sgml.
I don't think the new header comments in TransactionIdDidCommit and
TransactionIdDidAbort are super-clear. I'm not sure you're going to
be able to explain it there in a reasonable number of words, but I
think that speaking of "testing against oldestClogXid" will leave
people wondering what exactly that means. Maybe just write "caller is
responsible for ensuring that the clog records covering XID being
looked up can't be truncated away while the lookup is in progress",
and then leave the bit about CLogTruncationLock to be explained by the
callers that do that. Or you could drop these comments entirely.
Overall, though, I think that 0001 looks far better than any previous
iteration. It's simple. It looks safe. It seems unlikely to break
anything that works now. Woo hoo!
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 March 2017 at 01:49, Robert Haas <robertmhaas@gmail.com> wrote:
/me smacks forehead. Actually, it should be CLogTruncationLock, with
a capital L, for consistency with CLogControlLock.
Will do.
The new lock needs to be added to the table in monitoring.sgml.
Same.
I don't think the new header comments in TransactionIdDidCommit and
TransactionIdDidAbort are super-clear. I'm not sure you're going to
be able to explain it there in a reasonable number of words, but I
think that speaking of "testing against oldestClogXid" will leave
people wondering what exactly that means. Maybe just write "caller is
responsible for ensuring that the clog records covering XID being
looked up can't be truncated away while the lookup is in progress",
and then leave the bit about CLogTruncationLock to be explained by the
callers that do that. Or you could drop these comments entirely.
OK. I'll revisit and see if I can clean it up, otherwise remove it.
Overall, though, I think that 0001 looks far better than any previous
iteration. It's simple. It looks safe. It seems unlikely to break
anything that works now. Woo hoo!
Funny that this started with "hey, here's a simple, non-invasive
function for looking up the status of an arbitrary xid".
Mature, complex systems eh?
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 March 2017 at 09:49, Craig Ringer <craig@2ndquadrant.com> wrote:
Overall, though, I think that 0001 looks far better than any previous
iteration. It's simple. It looks safe. It seems unlikely to break
anything that works now. Woo hoo!Funny that this started with "hey, here's a simple, non-invasive
function for looking up the status of an arbitrary xid".
Changes made per discussion.
Removed the comments on TransactionIdDidCommit and
TransactionIdDidAbort . It's not going to be relevant for the immense
majority of callers anyway, and callers that are looking up arbitrary
user supplied XIDs will (hopefully) be looking at
TransactionIdInRecentPast anyway.
I'll be leaving the 'xid' vs 'bigint' issues elsewhere in Pg for next
release, nowhere near time for that now.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Fix-race-between-clog-truncation-and-lookup.patchtext/x-patch; charset=US-ASCII; name=0001-Fix-race-between-clog-truncation-and-lookup.patchDownload
From 5766e6506e569b0d91e22e90c5e6786ce9fefdf6 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:25:30 +0800
Subject: [PATCH 1/3] Fix race between clog truncation and lookup
There was previously no way to look up an arbitrary xid without
running the risk of having clog truncated out from under you.
This hasn't previously been a problem because anything looking up xids in clog
knows they're protected by datminxid, but that's not the case for arbitrary
user-supplied XIDs. clog is truncated before we advanced oldestXid under
XidGenLock, so holding XidGenLock during a clog lookup is insufficient to
prevent the race. There's no way to look up a SLRU with soft-failure;
attempting a lookup produces an I/O error. There's also no safe way to trap and
swallow the SLRU lookup error due mainly to locking issues.
To address this, introduce a copy of oldestXid, oldestClogXid, that is advanced
before clog truncation under a new LWLock, CLogTruncationLock. Lookups of
arbitrary XIDs must take and hold CLogTruncationLock to prevent concurrent
advance of the minimum valid xid in clog.
This race also exists in a worse form on standby servers. On a standby we only
advance oldestXid when we replay the next checkpoint, so there's a much larger
window between clog truncation and subsequent updating of the limit. Fix this
by recording the oldest xid in clog truncation records and applying the update
to oldestClogXid under ClogTruncationLock before replaying the clog truncation.
No attempt is made to eagerly update oldestXid on the standby, so it may fall
behind oldestClogXid until the next checkpoint.
Note that there's no need to take ClogTruncationLock for normal clog lookups
protected by datfrozenxid, only if accepting arbitrary XIDs that might not be
protected by vacuum thresholds.
---
doc/src/sgml/monitoring.sgml | 4 +++
src/backend/access/rmgrdesc/clogdesc.c | 12 +++++++--
src/backend/access/transam/clog.c | 46 +++++++++++++++++++++++++-------
src/backend/access/transam/transam.c | 4 +--
src/backend/access/transam/varsup.c | 23 +++++++++++++++-
src/backend/access/transam/xlog.c | 11 ++++++++
src/backend/commands/vacuum.c | 2 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/include/access/clog.h | 8 +++++-
src/include/access/transam.h | 7 +++++
10 files changed, 101 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index dcb2d33..1c84ce5 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1018,6 +1018,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry>Waiting to read or update old snapshot control information.</entry>
</row>
<row>
+ <entry><literal>CLogTruncationLock</></entry>
+ <entry>Waiting to truncate the transaction log or waiting for transaction log truncation to finish.</entry>
+ </row>
+ <row>
<entry><literal>clog</></entry>
<entry>Waiting for I/O on a clog (transaction status) buffer.</entry>
</row>
diff --git a/src/backend/access/rmgrdesc/clogdesc.c b/src/backend/access/rmgrdesc/clogdesc.c
index 352de48..ef268c5 100644
--- a/src/backend/access/rmgrdesc/clogdesc.c
+++ b/src/backend/access/rmgrdesc/clogdesc.c
@@ -23,12 +23,20 @@ clog_desc(StringInfo buf, XLogReaderState *record)
char *rec = XLogRecGetData(record);
uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
- if (info == CLOG_ZEROPAGE || info == CLOG_TRUNCATE)
+ if (info == CLOG_ZEROPAGE)
{
int pageno;
memcpy(&pageno, rec, sizeof(int));
- appendStringInfo(buf, "%d", pageno);
+ appendStringInfo(buf, "page %d", pageno);
+ }
+ else if (info == CLOG_TRUNCATE)
+ {
+ xl_clog_truncate xlrec;
+
+ memcpy(&xlrec, rec, sizeof(xl_clog_truncate));
+ appendStringInfo(buf, "page %d; oldestXact %u",
+ xlrec.pageno, xlrec.oldestXact);
}
}
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 5b1d13d..2d33510 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -83,7 +83,8 @@ static SlruCtlData ClogCtlData;
static int ZeroCLOGPage(int pageno, bool writeXlog);
static bool CLOGPagePrecedes(int page1, int page2);
static void WriteZeroPageXlogRec(int pageno);
-static void WriteTruncateXlogRec(int pageno);
+static void WriteTruncateXlogRec(int pageno, TransactionId oldestXact,
+ Oid oldestXidDb);
static void TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status,
XLogRecPtr lsn, int pageno);
@@ -640,7 +641,7 @@ ExtendCLOG(TransactionId newestXact)
* the XLOG flush unless we have confirmed that there is a removable segment.
*/
void
-TruncateCLOG(TransactionId oldestXact)
+TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid)
{
int cutoffPage;
@@ -654,8 +655,26 @@ TruncateCLOG(TransactionId oldestXact)
if (!SlruScanDirectory(ClogCtl, SlruScanDirCbReportPresence, &cutoffPage))
return; /* nothing to remove */
- /* Write XLOG record and flush XLOG to disk */
- WriteTruncateXlogRec(cutoffPage);
+ /*
+ * Advance oldestClogXid before truncating clog, so concurrent xact status
+ * lookups can ensure they don't attempt to access truncated-away clog.
+ *
+ * It's only necessary to do this if we will actually truncate away clog
+ * pages.
+ */
+ AdvanceOldestClogXid(oldestXact);
+
+ /* vac_truncate_clog already advanced oldestXid */
+ Assert(TransactionIdPrecedesOrEquals(oldestXact,
+ ShmemVariableCache->oldestXid));
+
+ /*
+ * Write XLOG record and flush XLOG to disk. We record the oldest xid we're
+ * keeping information about here so we can ensure that it's always ahead
+ * of clog truncation in case we crash, and so a standby finds out the new
+ * valid xid before the next checkpoint.
+ */
+ WriteTruncateXlogRec(cutoffPage, oldestXact, oldestxid_datoid);
/* Now we can remove the old CLOG segment(s) */
SimpleLruTruncate(ClogCtl, cutoffPage);
@@ -704,12 +723,17 @@ WriteZeroPageXlogRec(int pageno)
* in TruncateCLOG().
*/
static void
-WriteTruncateXlogRec(int pageno)
+WriteTruncateXlogRec(int pageno, TransactionId oldestXact, Oid oldestXactDb)
{
XLogRecPtr recptr;
+ xl_clog_truncate xlrec;
+
+ xlrec.pageno = pageno;
+ xlrec.oldestXact = oldestXact;
+ xlrec.oldestXactDb = oldestXactDb;
XLogBeginInsert();
- XLogRegisterData((char *) (&pageno), sizeof(int));
+ XLogRegisterData((char *) (&xlrec), sizeof(xl_clog_truncate));
recptr = XLogInsert(RM_CLOG_ID, CLOG_TRUNCATE);
XLogFlush(recptr);
}
@@ -742,17 +766,19 @@ clog_redo(XLogReaderState *record)
}
else if (info == CLOG_TRUNCATE)
{
- int pageno;
+ xl_clog_truncate xlrec;
- memcpy(&pageno, XLogRecGetData(record), sizeof(int));
+ memcpy(&xlrec, XLogRecGetData(record), sizeof(xl_clog_truncate));
/*
* During XLOG replay, latest_page_number isn't set up yet; insert a
* suitable value to bypass the sanity test in SimpleLruTruncate.
*/
- ClogCtl->shared->latest_page_number = pageno;
+ ClogCtl->shared->latest_page_number = xlrec.pageno;
- SimpleLruTruncate(ClogCtl, pageno);
+ AdvanceOldestClogXid(xlrec.oldestXact);
+
+ SimpleLruTruncate(ClogCtl, xlrec.pageno);
}
else
elog(PANIC, "clog_redo: unknown op code %u", info);
diff --git a/src/backend/access/transam/transam.c b/src/backend/access/transam/transam.c
index b91a259..562b53b 100644
--- a/src/backend/access/transam/transam.c
+++ b/src/backend/access/transam/transam.c
@@ -119,7 +119,7 @@ TransactionLogFetch(TransactionId transactionId)
* True iff transaction associated with the identifier did commit.
*
* Note:
- * Assumes transaction identifier is valid.
+ * Assumes transaction identifier is valid and exists in clog.
*/
bool /* true if given transaction committed */
TransactionIdDidCommit(TransactionId transactionId)
@@ -175,7 +175,7 @@ TransactionIdDidCommit(TransactionId transactionId)
* True iff transaction associated with the identifier did abort.
*
* Note:
- * Assumes transaction identifier is valid.
+ * Assumes transaction identifier is valid and exists in clog.
*/
bool /* true if given transaction aborted */
TransactionIdDidAbort(TransactionId transactionId)
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 42fc351..5efbfbd 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -259,7 +259,28 @@ ReadNewTransactionId(void)
}
/*
- * Determine the last safe XID to allocate given the currently oldest
+ * Advance the cluster-wide value for the oldest valid clog entry.
+ *
+ * We must acquire CLogTruncationLock to advance the oldestClogXid. It's not
+ * necessary to hold the lock during the actual clog truncation, only when we
+ * advance the limit, as code looking up arbitrary xids is required to hold
+ * CLogTruncationLock from when it tests oldestClogXid through to when it
+ * completes the clog lookup.
+ */
+void
+AdvanceOldestClogXid(TransactionId oldest_datfrozenxid)
+{
+ LWLockAcquire(CLogTruncationLock, LW_EXCLUSIVE);
+ if (TransactionIdPrecedes(ShmemVariableCache->oldestClogXid,
+ oldest_datfrozenxid))
+ {
+ ShmemVariableCache->oldestClogXid = oldest_datfrozenxid;
+ }
+ LWLockRelease(CLogTruncationLock);
+}
+
+/*
+ * Determine the last safe XID to allocate using the currently oldest
* datfrozenxid (ie, the oldest XID that might exist in any database
* of our cluster), and the OID of the (or a) database with that value.
*/
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 9480377..fbdff55 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -5016,6 +5016,7 @@ BootStrapXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestClogXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB, true);
SetCommitTsLimit(InvalidTransactionId, InvalidTransactionId);
@@ -6622,6 +6623,7 @@ StartupXLOG(void)
ShmemVariableCache->nextOid = checkPoint.nextOid;
ShmemVariableCache->oidCount = 0;
MultiXactSetNextMXact(checkPoint.nextMulti, checkPoint.nextMultiOffset);
+ AdvanceOldestClogXid(checkPoint.oldestXid);
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
SetMultiXactIdLimit(checkPoint.oldestMulti, checkPoint.oldestMultiDB, true);
SetCommitTsLimit(checkPoint.oldestCommitTsXid,
@@ -8687,6 +8689,11 @@ CreateCheckPoint(int flags)
/*
* Get the other info we need for the checkpoint record.
+ *
+ * We don't need to save oldestClogXid in the checkpoint, it only matters
+ * for the short period in which clog is being truncated, and if we crash
+ * during that we'll redo the clog truncation and fix up oldestClogXid
+ * there.
*/
LWLockAcquire(XidGenLock, LW_SHARED);
checkPoint.nextXid = ShmemVariableCache->nextXid;
@@ -9616,6 +9623,10 @@ xlog_redo(XLogReaderState *record)
MultiXactAdvanceOldest(checkPoint.oldestMulti,
checkPoint.oldestMultiDB);
+ /*
+ * No need to set oldestClogXid here as well; it'll be set when we
+ * redo an xl_clog_truncate if it changed since initialization.
+ */
SetTransactionIdLimit(checkPoint.oldestXid, checkPoint.oldestXidDB);
/*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ff633fa..c4a0f89 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1194,7 +1194,7 @@ vac_truncate_clog(TransactionId frozenXID,
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
- TruncateCLOG(frozenXID);
+ TruncateCLOG(frozenXID, oldestxid_datoid);
TruncateCommitTs(frozenXID);
TruncateMultiXact(minMulti, minmulti_datoid);
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index cd8b08f..e6025ec 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -49,3 +49,4 @@ MultiXactTruncationLock 41
OldSnapshotTimeMapLock 42
BackendRandomLock 43
LogicalRepWorkerLock 44
+CLogTruncationLock 45
diff --git a/src/include/access/clog.h b/src/include/access/clog.h
index 2894bd5..60a9e11 100644
--- a/src/include/access/clog.h
+++ b/src/include/access/clog.h
@@ -28,6 +28,12 @@ typedef int XidStatus;
#define TRANSACTION_STATUS_ABORTED 0x02
#define TRANSACTION_STATUS_SUB_COMMITTED 0x03
+typedef struct xl_clog_truncate
+{
+ int pageno;
+ TransactionId oldestXact;
+ Oid oldestXactDb;
+} xl_clog_truncate;
extern void TransactionIdSetTreeStatus(TransactionId xid, int nsubxids,
TransactionId *subxids, XidStatus status, XLogRecPtr lsn);
@@ -42,7 +48,7 @@ extern void TrimCLOG(void);
extern void ShutdownCLOG(void);
extern void CheckPointCLOG(void);
extern void ExtendCLOG(TransactionId newestXact);
-extern void TruncateCLOG(TransactionId oldestXact);
+extern void TruncateCLOG(TransactionId oldestXact, Oid oldestxid_datoid);
/* XLOG stuff */
#define CLOG_ZEROPAGE 0x00
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 522c104..d25a2dd 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -134,6 +134,12 @@ typedef struct VariableCacheData
*/
TransactionId latestCompletedXid; /* newest XID that has committed or
* aborted */
+
+ /*
+ * These fields are protected by CLogTruncationLock
+ */
+ TransactionId oldestClogXid; /* oldest it's safe to look up in clog */
+
} VariableCacheData;
typedef VariableCacheData *VariableCache;
@@ -173,6 +179,7 @@ extern TransactionId GetNewTransactionId(bool isSubXact);
extern TransactionId ReadNewTransactionId(void);
extern void SetTransactionIdLimit(TransactionId oldest_datfrozenxid,
Oid oldest_datoid);
+extern void AdvanceOldestClogXid(TransactionId oldest_datfrozenxid);
extern bool ForceTransactionIdLimitUpdate(void);
extern Oid GetNewObjectId(void);
--
2.5.5
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 2ba56472d01e3d9db03d21faed68829998dd4b76 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH 2/3] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 +++++++++
src/backend/utils/adt/txid.c | 120 +++++++++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++++
src/test/regress/sql/txid.sql | 38 ++++++++++++
5 files changed, 255 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9408a25..c8639b8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17523,6 +17523,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17573,6 +17577,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17643,6 +17652,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..2b64a58 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,67 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ *
+ * The caller must hold CLogTruncationLock since it's dealing with arbitrary
+ * XIDs, and must continue to hold it until it's done with any clog lookups
+ * relating to those XIDs.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestClogXid is protected by CLogTruncationLock,
+ * but we don't acquire that lock here. Instead, we require the caller to
+ * acquire it, because the caller is presumably going to look up the
+ * returned XID. If we took and released the lock within this function, a
+ * CLOG truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(CLogTruncationLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestClogXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestClogXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +417,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +724,57 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must protect against concurrent truncation of clog entries to avoid
+ * an I/O error on SLRU lookup.
+ */
+ LWLockAcquire(CLogTruncationLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(CLogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 836d6ff..39bd295 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4975,6 +4975,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 March 2017 at 03:35, Craig Ringer <craig@2ndquadrant.com> wrote:
On 22 March 2017 at 09:49, Craig Ringer <craig@2ndquadrant.com> wrote:
Overall, though, I think that 0001 looks far better than any previous
iteration. It's simple. It looks safe. It seems unlikely to break
anything that works now. Woo hoo!Funny that this started with "hey, here's a simple, non-invasive
function for looking up the status of an arbitrary xid".Changes made per discussion.
This looks good to me also. Does what we need it to do.
I was a little worried by possible performance of new lock, but its
not intended to be run frequently, so its OK.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Wed, Mar 22, 2017 at 3:13 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
Changes made per discussion.
This looks good to me also. Does what we need it to do.
I was a little worried by possible performance of new lock, but its
not intended to be run frequently, so its OK.
Agreed.
Reviewing 0002:
+ if (!TransactionIdIsValid(xid))
+ {
+ LWLockRelease(XidGenLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is an invalid xid",
+ xid_with_epoch)));
+ }It's unnecessary to release LWLockRelease() before throwing an error,
and it's also wrong because we haven't acquired XidGenLock in this
code path. But maybe it would be better to just remove this entirely
and instead have TransactionIdInRecentPast return false for
InvalidTransactionId. Then we'd avoid adding a translatable message
for a case that is basically harmless to allow.
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+
+ /*
+ * can't test TransactionIdIsInProgress here or we race with
+ * concurrent commit/abort. There's no point anyway, since it
+ * might then commit/abort just after we check.
+ */
+ status = gettext_noop("in progress");I am not sure this is going to do the right thing for transactions
that are aborted by a crash without managing to write an abort record.
It seems that the first check will say the transaction isn't in
progress, and the second and third checks will say it isn't either
committed or aborted since, if I am reading this correctly, they just
read clog directly. Compare HeapTupleSatisfiesMVCC, which assumes
that a not-in-progress, not-committed transaction must necessarily
have aborted. I think your comment is pointing to a real issue,
though. It seems like what might be needed is to add one more check.
Before where you have the "else" clause, check if the XID is old, e.g.
precedes our snapshot's xmin. If so, it must be committed or aborted
and, since it didn't commit, it aborted. If not, it must've changed
from in progress to not-in-progress just as we were in the midst
checking, so labeling it as in progress is fine.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 22 March 2017 at 17:41, Robert Haas <robertmhaas@gmail.com> wrote:
+ if (TransactionIdIsCurrentTransactionId(xid)) + status = gettext_noop("in progress"); + else if (TransactionIdDidCommit(xid)) + status = gettext_noop("committed"); + else if (TransactionIdDidAbort(xid)) + status = gettext_noop("aborted"); + else + + /* + * can't test TransactionIdIsInProgress here or we race with + * concurrent commit/abort. There's no point anyway, since it + * might then commit/abort just after we check. + */ + status = gettext_noop("in progress");I am not sure this is going to do the right thing for transactions
that are aborted by a crash without managing to write an abort record.
Yes, perhaps we should report that state as "aborted - incomplete".
And of course, we might return "subcommitted" also, which could
technically also be an abort in some cases, so we'd need to do a wait
loop on that.
Which makes me think it would be confusing to say "in progress" for
when it is our current xid, since the user might wait until it is
complete and then wait forever. Prefer it if it said "in progress -
current transaction"
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Wed, Mar 22, 2017 at 2:08 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 22 March 2017 at 17:41, Robert Haas <robertmhaas@gmail.com> wrote:
+ if (TransactionIdIsCurrentTransactionId(xid)) + status = gettext_noop("in progress"); + else if (TransactionIdDidCommit(xid)) + status = gettext_noop("committed"); + else if (TransactionIdDidAbort(xid)) + status = gettext_noop("aborted"); + else + + /* + * can't test TransactionIdIsInProgress here or we race with + * concurrent commit/abort. There's no point anyway, since it + * might then commit/abort just after we check. + */ + status = gettext_noop("in progress");I am not sure this is going to do the right thing for transactions
that are aborted by a crash without managing to write an abort record.Yes, perhaps we should report that state as "aborted - incomplete".
And of course, we might return "subcommitted" also, which could
technically also be an abort in some cases, so we'd need to do a wait
loop on that.
I actually don't think those are things we should expose to users.
They're internal implementation details. The user had better not care
whether an abort was the type of abort that wrote an abort record or
the type that didn't.
Which makes me think it would be confusing to say "in progress" for
when it is our current xid, since the user might wait until it is
complete and then wait forever. Prefer it if it said "in progress -
current transaction"
Hmm, or just "current transaction", maybe?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 March 2017 at 02:08, Simon Riggs <simon@2ndquadrant.com> wrote:
And of course, we might return "subcommitted" also, which could
technically also be an abort in some cases, so we'd need to do a wait
loop on that.
Users generally don't see subxact IDs, so it wasn't something I was
overly concerned by. Most notably, txid_current() doesn't report
them.
Users who want to know the commit status of an xact that had a commit
in-flight when they lost access to it due to server crash, network
loss, etc aren't going to care about subxacts. If you lose your
connection after a RELEASE SAVEPOINT you know the outer xact will get
aborted or the state of the individual subxacts.
They're visible in heap tuples, but you can only see uncommitted heap
tuples from your own top-level xid. For anything else, if you can see
the xid in xmin you know it committed. There isn't really any reason
you'd be looking up tuple xids with txid_status anyway, and if you did
you'd have to pay attention to mess like multixacts in xmax ... but
you can't tell from the xid alone if it's a multixact or not, so this
doesn't make sense with xids that could be multis anyway.
If we're going to handle subxacts specially, we should probably report
them as "sub-committed" if we find that the current xid is a committed
subxact member of an outer xact that is still in-progress. But IMO
it's pretty pointless since you won't be dealing with subxact IDs in
the application anyway.
Which makes me think it would be confusing to say "in progress" for
when it is our current xid, since the user might wait until it is
complete and then wait forever. Prefer it if it said "in progress -
current transaction"
I'm fine with "current transaction". Though I think it's kind of a
moot point as I don't see any reason for an application to ever be
doing the equivalent of
txid_status(txid_current())
in the first place. But it's harmless enough.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 March 2017 at 01:41, Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Mar 22, 2017 at 3:13 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
Changes made per discussion.
This looks good to me also. Does what we need it to do.
I was a little worried by possible performance of new lock, but its
not intended to be run frequently, so its OK.Agreed.
Reviewing 0002:
+ if (!TransactionIdIsValid(xid)) + { + LWLockRelease(XidGenLock); + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("transaction ID " UINT64_FORMAT " is an invalid xid", + xid_with_epoch))); + }It's unnecessary to release LWLockRelease() before throwing an error,
and it's also wrong because we haven't acquired XidGenLock in this
code path. But maybe it would be better to just remove this entirely
and instead have TransactionIdInRecentPast return false for
InvalidTransactionId. Then we'd avoid adding a translatable message
for a case that is basically harmless to allow.
Agreed, that's better.
+ if (TransactionIdIsCurrentTransactionId(xid)) + status = gettext_noop("in progress"); + else if (TransactionIdDidCommit(xid)) + status = gettext_noop("committed"); + else if (TransactionIdDidAbort(xid)) + status = gettext_noop("aborted"); + else + + /* + * can't test TransactionIdIsInProgress here or we race with + * concurrent commit/abort. There's no point anyway, since it + * might then commit/abort just after we check. + */ + status = gettext_noop("in progress");I am not sure this is going to do the right thing for transactions
that are aborted by a crash without managing to write an abort record.
You're right. It'll report in-progress, since the clog entry will
still be 0. We don't appear to explicitly update the clog during
recovery.
Funny, I got so focused on the clog access safety stuff in the end
that I missed the bigger picture.
Given that part of the point of this is xact status after crash,
that's kind of important. I've added a TAP test for this, as part of a
new test set, "011_crash_recovery.pl". The recovery tests don't really
have much on crash behaviour at the moment so might as well start it
and add more as we go. It doesn't make sense to add a whole new top
level TAP test just for txid_status.
(I *love* the TAP framework now, it took 10 mins to write a test
that's trivial to repeat in 5 seconds per run for this).
It seems that the first check will say the transaction isn't in
progress, and the second and third checks will say it isn't either
committed or aborted since, if I am reading this correctly, they just
read clog directly.
Right. They're not concerned with subxacts or multixacts.
Compare HeapTupleSatisfiesMVCC, which assumes
that a not-in-progress, not-committed transaction must necessarily
have aborted.
Right.
We don't have a HeapTuple here, of course, so we can't test flags.
Most importantly this means we can't handle multixacts. If we ever did
decide to expose multixacts as bigint epoch-qualified xids for general
users, we'd probably reserve the high bit of the epoch the bigint xid
representation for the equivalent of HEAP_XMAX_IS_MULTI, and maybe
it's worth reserving that bit now and ERROR'ing if we find it set. But
right now we never produce a bigint xid with a multixact.
I wonder if a note in the docs warning not to cast xid to bigint is
worth it. Probably not. You have to cast xid::text::bigint which is
kind of a hint, plus it doesn't make sense to test txid_status() for
tuples' xmin/xmax . I guess you might use it with xids from
pg_stat_activity, but there's not much point when you can just look at
pg_stat_activity again to see if the proc of interest is still there
and has the same xid.
Anyway...
I think your comment is pointing to a real issue,
though. It seems like what might be needed is to add one more check.
Before where you have the "else" clause, check if the XID is old, e.g.
precedes our snapshot's xmin. If so, it must be committed or aborted
and, since it didn't commit, it aborted. If not, it must've changed
from in progress to not-in-progress just as we were in the midst
checking, so labeling it as in progress is fine.
That seems clear and sensible.
XidInMVCCSnapshot simply tests TransactionIdPrecedes(xid,
snapshot->xmin), as does HeapTupleSatisfiesHistoricMVCC . It seems
reasonable enough to just examine the snapshot xmin directly in
txid_status too; it's already done in replication/logical/snapbuild.c
and lmgr/predicate.c.
We're running in an SQL-called function so we'll have a good snapshot
and GetSnapshotData will have run, so GetActiveSnapshot()->xmin should
be sufficient.
Amended patch attached, with added TAP test included.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0001-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From 25ee92d26ad50a3e62104de37ccdb2bbb670f3fe Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++++
src/backend/utils/adt/txid.c | 132 +++++++++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/regress/expected/txid.out | 68 +++++++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++++
5 files changed, 267 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9408a25..c8639b8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17523,6 +17523,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17573,6 +17577,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17643,6 +17652,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..5c64e32 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,70 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ *
+ * The caller must hold CLogTruncationLock since it's dealing with arbitrary
+ * XIDs, and must continue to hold it until it's done with any clog lookups
+ * relating to those XIDs.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ if (!TransactionIdIsValid(xid))
+ return false;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestClogXid is protected by CLogTruncationLock,
+ * but we don't acquire that lock here. Instead, we require the caller to
+ * acquire it, because the caller is presumably going to look up the
+ * returned XID. If we took and released the lock within this function, a
+ * CLOG truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(CLogTruncationLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestClogXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestClogXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +420,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +727,66 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ *
+ * The passed epoch-qualified xid is treated as a normal xid, not a
+ * multixact id.
+ *
+ * If it points to a committed subxact the result is the subxact status even
+ * though the parent xact may still be in progress or may have aborted.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must protect against concurrent truncation of clog entries to avoid
+ * an I/O error on SLRU lookup.
+ */
+ LWLockAcquire(CLogTruncationLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ Assert(TransactionIdIsValid(xid));
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ {
+ /*
+ * The xact is not marked as either committed or aborted in clog.
+ *
+ * It could be a transaction that ended without updating clog or
+ * writing an abort record due to a crash. We can safely assume
+ * it's aborted if it isn't committed and is older than our
+ * snapshot xmin.
+ *
+ * Otherwise it must be in-progress (or have been at the time
+ * we checked commit/abort status).
+ */
+ if (TransactionIdPrecedes(xid, GetActiveSnapshot()->xmin))
+ status = gettext_noop("aborted");
+ else
+ status = gettext_noop("in progress");
+ }
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(CLogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 836d6ff..39bd295 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4975,6 +4975,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 23 March 2017 at 11:25, Craig Ringer <craig@2ndquadrant.com> wrote:
Amended patch attached, with added TAP test included.
Managed to omit it, sigh.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Changes made per discussion.
Committed 0001.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 24 March 2017 at 02:29, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Changes made per discussion.
Committed 0001.
Much appreciated.
Here's the 2nd patch rebased on top of master, with the TAP test
included this time. Looks ready to go.
I really appreciate the time you've taken to look at this. Point me at
anything from your team you want some outside review on.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchtext/x-patch; charset=US-ASCII; name=0002-Introduce-txid_status-bigint-to-get-status-of-an-xac.patchDownload
From eca78655c966613b1b98baf9049b5c4d519d375a Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++
src/backend/utils/adt/txid.c | 132 ++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/recovery/t/011_crash_recovery.pl | 46 +++++++++++
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
6 files changed, 313 insertions(+)
create mode 100644 src/test/recovery/t/011_crash_recovery.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 64f86ce..356fd33 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17523,6 +17523,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17573,6 +17577,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17643,6 +17652,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..5c64e32 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,70 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ *
+ * The caller must hold CLogTruncationLock since it's dealing with arbitrary
+ * XIDs, and must continue to hold it until it's done with any clog lookups
+ * relating to those XIDs.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ if (!TransactionIdIsValid(xid))
+ return false;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestClogXid is protected by CLogTruncationLock,
+ * but we don't acquire that lock here. Instead, we require the caller to
+ * acquire it, because the caller is presumably going to look up the
+ * returned XID. If we took and released the lock within this function, a
+ * CLOG truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(CLogTruncationLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestClogXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestClogXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +420,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +727,66 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ *
+ * The passed epoch-qualified xid is treated as a normal xid, not a
+ * multixact id.
+ *
+ * If it points to a committed subxact the result is the subxact status even
+ * though the parent xact may still be in progress or may have aborted.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must protect against concurrent truncation of clog entries to avoid
+ * an I/O error on SLRU lookup.
+ */
+ LWLockAcquire(CLogTruncationLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ Assert(TransactionIdIsValid(xid));
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ {
+ /*
+ * The xact is not marked as either committed or aborted in clog.
+ *
+ * It could be a transaction that ended without updating clog or
+ * writing an abort record due to a crash. We can safely assume
+ * it's aborted if it isn't committed and is older than our
+ * snapshot xmin.
+ *
+ * Otherwise it must be in-progress (or have been at the time
+ * we checked commit/abort status).
+ */
+ if (TransactionIdPrecedes(xid, GetActiveSnapshot()->xmin))
+ status = gettext_noop("aborted");
+ else
+ status = gettext_noop("in progress");
+ }
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(CLogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0d18ab8..a66d045 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4978,6 +4978,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/recovery/t/011_crash_recovery.pl b/src/test/recovery/t/011_crash_recovery.pl
new file mode 100644
index 0000000..e5d1a0f
--- /dev/null
+++ b/src/test/recovery/t/011_crash_recovery.pl
@@ -0,0 +1,46 @@
+#
+# Tests relating to PostgreSQL crash recovery and redo
+#
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 3;
+
+my $node = get_new_node('master');
+$node->init(allows_streaming => 1);
+$node->start;
+
+my ($stdin, $stdout, $stderr) = ('', '', '');
+
+# Ensure that txid_status reports 'aborted' for xacts
+# that were in-progress during crash. To do that, we need
+# an xact to be in-progress when we crash and we need to know
+# its xid.
+my $tx = IPC::Run::start(
+ ['psql', '-qAt', '-v', 'ON_ERROR_STOP=1', '-f', '-', '-d', $node->connstr('postgres')],
+ '<', \$stdin, '>', \$stdout, '2>', \$stderr);
+$stdin .= q[
+BEGIN;
+CREATE TABLE mine(x integer);
+SELECT txid_current();
+];
+$tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/;
+
+# Status should be in-progress
+my $xid = $stdout;
+chomp($xid);
+
+is($node->safe_psql('postgres', qq[SELECT txid_status('$xid');]), 'in progress', 'own xid is in-progres');
+
+# Crash and restart the postmaster
+$node->stop('immediate');
+$node->start;
+
+# Make sure we really got a new xid
+cmp_ok($node->safe_psql('postgres', 'SELECT txid_current()'), '>', $xid,
+ 'new xid after restart is greater');
+# and make sure we show the in-progress xact as aborted
+is($node->safe_psql('postgres', qq[SELECT txid_status('$xid');]), 'aborted', 'xid is aborted after crash');
+
+$tx->kill_kill;
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Fri, Mar 24, 2017 at 2:27 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 24 March 2017 at 02:29, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Changes made per discussion.
Committed 0001.
Much appreciated.
Here's the 2nd patch rebased on top of master, with the TAP test
included this time. Looks ready to go.
Committed.
I really appreciate the time you've taken to look at this. Point me at
anything from your team you want some outside review on.
Thanks, that is a valuable offer which I will be pleased to accept!
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 3/24/17 02:27, Craig Ringer wrote:
On 24 March 2017 at 02:29, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Changes made per discussion.
Committed 0001.
Much appreciated.
Here's the 2nd patch rebased on top of master, with the TAP test
included this time. Looks ready to go.
I'm experiencing hangs in the new t/011_crash_recovery.pl test. It
seems to hang after the first call to SELECT txid_current();.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 25 March 2017 at 07:31, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 3/24/17 02:27, Craig Ringer wrote:
On 24 March 2017 at 02:29, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Changes made per discussion.
Committed 0001.
Much appreciated.
Here's the 2nd patch rebased on top of master, with the TAP test
included this time. Looks ready to go.I'm experiencing hangs in the new t/011_crash_recovery.pl test. It
seems to hang after the first call to SELECT txid_current();.
if you add
note "txid is $xid";
after
+chomp($xid);
does it report the xid?
Alternately, can you see a 'psql' process and a backend doing a SELECT
when it stops progressing?
I'm wondering if this is a perl version/platform issue around
$tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/;
where we're not recognising the required output from psql when we get it.
What's in src/test/recovery/tmp_check/log/regress_log_011* ?
I couldn't use PostgresNode::psql or PostgresNode::safe_psql here
because the session must stay open.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
I'm wondering if this is a perl version/platform issue around
$tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/;
where we're not recognising the required output from psql when we get it.
What's in src/test/recovery/tmp_check/log/regress_log_011* ?
I couldn't use PostgresNode::psql or PostgresNode::safe_psql here
because the session must stay open.
The problem was that psql needs to be called with -X. Fix committed.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 3/25/17 12:12 AM, Peter Eisentraut wrote:
I'm wondering if this is a perl version/platform issue around
$tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/;
where we're not recognising the required output from psql when we get it.
What's in src/test/recovery/tmp_check/log/regress_log_011* ?
I couldn't use PostgresNode::psql or PostgresNode::safe_psql here
because the session must stay open.The problem was that psql needs to be called with -X. Fix committed.
It's not clear to me what remains to be done on this patch. I feel, at
the least, that patches 3 and 4 need to be rebased and the status set
back to "Needs Review".
This thread has been idle for six days. Please respond with a new patch
by 2017-04-04 00:00 AoE (UTC-12) or this submission will be marked
"Returned with Feedback".
--
-David
david@pgmasters.net
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 31 Mar. 2017 22:31, "David Steele" <david@pgmasters.net> wrote:
On 3/25/17 12:12 AM, Peter Eisentraut wrote:
I'm wondering if this is a perl version/platform issue around
$tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/;
where we're not recognising the required output from psql when we get it.
What's in src/test/recovery/tmp_check/log/regress_log_011* ?
I couldn't use PostgresNode::psql or PostgresNode::safe_psql here
because the session must stay open.The problem was that psql needs to be called with -X. Fix committed.
It's not clear to me what remains to be done on this patch. I feel, at the
least, that patches 3 and 4 need to be rebased and the status set back to
"Needs Review".
This thread has been idle for six days. Please respond with a new patch by
2017-04-04 00:00 AoE (UTC-12) or this submission will be marked "Returned
with Feedback".
I consider the feature complete and committed.
Patch 3 is optional and per discussion buried upthread we generally agreed
that it'd be better to replace most user visible uses of the 'xid' data
type with a new epoch extended 'xid64' or similar.
Patch 4, txid_incinerate, has never been intended for commit. It's a
testing tool.
Patches 1 and 2 were the key parts and thanks to Robert's helpful review,
advice and edits they're committed now.
Committed, done. Yay.
Re: [PATCH] Transaction traceability - txid_status(bigint)
On 3/31/17 10:46 AM, Craig Ringer wrote:
Patches 1 and 2 were the key parts and thanks to Robert's helpful
review, advice and edits they're committed now.Committed, done. Yay.
Excellent. I have marked this a "Committed" by Robert.
One down...
--
-David
david@pgmasters.net
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCH] Transaction traceability - txid_status(bigint)
On Fri, Mar 31, 2017 at 11:08 AM, David Steele <david@pgmasters.net> wrote:
On 3/31/17 10:46 AM, Craig Ringer wrote:
Patches 1 and 2 were the key parts and thanks to Robert's helpful
review, advice and edits they're committed now.Committed, done. Yay.
Excellent. I have marked this a "Committed" by Robert.
One down...
...71 to go?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers