pgsql: Don't force-assign transaction id when exporting a snapshot.
Don't force-assign transaction id when exporting a snapshot.
Previously we required every exported transaction to have an xid
assigned. That was used to check that the exporting transaction is
still running, which in turn is needed to guarantee that that
necessary rows haven't been removed in between exporting and importing
the snapshot.
The exported xid caused unnecessary problems with logical decoding,
because slot creation has to wait for all concurrent xid to finish,
which in turn serializes concurrent slot creation. It also
prohibited snapshots to be exported on hot-standby replicas.
Instead export the virtual transactionid, which avoids the unnecessary
serialization and the inability to export snapshots on standbys. This
changes the file name of the exported snapshot, but since we never
documented what that one means, that seems ok.
Author: Petr Jelinek, slightly editorialized by me
Reviewed-By: Andres Freund
Discussion: /messages/by-id/f598b4b8-8cd7-0d54-0939-adda763d8c34@2ndquadrant.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/6c2003f8a1bbc7c192a2e83ec51581c018aa162f
Modified Files
--------------
doc/src/sgml/ref/set_transaction.sgml | 6 +-
src/backend/replication/logical/snapbuild.c | 14 ++--
src/backend/storage/ipc/procarray.c | 11 ++-
src/backend/storage/lmgr/predicate.c | 26 +++---
src/backend/utils/time/snapmgr.c | 119 ++++++++++++++++++----------
src/include/storage/predicate.h | 4 +-
src/include/storage/procarray.h | 2 +-
7 files changed, 115 insertions(+), 67 deletions(-)
--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers
On 14/06/17 20:57, Andres Freund wrote:
Don't force-assign transaction id when exporting a snapshot.
Previously we required every exported transaction to have an xid
assigned. That was used to check that the exporting transaction is
still running, which in turn is needed to guarantee that that
necessary rows haven't been removed in between exporting and importing
the snapshot.The exported xid caused unnecessary problems with logical decoding,
because slot creation has to wait for all concurrent xid to finish,
which in turn serializes concurrent slot creation. It also
prohibited snapshots to be exported on hot-standby replicas.Instead export the virtual transactionid, which avoids the unnecessary
serialization and the inability to export snapshots on standbys. This
changes the file name of the exported snapshot, but since we never
documented what that one means, that seems ok.
This commit has side effect that it makes it possible to export
snapshots on the standbys. This makes it possible to do pg_dump -j on
standby with consistent snapshot. Here is one line patch (+ doc update)
which allows doing that when pg_dumping from PG10+.
I also wonder if it should be mentioned in release notes. If the
attached patch would make it into PG10 it would be no brainer to mention
it as feature under pg_dump section, but exporting snapshots alone I am
not sure about.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Allow-pg_dump-j-on-standby-when-dumping-from-PG10.patchtext/x-patch; name=0001-Allow-pg_dump-j-on-standby-when-dumping-from-PG10.patchDownload+11-10
On 8/12/17 07:32, Petr Jelinek wrote:
This commit has side effect that it makes it possible to export
snapshots on the standbys. This makes it possible to do pg_dump -j on
standby with consistent snapshot. Here is one line patch (+ doc update)
which allows doing that when pg_dumping from PG10+.I also wonder if it should be mentioned in release notes. If the
attached patch would make it into PG10 it would be no brainer to mention
it as feature under pg_dump section, but exporting snapshots alone I am
not sure about.
Any other opinions on this patch?
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2017-08-14 13:55:29 -0400, Peter Eisentraut wrote:
On 8/12/17 07:32, Petr Jelinek wrote:
This commit has side effect that it makes it possible to export
snapshots on the standbys. This makes it possible to do pg_dump -j on
standby with consistent snapshot. Here is one line patch (+ doc update)
which allows doing that when pg_dumping from PG10+.I also wonder if it should be mentioned in release notes. If the
attached patch would make it into PG10 it would be no brainer to mention
it as feature under pg_dump section, but exporting snapshots alone I am
not sure about.Any other opinions on this patch?
I'd be oh so very slightly inclined to push this, including the modified
pg_dump check. We're going to be on the hook for supporting snapshots on
standbys anyway, unless we put in additional unnecessary error check.
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 8/14/17 13:57, Andres Freund wrote:
On 2017-08-14 13:55:29 -0400, Peter Eisentraut wrote:
On 8/12/17 07:32, Petr Jelinek wrote:
This commit has side effect that it makes it possible to export
snapshots on the standbys. This makes it possible to do pg_dump -j on
standby with consistent snapshot. Here is one line patch (+ doc update)
which allows doing that when pg_dumping from PG10+.I also wonder if it should be mentioned in release notes. If the
attached patch would make it into PG10 it would be no brainer to mention
it as feature under pg_dump section, but exporting snapshots alone I am
not sure about.Any other opinions on this patch?
I'd be oh so very slightly inclined to push this, including the modified
pg_dump check. We're going to be on the hook for supporting snapshots on
standbys anyway, unless we put in additional unnecessary error check.
committed to 10 and master
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers