pgsql: logical decoding: Tell reorderbuffer about all xids.

Started by Andres Freundover 10 years ago5 messagescomitters
Jump to latest
#1Andres Freund
andres@anarazel.de

logical decoding: Tell reorderbuffer about all xids.

Logical decoding's reorderbuffer keeps transactions in an LSN ordered
list for efficiency. To make that's efficiently possible upper-level
xids are forced to be logged before nested subtransaction xids. That
only works though if these records are all looked at: Unfortunately we
didn't do so for e.g. row level locks, which are otherwise uninteresting
for logical decoding.

This could lead to errors like:
"ERROR: subxact logged without previous toplevel record".

It's not sufficient to just look at row locking records, the xid could
appear first due to a lot of other types of records (which will trigger
the transaction to be marked logged with MarkCurrentTransactionIdLoggedIfAny).
So invent infrastructure to tell reorderbuffer about xids seen, when
they'd otherwise not pass through reorderbuffer.c.

Reported-By: Jarred Ward
Bug: #13844
Discussion: 20160105033249.1087.66040@wrigleys.postgresql.org
Backpatch: 9.4, where logical decoding was added

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/d9e903f3cbbd00c7ba7d4974e6852c3d2cbf4447

Modified Files
--------------
contrib/test_decoding/Makefile | 4 +--
contrib/test_decoding/expected/xact.out | 42 +++++++++++++++++++++++++
contrib/test_decoding/sql/xact.sql | 22 +++++++++++++
src/backend/replication/logical/decode.c | 28 ++++++++++++++++-
src/backend/replication/logical/reorderbuffer.c | 21 ++++++++-----
src/backend/replication/logical/snapbuild.c | 6 +---
src/include/replication/reorderbuffer.h | 2 +-
7 files changed, 108 insertions(+), 17 deletions(-)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#1)
Re: pgsql: logical decoding: Tell reorderbuffer about all xids.

Andres Freund <andres@anarazel.de> writes:

logical decoding: Tell reorderbuffer about all xids.

One or another of these logical-decoding fixes is making all the ppc64
buildfarm members unhappy.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#3Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#2)
Re: pgsql: logical decoding: Tell reorderbuffer about all xids.

On 2016-03-05 22:49:18 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

logical decoding: Tell reorderbuffer about all xids.

One or another of these logical-decoding fixes is making all the ppc64
buildfarm members unhappy.

Just saw, looking into it. Might take me till tomorrow morning to finish.

It's very likely the last of the three commits, but we'll see.

Andres

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#4Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#2)
Re: pgsql: logical decoding: Tell reorderbuffer about all xids.

On 2016-03-05 22:49:18 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

logical decoding: Tell reorderbuffer about all xids.

One or another of these logical-decoding fixes is making all the ppc64
buildfarm members unhappy.

http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=termite&amp;dt=2016-03-07%2000%3A37%3A22

Looks like my attempt at fixing this was successful. While debugging
this I found some minor further improvements, which I'll fix together
with the issue coverity spotted.

Andres

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#4)
Re: pgsql: logical decoding: Tell reorderbuffer about all xids.

Andres Freund <andres@anarazel.de> writes:

On 2016-03-05 22:49:18 -0500, Tom Lane wrote:

One or another of these logical-decoding fixes is making all the ppc64
buildfarm members unhappy.

http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=termite&amp;dt=2016-03-07%2000%3A37%3A22

Looks like my attempt at fixing this was successful.

Yeah, gaur is happy too. No ARM reports yet though :-(

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers