Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.
Hi all,
I faced suspicious behaviour on hot standby server related to visibility map.
The scenario is,
1. Create table and check internal of visibility map on master server.
postgres(1)=# create table hoge (col int);
CREATE TABLE
postgres(1)=# insert into hoge select generate_series(1,10);
INSERT 0 10
postgres(1)=# select * from pg_visibility('hoge');
blkno | all_visible | all_frozen | pd_all_visible
-------+-------------+------------+----------------
0 | f | f | f
(1 row)
2. Check internal of visibility map on standby server.
postgres(1)=# select * from pg_visibility('hoge');
blkno | all_visible | all_frozen | pd_all_visible
-------+-------------+------------+----------------
0 | f | f | f
(1 row)
3. Do VACUUM on master server.
postgres(1)=# vacuum hoge;
VACUUM
postgres(1)=# select * from pg_visibility('hoge');
blkno | all_visible | all_frozen | pd_all_visible
-------+-------------+------------+----------------
0 | t | f | t
(1 row)
4. Check internal of visibility map on standby server again.
** Note that the we use the connection we established at #2 again **
postgres(1)=# select * from pg_visibility('hoge');
blkno | all_visible | all_frozen | pd_all_visible
-------+-------------+------------+----------------
0 | f | f | t
(1 row)
Even on standby server, the result should be (t,f,t), but returned (f,f,t)
(XLOG_HEAP2_VISIBLE record actually has been reached to standby, and
has been surely applied)
As a result of looked into code around the recvoery, ISTM that the
cause is related to relation cache clear.
In heap_xlog_visible, if the standby server receives WAL record then
relation cache is eventually cleared in vm_extend, but If standby
server receives FPI then relation cache would not be cleared.
For example, after I applied attached patch to HEAD, (it might not be
right way but) this problem seems to be resolved.
Is this a bug? or not?
Regards,
--
Masahiko Sawada
Attachments:
heap_xlog_visible_invalidate_cache.patchapplication/octet-stream; name=heap_xlog_visible_invalidate_cache.patchDownload+7-3
On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote:
As a result of looked into code around the recvoery, ISTM that the
cause is related to relation cache clear.
In heap_xlog_visible, if the standby server receives WAL record then
relation cache is eventually cleared in vm_extend, but If standby
server receives FPI then relation cache would not be cleared.
For example, after I applied attached patch to HEAD, (it might not be
right way but) this problem seems to be resolved.Is this a bug? or not?
It's a bug. I don't expect it causes queries to return wrong answers, because
visibilitymap.c says "it's always safe to clear a bit in the map from
correctness point of view." (The bug makes a visibility map bit temporarily
appear to have been cleared.) I still call it a bug, because recovery
behavior becomes too difficult to verify when xlog replay produces conditions
that don't happen outside of recovery. Even if there's no way to get a wrong
query answer today, this would be too easy to break later. I wonder if we
make the same omission in other xlog replay functions. Similar omissions may
cause wrong query answers, even if this particular one does not.
Would you like to bisect for the commit, or at least the major release, at
which the bug first appeared?
I wonder if your discovery has any relationship to this recently-reported case
of insufficient smgr invalidation:
/messages/by-id/CAB7nPqSBFmh5cQjpRbFBp9Rkv1nF=Nh2o1FxKkJ6yvOBtvYDBA@mail.gmail.com
Thanks,
nm
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch <noah@leadboat.com> wrote:
On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote:
As a result of looked into code around the recvoery, ISTM that the
cause is related to relation cache clear.
In heap_xlog_visible, if the standby server receives WAL record then
relation cache is eventually cleared in vm_extend, but If standby
server receives FPI then relation cache would not be cleared.
For example, after I applied attached patch to HEAD, (it might not be
right way but) this problem seems to be resolved.Is this a bug? or not?
It's a bug. I don't expect it causes queries to return wrong answers, because
visibilitymap.c says "it's always safe to clear a bit in the map from
correctness point of view." (The bug makes a visibility map bit temporarily
appear to have been cleared.) I still call it a bug, because recovery
behavior becomes too difficult to verify when xlog replay produces conditions
that don't happen outside of recovery. Even if there's no way to get a wrong
query answer today, this would be too easy to break later. I wonder if we
make the same omission in other xlog replay functions. Similar omissions may
cause wrong query answers, even if this particular one does not.Would you like to bisect for the commit, or at least the major release, at
which the bug first appeared?I wonder if your discovery has any relationship to this recently-reported case
of insufficient smgr invalidation:
/messages/by-id/CAB7nPqSBFmh5cQjpRbFBp9Rkv1nF=Nh2o1FxKkJ6yvOBtvYDBA@mail.gmail.com
I'm not sure this bug has relationship to another issue you mentioned
but after further investigation, this bug seems to be reproduced even
on more older version.
At least I reproduced it at 9.0.0.
Regards,
--
Masahiko Sawada
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Mar 31, 2016 at 04:48:26PM +0900, Masahiko Sawada wrote:
On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch <noah@leadboat.com> wrote:
On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote:
As a result of looked into code around the recvoery, ISTM that the
cause is related to relation cache clear.
In heap_xlog_visible, if the standby server receives WAL record then
relation cache is eventually cleared in vm_extend, but If standby
server receives FPI then relation cache would not be cleared.
For example, after I applied attached patch to HEAD, (it might not be
right way but) this problem seems to be resolved.Is this a bug? or not?
It's a bug. I don't expect it causes queries to return wrong answers, because
visibilitymap.c says "it's always safe to clear a bit in the map from
correctness point of view." (The bug makes a visibility map bit temporarily
appear to have been cleared.) I still call it a bug, because recovery
behavior becomes too difficult to verify when xlog replay produces conditions
that don't happen outside of recovery. Even if there's no way to get a wrong
query answer today, this would be too easy to break later. I wonder if we
make the same omission in other xlog replay functions. Similar omissions may
cause wrong query answers, even if this particular one does not.Would you like to bisect for the commit, or at least the major release, at
which the bug first appeared?I wonder if your discovery has any relationship to this recently-reported case
of insufficient smgr invalidation:
/messages/by-id/CAB7nPqSBFmh5cQjpRbFBp9Rkv1nF=Nh2o1FxKkJ6yvOBtvYDBA@mail.gmail.comI'm not sure this bug has relationship to another issue you mentioned
but after further investigation, this bug seems to be reproduced even
on more older version.
At least I reproduced it at 9.0.0.
Would you try PostgreSQL 9.2.16? The visibility map was not crash safe and
had no correctness implications until 9.2. If 9.2 behaves this way, it's
almost certainly not a recent regression.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Apr 1, 2016 at 9:10 AM, Noah Misch <noah@leadboat.com> wrote:
On Thu, Mar 31, 2016 at 04:48:26PM +0900, Masahiko Sawada wrote:
On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch <noah@leadboat.com> wrote:
On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote:
As a result of looked into code around the recvoery, ISTM that the
cause is related to relation cache clear.
In heap_xlog_visible, if the standby server receives WAL record then
relation cache is eventually cleared in vm_extend, but If standby
server receives FPI then relation cache would not be cleared.
For example, after I applied attached patch to HEAD, (it might not be
right way but) this problem seems to be resolved.Is this a bug? or not?
It's a bug. I don't expect it causes queries to return wrong answers, because
visibilitymap.c says "it's always safe to clear a bit in the map from
correctness point of view." (The bug makes a visibility map bit temporarily
appear to have been cleared.) I still call it a bug, because recovery
behavior becomes too difficult to verify when xlog replay produces conditions
that don't happen outside of recovery. Even if there's no way to get a wrong
query answer today, this would be too easy to break later. I wonder if we
make the same omission in other xlog replay functions. Similar omissions may
cause wrong query answers, even if this particular one does not.Would you like to bisect for the commit, or at least the major release, at
which the bug first appeared?I wonder if your discovery has any relationship to this recently-reported case
of insufficient smgr invalidation:
/messages/by-id/CAB7nPqSBFmh5cQjpRbFBp9Rkv1nF=Nh2o1FxKkJ6yvOBtvYDBA@mail.gmail.comI'm not sure this bug has relationship to another issue you mentioned
but after further investigation, this bug seems to be reproduced even
on more older version.
At least I reproduced it at 9.0.0.Would you try PostgreSQL 9.2.16? The visibility map was not crash safe and
had no correctness implications until 9.2. If 9.2 behaves this way, it's
almost certainly not a recent regression.
Yeah, I reproduced it on 9.2.0 and 9.2.16, it's not recent regression.
The commit is 503c7305a1e379f95649eef1a694d0c1dbdc674a which
introduces crash-safe visibility map.
Regards,
--
Masahiko Sawada
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-03-31 01:02:06 -0400, Noah Misch wrote:
On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote:
As a result of looked into code around the recvoery, ISTM that the
cause is related to relation cache clear.
In heap_xlog_visible, if the standby server receives WAL record then
relation cache is eventually cleared in vm_extend, but If standby
server receives FPI then relation cache would not be cleared.
For example, after I applied attached patch to HEAD, (it might not be
right way but) this problem seems to be resolved.Is this a bug? or not?
It's a bug.
I agree it's a bug. If I understand correctly it's not really
visibilitymap related though:
Debugging shows that vm_extend() (on the master) correctly issues a
CacheInvalidateSmgr(), which ought to clear the smgr state on the
standby. But replay doesn't do anything like that. That kind of sounded
familiar. The issue is that vacuum doesn't assign an xid and thus
RecordTransactionCommit() doesn't emit a commit record, which in turn
means invalidation messages aren't sent to the standby.
Ugh.
We've recently discussed a very similar issue around
/messages/by-id/20160227002958.peftvmcx4dxwe244@alap3.anarazel.de
Unfortunately Simon over in that thread disagreed there about fixing
this by always emitting a commit record when nmsgs > 0 in
RecordTransactionCommit(). I think this thread is a pretty strong hint
that we actually should do so.
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund <andres@anarazel.de> wrote:
We've recently discussed a very similar issue around
/messages/by-id/20160227002958.peftvmcx4dxwe244@alap3.anarazel.deUnfortunately Simon over in that thread disagreed there about fixing
this by always emitting a commit record when nmsgs > 0 in
RecordTransactionCommit(). I think this thread is a pretty strong hint
that we actually should do so.
Yes. I'm pretty confident that you had the right idea there, and that
Simon's objection was off-base.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-04-14 11:50:58 -0400, Robert Haas wrote:
On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund <andres@anarazel.de> wrote:
We've recently discussed a very similar issue around
/messages/by-id/20160227002958.peftvmcx4dxwe244@alap3.anarazel.deUnfortunately Simon over in that thread disagreed there about fixing
this by always emitting a commit record when nmsgs > 0 in
RecordTransactionCommit(). I think this thread is a pretty strong hint
that we actually should do so.Yes. I'm pretty confident that you had the right idea there, and that
Simon's objection was off-base.
The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should be safe
to do so at commit time. Seems less invasive to backpatch than to either
support commit records without xids, or a separate record just
transporting invalidation messages.
Although a separate record with invalidations is going to be needed to
support logical decoding of running xacts anyway...
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund <andres@anarazel.de> wrote:
On 2016-04-14 11:50:58 -0400, Robert Haas wrote:
On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund <andres@anarazel.de> wrote:
We've recently discussed a very similar issue around
/messages/by-id/20160227002958.peftvmcx4dxwe244@alap3.anarazel.deUnfortunately Simon over in that thread disagreed there about fixing
this by always emitting a commit record when nmsgs > 0 in
RecordTransactionCommit(). I think this thread is a pretty strong hint
that we actually should do so.Yes. I'm pretty confident that you had the right idea there, and that
Simon's objection was off-base.The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should be safe
to do so at commit time. Seems less invasive to backpatch than to either
support commit records without xids, or a separate record just
transporting invalidation messages.
I agree that's better for back-patching. I hope it won't suck
performance-wise. In master, we might think of inventing something
new.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-04-15 13:07:19 -0400, Robert Haas wrote:
On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund <andres@anarazel.de> wrote:
On 2016-04-14 11:50:58 -0400, Robert Haas wrote:
On Wed, Apr 13, 2016 at 9:58 PM, Andres Freund <andres@anarazel.de> wrote:
We've recently discussed a very similar issue around
/messages/by-id/20160227002958.peftvmcx4dxwe244@alap3.anarazel.deUnfortunately Simon over in that thread disagreed there about fixing
this by always emitting a commit record when nmsgs > 0 in
RecordTransactionCommit(). I think this thread is a pretty strong hint
that we actually should do so.Yes. I'm pretty confident that you had the right idea there, and that
Simon's objection was off-base.The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should be safe
to do so at commit time. Seems less invasive to backpatch than to either
support commit records without xids, or a separate record just
transporting invalidation messages.I agree that's better for back-patching.
It's a bit ugly though, since we're at that stage pretty heavily
assuming there's no xid assigned (on the caller level)... I've toyed
with the idea of emitting a commit record that doesn't have an assigned
xid, but that'd require changes on the apply side, which would be uglier
than a new record type :(
I hope it won't suck performance-wise.
I can't see that be the case, there's not many places where we send
invalidation messages without an assigned xid. Running an instrumented
build with an appropriate wal level reveals about a handfull places.
Greetings,
Andres Freund
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund <andres@anarazel.de> wrote:
The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should be safe
to do so at commit time. Seems less invasive to backpatch than to either
support commit records without xids, or a separate record just
transporting invalidation messages.
I agree that's better for back-patching. I hope it won't suck
performance-wise. In master, we might think of inventing something
new.
I'm a little worried about whether this will break assumptions that
vacuum doesn't have an XID. I don't immediately see how it would,
but it seems a bit shaky.
I find it hard to believe that the act of assigning an XID would add
measurably to the cost of a vacuum, so Robert's performance concern
doesn't sound very exciting. If this works, I think it's fine to
adopt as a permanent solution.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Apr 15, 2016 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund <andres@anarazel.de> wrote:
The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should be safe
to do so at commit time. Seems less invasive to backpatch than to either
support commit records without xids, or a separate record just
transporting invalidation messages.I agree that's better for back-patching. I hope it won't suck
performance-wise. In master, we might think of inventing something
new.I'm a little worried about whether this will break assumptions that
vacuum doesn't have an XID. I don't immediately see how it would,
but it seems a bit shaky.
Actually, I think there's a bigger problem. If you manage to almost
wrap around the XID space, and the cluster shuts down, you are
guaranteed to be able to vacuum the whole cluster without actually
running out of XIDs. Forcing an XID to be assigned here would make
that uncertain - it would depend on how many tables you have versus
how many XIDs you have left. That seems unacceptable to me.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-04-15 13:44:27 -0400, Robert Haas wrote:
On Fri, Apr 15, 2016 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund <andres@anarazel.de> wrote:
The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should be safe
to do so at commit time. Seems less invasive to backpatch than to either
support commit records without xids, or a separate record just
transporting invalidation messages.I agree that's better for back-patching. I hope it won't suck
performance-wise. In master, we might think of inventing something
new.I'm a little worried about whether this will break assumptions that
vacuum doesn't have an XID. I don't immediately see how it would,
but it seems a bit shaky.Actually, I think there's a bigger problem. If you manage to almost
wrap around the XID space, and the cluster shuts down, you are
guaranteed to be able to vacuum the whole cluster without actually
running out of XIDs.
Currently you're unfortunately not, c.f.
/messages/by-id/CAPpHfdspOkmiQsxh-UZw2chM6dRMwXAJGEmmbmqYR=yvM7-s6A@mail.gmail.com
Forcing an XID to be assigned here would make
that uncertain - it would depend on how many tables you have versus
how many XIDs you have left. That seems unacceptable to me.
But I agree that that's a concern. I'm kinda leaning towards
introducing an invalidation WAL record for that case atm. It's quite
possible to force an xid to be assigned in xact.c, but it's certainly
not pretty (the least ugly way is to duplicate the
xactGetCommittedInvalidationMessages() call, and force an xid to be
assigned early in CommitTransaction().
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 15 April 2016 at 18:44, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Apr 15, 2016 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Apr 14, 2016 at 12:11 PM, Andres Freund <andres@anarazel.de>
wrote:
The easiest way to achieve that seems to be to just assign an xid if
that's the case; while it's not necessarily safe/efficient to do so at
the point the invalidation message was queued, I think it should besafe
to do so at commit time. Seems less invasive to backpatch than to
either
support commit records without xids, or a separate record just
transporting invalidation messages.I agree that's better for back-patching. I hope it won't suck
performance-wise. In master, we might think of inventing something
new.I'm a little worried about whether this will break assumptions that
vacuum doesn't have an XID. I don't immediately see how it would,
but it seems a bit shaky.Actually, I think there's a bigger problem. If you manage to almost
wrap around the XID space, and the cluster shuts down, you are
guaranteed to be able to vacuum the whole cluster without actually
running out of XIDs. Forcing an XID to be assigned here would make
that uncertain - it would depend on how many tables you have versus
how many XIDs you have left. That seems unacceptable to me.
I agree this is a bug, similar to the last one.
For me, the issue is that we need to do something to catch bugs. The
existing code does not have any test at all to check whether we are doing
the wrong thing - it just lets the wrong thing happen.
Fixing it by forcing a new behaviour might be the right thing to do going
forwards, but I don't much like the idea of forcing new behaviour in back
branches. It might fix this bug, but can easily cause others.
--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2016-04-15 19:59:06 +0100, Simon Riggs wrote:
For me, the issue is that we need to do something to catch bugs. The
existing code does not have any test at all to check whether we are doing
the wrong thing - it just lets the wrong thing happen.
But sending the message, without assigning an xid, *IS* the right thing
to do here? We shouldn't assign an xid, and we need to send the message
out to the standbys.
Fixing it by forcing a new behaviour might be the right thing to do going
forwards, but I don't much like the idea of forcing new behaviour in back
branches. It might fix this bug, but can easily cause others.
What's your alternative? Assigning an xid in the middle of vacuum isn't
ok, breaking vacuum isn't either?
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 15 April 2016 at 20:01, Andres Freund <andres@anarazel.de> wrote:
On 2016-04-15 19:59:06 +0100, Simon Riggs wrote:
For me, the issue is that we need to do something to catch bugs. The
existing code does not have any test at all to check whether we are doing
the wrong thing - it just lets the wrong thing happen.But sending the message, without assigning an xid, *IS* the right thing
to do here? We shouldn't assign an xid, and we need to send the message
out to the standbys.Fixing it by forcing a new behaviour might be the right thing to do going
forwards, but I don't much like the idea of forcing new behaviour in back
branches. It might fix this bug, but can easily cause others.What's your alternative? Assigning an xid in the middle of vacuum isn't
ok, breaking vacuum isn't either?
In my understanding we have two choices for this bug
1) assign an xid so it forces sending a message (message plus xid)
2) send a message without assigning an xid (message only)
(1) seems like it is worse for backpatching, IMHO, though I am willing to
hear other thoughts or options
--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Simon Riggs <simon@2ndQuadrant.com> writes:
In my understanding we have two choices for this bug
1) assign an xid so it forces sending a message (message plus xid)
2) send a message without assigning an xid (message only)
(1) seems like it is worse for backpatching, IMHO, though I am willing to
hear other thoughts or options
The problem with (1) is that it creates side-effects that could be bad;
Robert's already pointed out one close-to-show-stopper consequence,
and I have little confidence that there are not others. In general,
if we got here without assigning an xid, there's a reason.
I think the bottom line is that we misdesigned the WAL representation
by assuming that this sort of info could always be piggybacked on a
transaction commit record. It's time to fix that.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-04-15 15:26:17 -0400, Tom Lane wrote:
I think the bottom line is that we misdesigned the WAL representation
by assuming that this sort of info could always be piggybacked on a
transaction commit record. It's time to fix that.
I think we got to piggyback it onto a commit record, as long as there's
one. Otherwise it's going to be more complex (queuing messages when
reading an inval record) and slower (more wal records). I can easily
develop a patch for that, the question is what we do on the back
branches...
Greetings,
Andres Freund
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Andres Freund wrote:
On 2016-04-15 15:26:17 -0400, Tom Lane wrote:
I think the bottom line is that we misdesigned the WAL representation
by assuming that this sort of info could always be piggybacked on a
transaction commit record. It's time to fix that.I think we got to piggyback it onto a commit record, as long as there's
one. Otherwise it's going to be more complex (queuing messages when
reading an inval record) and slower (more wal records). I can easily
develop a patch for that, the question is what we do on the back
branches...
We have introduced new wal records in back branches previously --
nothing new (c.f. 8e9a16ab8f7f0e5813644975cc3f336e5b064b6e). The user
just needs to make sure to upgrade the standbys first. If they don't,
they would die upon replay of the first such record, which they can take
as an indication that they need to be upgraded; the standby is down for
some time, but there is no data loss or corruption.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Andres Freund <andres@anarazel.de> writes:
On 2016-04-15 15:26:17 -0400, Tom Lane wrote:
I think the bottom line is that we misdesigned the WAL representation
by assuming that this sort of info could always be piggybacked on a
transaction commit record. It's time to fix that.
I think we got to piggyback it onto a commit record, as long as there's
one.
No objection to that part. What I'm saying is that when there isn't one,
the answer is a new record type, not forcing xid assignment. It might
look almost like a commit record, but it shouldn't imply that there
was a transaction.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
