typo: XIDs are actually compared using modulo-2^32 arithmetic

Gianni Ciolli <gianni.ciolli@2ndquadrant.it> writes:

It seems there is a typo here:
http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND
where we say that we compare XIDs using arithmetic modulo 2^31, which
should instead be 2^32 (as it is with uint32, e.g. xid_age).

[ thinks about that for awhile... ] Yeah, I think you're right.
Patch pushed, thanks!

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

I don't have a source tree handy but iirc we treaty 2^31 values as being in
the past and 2^31 values as being in the future.

I've been trying to think how to protect better against the recent vacuum
freeze bug. If someone ruins vacuum freeze now and has any wrapped values
they'll destroy their possibly recoverable data.

It seems to me we shouldn't really need 2^31 values in the future. If
vacuum or hot pruning comes across an xid far in the future, say a million
xids further into the future than the most recent transaction, then it
should signal an error rather than just treat it as being in the future.

--
greg
On 12 Dec 2013 09:41, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

On 2013-12-14 20:19:11 +0000, Greg Stark wrote:

I don't have a source tree handy but iirc we treaty 2^31 values as being in
the past and 2^31 values as being in the future.

I've been trying to think how to protect better against the recent vacuum
freeze bug. If someone ruins vacuum freeze now and has any wrapped values
they'll destroy their possibly recoverable data.

Fortunately that's exceedingly unlikely to happen. There's basically two
consequences the bug can have:
a) we don't freeze tuples on pages that are already marked all-visible
because we're doing a partial scan and thus don't scan them.
b) (9.2+) we don't freeze tuples on a page not marked all visible,
because a buffer is pinned and we skip those when !scan_all.

a) can lead to the tuple vanishing again because they are reported as
being in progress, after 2^31 xids passed. But by virtue of being on an
all-visible page, they are fully hinted. Which means, that after the
wraparound they will be reported as delete-in-progress or
insert-in-progress. Luckily neither will get vacuumed away. They will
"just" be invisible.

What can happen with b) is that the clog gets truncated to somewhere
between the real relfrozenxid and the computed relfrozenxid. In that
case we'll get errors when later doing a
HeapTupleSatisfiesVacuum/HTSMVCC. But it's quite likely that the tuple
will get vacuumed at some point before 2^31 xids have passed since its
not marked all visible and thus will be scanned with each future vacuum.

So, for the data to be removed permanently you'd have to hit b) with
partial vacuums (scan_all vacuums do wait!) several times in a row. That
seems unlikely.

It seems to me we shouldn't really need 2^31 values in the future. If
vacuum or hot pruning comes across an xid far in the future, say a million
xids further into the future than the most recent transaction, then it
should signal an error rather than just treat it as being in the future.

Yea, I have wondered about that as well.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers