Freezing is not WAL-logged
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged. That leads to this scenario:
postgres=# CREATE TABLE foo (bar int);
CREATE TABLE
postgres=# INSERT INTO foo VALUES (1);
INSERT 0 1
postgres=# SELECT xmin,xmax, bar FROM foo;
xmin | xmax | bar
------+------+-----
669 | 0 | 1
(1 row)
postgres=# VACUUM FREEZE foo;
VACUUM
killall -9 postgres + restart
postgres=# SELECT xmin,xmax, bar FROM foo;
xmin | xmax | bar
------+------+-----
669 | 0 | 1
(1 row)
postgres=# SELECT relminxid FROM pg_class WHERE relname='foo';
relminxid
-----------
672
(1 row)
It looks like a bug to me.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged.
The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don't
WAL-log either.
regards, tom lane
Tom Lane wrote:
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged.The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don't
WAL-log either.
Also it'd be hugely expensive to log each freeze operation. The
alternative would be to log a VACUUM FREEZE, but that has the potential
to cause enormous recovery runtime.
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Tom Lane wrote:
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged.The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don't
WAL-log either.
Because the relminxid-update is WAL-logged, you can get into situation
where relminxid > the real smallest xid of the table, as demonstrated by
the example I gave. relminxid is used to determine the safe clog cut-off
point, so that's not harmless.
Granted, the chances of getting data corruption from this are small, but
it's possible.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Alvaro Herrera wrote:
Tom Lane wrote:
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged.The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don'ton
WAL-log either.Also it'd be hugely expensive to log each freeze operation. The
alternative would be to log a VACUUM FREEZE, but that has the potential
to cause enormous recovery runtime.
Freezing isn't very common, and a single WAL record per page would be
enough. I can write the patch.
It does have the potential to increase recovery times, but I don't think
we can just cross our fingers and hope that no crash happens after
freezing some tuples.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas wrote:
Alvaro Herrera wrote:
Tom Lane wrote:
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged.The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don'ton
WAL-log either.Also it'd be hugely expensive to log each freeze operation. The
alternative would be to log a VACUUM FREEZE, but that has the potential
to cause enormous recovery runtime.Freezing isn't very common, and a single WAL record per page would be
enough. I can write the patch.
Yeah, you are right, I was thinking that we would freeze a lot of tuples
a lot of the time, but this is obviously not true.
It does have the potential to increase recovery times, but I don't think
we can just cross our fingers and hope that no crash happens after
freezing some tuples.
Agreed.
My idea was to move relminxid to a new pg_class_nt non-transactional
catalog in 8.3, but we certainly need to give a solution for the problem
in 8.2. I don't remember why I concluded that we were safe against this
problem, but you are right that we aren't.
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera <alvherre@commandprompt.com> writes:
My idea was to move relminxid to a new pg_class_nt non-transactional
catalog in 8.3, but we certainly need to give a solution for the problem
in 8.2.
Didn't we pretty much shoot down the idea of a nontransactional
extension catalog as being not worth the trouble?
I don't remember why I concluded that we were safe against this
problem, but you are right that we aren't.
I think there may be another set of issues here too: what about PITR?
As things stand, neither hint-bit settings nor FrozenXID replacement
are guaranteed to get propagated to a PITR slave.
We don't propagate CLOG truncation to a slave either, which means that
to some extent it'd have the ability to re-set hint bits for itself
after coming up. This isn't bulletproof though; most obviously in
the case where the PITR replay (or hot-slave lifespan) exceeded
4G transactions.
Can we fix it so it's sufficient to propagate FrozenXID replacement
to the slave? If we have to turn hint-bit setting into a logged
operation, that *will* be expensive.
regards, tom lane
On Fri, Oct 20, 2006 at 09:49:47AM +0100, Heikki Linnakangas wrote:
Alvaro Herrera wrote:
Tom Lane wrote:
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged.The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don'ton
WAL-log either.Also it'd be hugely expensive to log each freeze operation. The
alternative would be to log a VACUUM FREEZE, but that has the potential
to cause enormous recovery runtime.Freezing isn't very common, and a single WAL record per page would be
enough. I can write the patch.It does have the potential to increase recovery times, but I don't think
we can just cross our fingers and hope that no crash happens after
freezing some tuples.
If we're worried enough about the recovery time, AFAIK it should be safe
to fire off a background process to freeze the table and let the rest of
recovery proceed. Perhaps it wouldn't be too difficult to have
autovacuum do this, though we'd have to decide what to do if it was
disabled.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Tom Lane wrote:
Alvaro Herrera <alvherre@commandprompt.com> writes:
My idea was to move relminxid to a new pg_class_nt non-transactional
catalog in 8.3, but we certainly need to give a solution for the problem
in 8.2.Didn't we pretty much shoot down the idea of a nontransactional
extension catalog as being not worth the trouble?
Yeah, for 8.2, but we said there were some things to be gained by doing
things that way and agreed to do'em in 8.3.
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.