crash / data recovery issues
I'm trying to do some data recovery on an 8.1.9 system. The brief history is
the system crashed, attempted to do xlog replay but that failed. I did a
pg_resetxlog to get something that would startup, and it looks as if the
indexes on pg_class have become corrupt. (ie. reindex claimes duplicate rows,
which do not show up when doing count() manipulations on the data). As it
turns out, I can't drop these indexes either (system refuses with message
indexes are needed by the system). This has kind of let the system in an
unworkable state.
I've tried to do a pg_dump, but get schema with OID 96568 does not exist
error. The database has a number (~100) temp schemas in it, so I was
suspecting that the problem was with some object referencing a temp schema
with broken dependencies, but I looked through pg_depend for any referencing
objects but found none. I also looked through pg_type, pg_proc, pg_class,
pg_constraint, pg_operator, pg_opclass, pg_conversion at their respective
*namespace fields and also found no matches. Any suggestions on what else
might cause this, or how to get past it?
I also did some digging to find the original error on xlog replay and it
was "failed to re-find parent key in "763769" for split pages 21032/21033".
I'm wondering if this is actually something you can push past with
pg_resetxlog, or if I need to do a pg_resetxlog and pass in values prior to
that error point (i guess essentially letting pg_resetxlog do a lookup)...
thoughts?
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
Robert Treat wrote:
I'm trying to do some data recovery on an 8.1.9 system. The brief history is
the system crashed, attempted to do xlog replay but that failed. I did a
pg_resetxlog to get something that would startup, and it looks as if the
indexes on pg_class have become corrupt. (ie. reindex claimes duplicate rows,
which do not show up when doing count() manipulations on the data). As it
turns out, I can't drop these indexes either (system refuses with message
indexes are needed by the system). This has kind of let the system in an
unworkable state.
You can work out of it by starting a standalone server with system
indexes disabled (postgres -O -P, I think) and do a REINDEX on it (the
form of it that reindexes all system indexes -- I think it's REINDEX
DATABASE).
I also did some digging to find the original error on xlog replay and it
was "failed to re-find parent key in "763769" for split pages 21032/21033".
I'm wondering if this is actually something you can push past with
pg_resetxlog, or if I need to do a pg_resetxlog and pass in values prior to
that error point (i guess essentially letting pg_resetxlog do a lookup)...
thoughts?
You should be able to get out of that by reindexing that index.
(Actually, after you do a pg_resetxlog I think the best is to pg_dump
the whole thing and reload it. That gives you at least the assurance
that your FKs are not b0rked)
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Robert Treat <xzilla@users.sourceforge.net> writes:
I'm trying to do some data recovery on an 8.1.9 system.
...
I also did some digging to find the original error on xlog replay and it
was "failed to re-find parent key in "763769" for split pages 21032/21033".
Hmm, the only known cause of that was fixed in 8.1.6. Don't suppose you made
a copy of everything before destroying the evidence with pg_resetxlog?
If you did, any chance I could get access to it?
regards, tom lane
On Wednesday 06 February 2008 13:56, Alvaro Herrera wrote:
Robert Treat wrote:
it looks as if the indexes on pg_class have become corrupt. (ie. reindex
claimes duplicate rows, which do not show up when doing count()
manipulations on the data). As it turns out, I can't drop these indexes
either (system refuses with message indexes are needed by the system).
This has kind of let the system in an unworkable state.You can work out of it by starting a standalone server with system
indexes disabled (postgres -O -P, I think) and do a REINDEX on it (the
form of it that reindexes all system indexes -- I think it's REINDEX
DATABASE).
Sorry, I should have mentioned I tried the above was under postgres -d
1 -P -O -D /path/to/data, but the reindex complains (doing reindex directly
on the pg_class indexes, or doing reindex system).
Personally I was surprised to find out it wouldn't let me drop the indexes
under this mode, but thats a different story. Oh, probably worth noting I
am able to reindex other system tables this way, just not pg_class.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL