CASCADING could not open relation with OID

Started by Jaime Silvelaover 19 years ago10 messagesgeneral
Jump to latest
#1Jaime Silvela
JSilvela@Bear.com

I've seen that some other people have had ERROR could not open relation
with OID ###

The suggested cause was somebody trying to drop a table in the middle of
VACUUM.

In my case, the error seems to be spreading. Initially it affected only
one table in a staging area that would get TRUNCATEd and populated every
night. Now I'm starting to see it in more tables with similar
functionality.

These get populated by scripts that run at night. Some of these are
crashing. You can see a log below.

Has somebody experienced this before? I'm getting worried, could this be
due to a faulty disk?

In the short term, I think I'm just going to recreate the tables from a
backup of the schema. Any suggestions?

Thanks

Jaime

WARNING: terminating connection because of crash of another server
process

DETAIL: The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.

HINT: In a moment you should be able to reconnect to the database and
repeat your command.

CONTEXT: writing block 5529 of relation 1663/16390/686426795

Attachments:

Disclaimer.txttext/plain; charset=us-ascii; name=DisclaimerDownload
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jaime Silvela (#1)
Re: CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@Bear.com> writes:

WARNING: terminating connection because of crash of another server
process

This is not an interesting message: the interesting message is the
previous one about exactly what happened to the other process. Look
earlier in the server log.

regards, tom lane

#3Jaime Silvela
JSilvela@Bear.com
In reply to: Tom Lane (#2)
Re: CASCADING could not open relation with OID

Actually the server logging was disabled, which I am now enabling.

But scripts have been complaining about not finding this or that
relation with OID x. I've located each of the tables and am trying to
recreate them.

If I try to read from one, I get ERROR: could not open relation with
OID 16896

If I try to redefine it, I get ERROR: relation "bb_master" already
exists

If I try to DROP it, ERROR: cache lookup failed for relation 16896

How can I get around this? What's happening?

Tomorrow I should have better logs on the initially reported problem.

Many thanks
Jaime

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Wednesday, August 23, 2006 5:20 PM
To: Silvela, Jaime (Exchange)
Cc: pgsql-general
Subject: Re: [GENERAL] CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@Bear.com> writes:

WARNING: terminating connection because of crash of another server
process

This is not an interesting message: the interesting message is the
previous one about exactly what happened to the other process. Look
earlier in the server log.

regards, tom lane

***********************************************************************
Bear Stearns is not responsible for any recommendation, solicitation,
offer or agreement or any information about any transaction, customer
account or account activity contained in this communication.

Bear Stearns does not provide tax, legal or accounting advice. You
should consult your own tax, legal and accounting advisors before
engaging in any transaction. In order for Bear Stearns to comply with
Internal Revenue Service Circular 230 (if applicable), you are notified
that any discussion of U.S. federal tax issues contained or referred to
herein is not intended or written to be used, and cannot be used, for
the purpose of: (A) avoiding penalties that may be imposed under the
Internal Revenue Code; nor (B) promoting, marketing or recommending to
another party any transaction or matter addressed herein.
***********************************************************************

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jaime Silvela (#3)
Re: CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@Bear.com> writes:

If I try to read from one, I get ERROR: could not open relation with
OID 16896
If I try to redefine it, I get ERROR: relation "bb_master" already
exists
If I try to DROP it, ERROR: cache lookup failed for relation 16896

What do you get from
select oid, * from pg_class where relname = 'bb_master';
select oid, * from pg_class where oid = 16896;

If either one fails to get a hit, try it again after doing
set enable_indexscan = off;

How can I get around this? What's happening?

Hard to tell. It sounds a bit like pg_class catalog damage, but the
above experiment will tell us more.

regards, tom lane

#5Jaime Silvela
JSilvela@Bear.com
In reply to: Tom Lane (#4)
Re: CASCADING could not open relation with OID

I get values frin the first statement but not from the second.
After setting indexscan to off, still the same thing.

Should this setting be off in general?

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Wednesday, August 23, 2006 6:15 PM
To: Silvela, Jaime (Exchange)
Cc: pgsql-general
Subject: Re: [GENERAL] CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@Bear.com> writes:

If I try to read from one, I get ERROR: could not open relation with
OID 16896
If I try to redefine it, I get ERROR: relation "bb_master" already
exists
If I try to DROP it, ERROR: cache lookup failed for relation 16896

What do you get from
select oid, * from pg_class where relname = 'bb_master';
select oid, * from pg_class where oid = 16896;

If either one fails to get a hit, try it again after doing
set enable_indexscan = off;

How can I get around this? What's happening?

Hard to tell. It sounds a bit like pg_class catalog damage, but the
above experiment will tell us more.

regards, tom lane

***********************************************************************
Bear Stearns is not responsible for any recommendation, solicitation,
offer or agreement or any information about any transaction, customer
account or account activity contained in this communication.

Bear Stearns does not provide tax, legal or accounting advice. You
should consult your own tax, legal and accounting advisors before
engaging in any transaction. In order for Bear Stearns to comply with
Internal Revenue Service Circular 230 (if applicable), you are notified
that any discussion of U.S. federal tax issues contained or referred to
herein is not intended or written to be used, and cannot be used, for
the purpose of: (A) avoiding penalties that may be imposed under the
Internal Revenue Code; nor (B) promoting, marketing or recommending to
another party any transaction or matter addressed herein.
***********************************************************************

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jaime Silvela (#5)
Re: CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@bear.com> writes:

I get values frin the first statement but not from the second.

What values exactly ... particularly the OID?

After setting indexscan to off, still the same thing.
Should this setting be off in general?

Certainly not! That was just an experiment to see if your problem was
corruption of the indexes on pg_class. Seems like not.

regards, tom lane

#7Jaime Silvela
JSilvela@Bear.com
In reply to: Tom Lane (#6)
Re: CASCADING could not open relation with OID

I get

Oid: 16896
Relname: bb_master
Relnamespace: 16392
Reltype: 16897
...
Reltablespace: 0
Relpages: 0
Reltuples: 0
...
Relkind: r
...

Thanks
Jaime

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Wednesday, August 23, 2006 6:54 PM
To: Silvela, Jaime (Exchange)
Cc: pgsql-general
Subject: Re: [GENERAL] CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@bear.com> writes:

I get values frin the first statement but not from the second.

What values exactly ... particularly the OID?

After setting indexscan to off, still the same thing.
Should this setting be off in general?

Certainly not! That was just an experiment to see if your problem was
corruption of the indexes on pg_class. Seems like not.

regards, tom lane

***********************************************************************
Bear Stearns is not responsible for any recommendation, solicitation,
offer or agreement or any information about any transaction, customer
account or account activity contained in this communication.

Bear Stearns does not provide tax, legal or accounting advice. You
should consult your own tax, legal and accounting advisors before
engaging in any transaction. In order for Bear Stearns to comply with
Internal Revenue Service Circular 230 (if applicable), you are notified
that any discussion of U.S. federal tax issues contained or referred to
herein is not intended or written to be used, and cannot be used, for
the purpose of: (A) avoiding penalties that may be imposed under the
Internal Revenue Code; nor (B) promoting, marketing or recommending to
another party any transaction or matter addressed herein.
***********************************************************************

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jaime Silvela (#7)
Re: CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@bear.com> writes:

I get
Oid: 16896
Relname: bb_master

Hmm ... but you're *sure* "where oid = 16896" can't find this row,
even with enable_indexscan = off? That doesn't make a lot of sense.

To cut to the chase, though: try "REINDEX pg_class" and see if it helps.

regards, tom lane

#9Jaime Silvela
JSilvela@Bear.com
In reply to: Tom Lane (#8)
Re: CASCADING could not open relation with OID

It hadn't occurred to me to reindex the pg_class!! Beginner...

After reindexing, both query lines were successful, and I was able to
access my missing tables!!

THANKS!!

What could be the possible cause for this?

Thanks again,
Jaime

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Wednesday, August 23, 2006 7:10 PM
To: Silvela, Jaime (Exchange)
Cc: pgsql-general
Subject: Re: [GENERAL] CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@bear.com> writes:

I get
Oid: 16896
Relname: bb_master

Hmm ... but you're *sure* "where oid = 16896" can't find this row,
even with enable_indexscan = off? That doesn't make a lot of sense.

To cut to the chase, though: try "REINDEX pg_class" and see if it helps.

regards, tom lane

***********************************************************************
Bear Stearns is not responsible for any recommendation, solicitation,
offer or agreement or any information about any transaction, customer
account or account activity contained in this communication.

Bear Stearns does not provide tax, legal or accounting advice. You
should consult your own tax, legal and accounting advisors before
engaging in any transaction. In order for Bear Stearns to comply with
Internal Revenue Service Circular 230 (if applicable), you are notified
that any discussion of U.S. federal tax issues contained or referred to
herein is not intended or written to be used, and cannot be used, for
the purpose of: (A) avoiding penalties that may be imposed under the
Internal Revenue Code; nor (B) promoting, marketing or recommending to
another party any transaction or matter addressed herein.
***********************************************************************

#10Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jaime Silvela (#9)
Re: CASCADING could not open relation with OID

"Silvela, Jaime \(Exchange\)" <JSilvela@bear.com> writes:

What could be the possible cause for this?

Hard to say ... have you had any hardware flakiness lately? Are you
running an up-to-date PG release?

regards, tom lane