too many Fatal Error 1: shutdown messages

Started by Johnson, Shaunnover 23 years ago4 messagesgeneral
Jump to latest
#1Johnson, Shaunn
SJohnson6@bcbsm.com

Running PostgreSQL 7.2.1 on RedHat Linux 7.2

Over the past few days, the database has been acting
strange and have been trying to shutdown. While my
postmaster is running debug level 2, when I check my
logs to see why, I get :

[snip from log]

Jan 8 09:54:16 test_srv postgres[19069]: [12354] FATAL 1:
The database system is shutting down

[/snip from log]

There doesn't seem to be any rhyme or reason for it;
yesterday, I thought it was one of the drives having
problems and the kernel gave an error.

Last night, I thought it was because of too many
users logging in (it had reached it's max and gave that
message).

Both seem to be red herrings.

I'm going to expand on the debug level to see the whys and
hows of some of these things, but, I'm running out of ideas.

Any suggestions?

-X

#2Johnson, Shaunn
SJohnson6@bcbsm.com
In reply to: Johnson, Shaunn (#1)
Re: too many Fatal Error 1: shutdown messages

Running PostgreSQL 7.2.1 on RedHat Linux 7.2

Can someone tell me why I'm getting this error when I
try to vacuumdb -a -U postgres -v -z (other than
there is no filed called '003' in that directory)?

[snip from log]

FATAL 2: open of /var/lib/pgsql/data/pg_clog/0003 failed: No such file or
directory
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
connection to server was lost

[/snip from log]

I have a '004' in that directory ...

Thanks!

-X

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Johnson, Shaunn (#2)
Re: too many Fatal Error 1: shutdown messages

"Johnson, Shaunn" <SJohnson6@bcbsm.com> writes:

Running PostgreSQL 7.2.1 on RedHat Linux 7.2

FATAL 2: open of /var/lib/pgsql/data/pg_clog/0003 failed: No such file or
directory
server closed the connection unexpectedly

You've run into one of the bugs that we fixed in 7.2.3.

I am not sure if you can recover from this :-( Do you have a recent
backup?

Memo to onlookers: there is a reason why we put out dot-releases.
We would not have bothered with a 7.2.3 release if we hadn't found
some pretty serious bugs. If you are running 7.2.x for x < 3, you
are living dangerously.

regards, tom lane

#4Bjoern Metzdorf
bm@turtle-entertainment.de
In reply to: Johnson, Shaunn (#2)
Re: too many Fatal Error 1: shutdown messages

"Johnson, Shaunn" <SJohnson6@bcbsm.com> writes:

Running PostgreSQL 7.2.1 on RedHat Linux 7.2

FATAL 2: open of /var/lib/pgsql/data/pg_clog/0003 failed: No such file

or

directory
server closed the connection unexpectedly

You've run into one of the bugs that we fixed in 7.2.3.

I am not sure if you can recover from this :-( Do you have a recent
backup?

We've run into these pg_clog issues quite a lot in the last time with 7.2.1.
Good to know that 7.2.3 might fix this.

In my experience you don't necessarily need to grab the latest backup. If
you really just want to get the database up and running again, you may use a
dummy clog and then dump and restore the database.

E.g. "dd if=/dev/zero of=/path/to/pgdata/pg_clog/0003 bs=1024k count=256"

Tom pointed that out in
http://groups.google.com/groups?q=tom+lane+pg_clog&amp;hl=en&amp;lr=&amp;ie=UTF-8&amp;oe=utf
-8&selm=27106.1031148278%40sss.pgh.pa.us&rnum=5

Regards,
Bjoern