URGENT - startup process (PID 29541) was terminated by signal 6

Started by andy rostover 19 years ago4 messagesgeneral
Jump to latest
#1andy rost
Andy.Rost@noaa.gov

Our Opteron DB server had a problem with its RAID controller requiring
an immediate shutdown of our Postgres server (8.1.3 on FreeBSD 6.0
release number 10). We used kill -QUIT on the postmaster PID.

After repairing and rebooting the server we tried to start Postgres and
get the following:

2006-10-06 12:32:40 CDT FATAL: the database system is starting up
2006-10-06 12:32:40 CDT FATAL: the database system is starting up
2006-10-06 12:32:40 CDT PANIC: heap_clean_redo: no block
2006-10-06 12:32:40 CDT LOG: startup process (PID 29541) was terminated
by signal 6
2006-10-06 12:32:40 CDT LOG: aborting startup due to startup process
failure

At shutdown we had the following:

2006-10-06 12:08:48 CDT PANIC: right sibling is not next child in
"winddir_idxu"
2006-10-06 12:08:48 CDT STATEMENT: VACUUM ANALYZE VERBOSE
2006-10-06 12:08:49 CDT LOG: server process (PID 91933) was terminated
by signal 6
2006-10-06 12:08:49 CDT LOG: terminating any other active server processes

Anything I can do?

--------------------------------------------------------------------------------
Andrew Rost
National Operational Hydrologic Remote Sensing Center (NOHRSC)
National Weather Service, NOAA
1735 Lake Dr. West, Chanhassen, MN 55317-8582
Voice: (952)361-6610 x 234
Fax: (952)361-6634
andy.rost@noaa.gov
http://www.nohrsc.noaa.gov
--------------------------------------------------------------------------------

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: andy rost (#1)
Re: URGENT - startup process (PID 29541) was terminated by signal 6

andy rost <Andy.Rost@noaa.gov> writes:

Our Opteron DB server had a problem with its RAID controller requiring
an immediate shutdown of our Postgres server (8.1.3 on FreeBSD 6.0
release number 10). We used kill -QUIT on the postmaster PID.

2006-10-06 12:32:40 CDT PANIC: heap_clean_redo: no block
...
2006-10-06 12:08:48 CDT PANIC: right sibling is not next child in
"winddir_idxu"

Um, were you running with full_page_writes off? Bad idea in 8.1 :-( ...
especially on hardware that turns out to not be 100% reliable.

I think your only hope of restarting the database is to use
pg_resetxlog. This is likely to leave you with some database
corruption, in the form of partially applied recent transactions.
I'd recommend a dump and reload, or at least REINDEXing all the
indexes, to forestall problems from that. While you are at it,
update to 8.1.4.

regards, tom lane

#3Emanuele Rocca
ema@linux.it
In reply to: Tom Lane (#2)
Re: URGENT - startup process (PID 29541) was terminated by signal 6

Hello Tom,

* Tom Lane <tgl@sss.pgh.pa.us>, [2006-10-06 15:16 -0400]:

Um, were you running with full_page_writes off? Bad idea in 8.1 :-( ...

The manual [1]http://www.postgresql.org/docs/8.1/static/runtime-config-wal.html#GUC-FULL-PAGE-WRITES says that full_page_writes is ignored and always treated
as if it was set to on. Is it wrong?

ciao,
ema

[1]: http://www.postgresql.org/docs/8.1/static/runtime-config-wal.html#GUC-FULL-PAGE-WRITES

#4Bruce Momjian
bruce@momjian.us
In reply to: Emanuele Rocca (#3)
Re: URGENT - startup process (PID 29541) was

Emanuele Rocca wrote:
-- Start of PGP signed section.

Hello Tom,

* Tom Lane <tgl@sss.pgh.pa.us>, [2006-10-06 15:16 -0400]:

Um, were you running with full_page_writes off? Bad idea in 8.1 :-( ...

The manual [1] says that full_page_writes is ignored and always treated
as if it was set to on. Is it wrong?

Well, early versions of 8.1.X did honor full_page_writes, but 8.1.4
disabled it.

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +