ERROR: invalid page header in block 502758 of relation base/16388/16403

i forgot the system informations

postgresql-server-8.4.0-1PGDG.rhel5
linux kernel 2.6.18-128.4.1.el5xen x86_64
centos release 5.3 (final)

fous velkej <honza801@gmail.com> writes:

i got following error while selecting from my db

ERROR: invalid page header in block 502758 of relation base/16388/16403

i've found some solution in this thread
http://www.mail-archive.com/pgsql-general@postgresql.org/msg101644.html

but my problem is, that there are multiple files with similar name in
the base/16388 directory.
16403 (1Gb)
16403.1 (1Gb)
16403.2 (1Gb)
16403.3 (almost 1 Gb)

so the question is, where can i find the right 502758 position (i
assume that this is the number of 8k block position).
each file with size 1Gb has 131072 of 8k blocks.

Right, so it would be in the .3 file at block number 109542.

regression=# select 502758 / 131072;
?column?
----------
3
(1 row)

regression=# select 502758 % 131072;
?column?
----------
109542
(1 row)

regards, tom lane

hi again

i've stopped postgres and done this
dd if=16403.3.orig of=16403.3 bs=8192 count=109541
dd if=/dev/zero bs=8192 count=1 >> 16403.3
dd if=16403.3.orig skip=109542 bs=8192 >> 16403.3

then started postgres, but did not help, the same error appears.
ERROR: invalid page header in block 502758 of relation base/16388/16403

am i doing anything wrong?

regards, fous

fous velkej <honza801@gmail.com> writes:

hi again
i've stopped postgres and done this
dd if=16403.3.orig of=16403.3 bs=8192 count=109541
dd if=/dev/zero bs=8192 count=1 >> 16403.3
dd if=16403.3.orig skip=109542 bs=8192 >> 16403.3

then started postgres, but did not help, the same error appears.
ERROR: invalid page header in block 502758 of relation base/16388/16403

am i doing anything wrong?

You appear to be zeroing block 109541 not 109542.

regards, tom lane

ok, fault on my side

this is maybe better code
dd if=16403.3.orig of=16403.3 bs=8192 count=109542
dd if=/dev/zero bs=8192 count=1 >> 16403.3
dd if=16403.3.orig skip=109543 bs=8192 >> 16403.3

but now, it says
ERROR: index "day" contains unexpected zero page at block 502758
HINT: Please REINDEX it.

ok, so i did reindexing ending with following error
proxylog=# REINDEX TABLE user_record ;
ERROR: concurrent insert in progress

what should i do now?
i think it's not the problem of hw

thanks for help
fous

fous velkej <honza801@gmail.com> writes:

ok, so i did reindexing ending with following error
proxylog=# REINDEX TABLE user_record ;
ERROR: concurrent insert in progress

So that page header wasn't the only corrupted data in your database.
Maybe you'd better take a step back and tell us some context about
your problem. I hope you've got recent backups, because if there's
two different corrupted spots there are probably a lot more.

regards, tom lane

there is a database running on xen dom U, but the node (dom0) went down,
we have a cluster, so the other node started the service and there is
nothing that shows any problem
unless i use SELECT on this database/table (this db contains only one table)
there is no backup of the database.
i can't see any solution and if you can't see any solution either, i'm
going to reinit the db
:)

fous

2009/9/17 Tom Lane <tgl@sss.pgh.pa.us>: