BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Started by Olivier Macchionialmost 12 years ago30 messagesbugs
Jump to latest
#1Olivier Macchioni
olivier.macchioni@wingo.ch

The following bug has been logged on the website:

Bug reference: 10329
Logged by: Olivier Macchioni
Email address: olivier.macchioni@wingo.ch
PostgreSQL version: 9.1.13
Operating system: Debian 7.4 and 6.0.9
Description:

Hello all,

- Streaming Replication setup, 2 DB servers
- 9.1.13 on both sides (from apt.postgresql.org)
- Debian 6.0.9 on master server, 7.4 on standby server

When doing the following query on the standby server:

select * from ir_translation where src = 'a';

We have the following error in the logfiles:

ERROR: could not read block 0 in file "base/56100265/57047884": read only 0
of 8192 bytes

When checking the file usage, the following is reported:

postgres@db-stdby:~/9.1/main/base/56100265$
/usr/lib/postgresql/9.1/bin/oid2name -d xx -f 57047884

From database "xx":

Filenode Table Name
---------------------------------------
57047884 ir_translation_src_hash_idx

ls -la shows that the file has a size of 0.

What's interesting as well is that this file has a last modification date of
April, 29th - when the DB has been created on the Master server much more
recently, by doing a "drop xx; rename xx_new xx;"

root@db-stdby:/var/lib/postgresql/9.1/main/base/56100265# stat 57047884
File: `57047884'
Size: 0 Blocks: 0 IO Block: 4096 regular empty
file
Device: 801h/2049d Inode: 1051439 Links: 1
Access: (0600/-rw-------) Uid: ( 106/postgres) Gid: ( 111/postgres)
Access: 2014-05-15 11:15:50.218038536 +0200
Modify: 2014-04-29 16:20:36.645605377 +0200
Change: 2014-04-29 16:20:36.645605377 +0200
Birth: -

root@db:/var/lib/postgresql/9.1/main/base/56100265# stat 57047884
File: `57047884'
Size: 2129920 Blocks: 4168 IO Block: 4096 regular file
Device: 801h/2049d Inode: 315480 Links: 1
Access: (0600/-rw-------) Uid: ( 104/postgres) Gid: ( 107/postgres)
Access: 2014-05-15 11:24:29.000000000 +0200
Modify: 2014-05-03 23:08:11.000000000 +0200
Change: 2014-05-03 23:08:11.000000000 +0200

The master server doesn't show this behavior.

The symptoms are similar to some previous (but much older) posts on this
list, for instance
/messages/by-id/E1TEjPD-0002Yq-7L@wrigleys.postgresql.org

So I think this bug is not fixed yet...

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Olivier Macchioni (#1)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

olivier.macchioni@wingo.ch writes:

When doing the following query on the standby server:

select * from ir_translation where src = 'a';

We have the following error in the logfiles:

ERROR: could not read block 0 in file "base/56100265/57047884": read only 0
of 8192 bytes

When checking the file usage, the following is reported:

postgres@db-stdby:~/9.1/main/base/56100265$
/usr/lib/postgresql/9.1/bin/oid2name -d xx -f 57047884

From database "xx":

Filenode Table Name
---------------------------------------
57047884 ir_translation_src_hash_idx

"hash_idx"? Is this a hash index? Those aren't supported for replication
purposes (no WAL code :-().

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#3Olivier Macchioni
olivier.macchioni@wingo.ch
In reply to: Tom Lane (#2)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

[...]
"hash_idx"? Is this a hash index? Those aren't supported for replication
purposes (no WAL code :-().

regards, tom lane

Bingo:

xx=# \d ir_translation_src_hash_idx
Index "public.ir_translation_src_hash_idx"
Column | Type | Definition
--------+---------+------------
src | integer | src
hash, for table "public.ir_translation"

(for the record, this is created by the stock version of OpenERP v6)

And it's the only hash index out of 1741 other indexes on my system...

I guess my best bet is to replace it by another kind of indexes... and maybe one day PostgreSQL will be clever enough to issue a warning / error in such a case for the people like me who don't read *all the doc* :P

Thanks a lot and sorry for the noise :)

Olivier

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#4Bruce Momjian
bruce@momjian.us
In reply to: Olivier Macchioni (#3)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Thu, May 15, 2014 at 05:20:35PM +0200, Olivier Macchioni wrote:

[...]
"hash_idx"? Is this a hash index? Those aren't supported for replication
purposes (no WAL code :-().

regards, tom lane

Bingo:

xx=# \d ir_translation_src_hash_idx
Index "public.ir_translation_src_hash_idx"
Column | Type | Definition
--------+---------+------------
src | integer | src
hash, for table "public.ir_translation"

(for the record, this is created by the stock version of OpenERP v6)

And it's the only hash index out of 1741 other indexes on my system...

I guess my best bet is to replace it by another kind of indexes... and maybe one day PostgreSQL will be clever enough to issue a warning / error in such a case for the people like me who don't read *all the doc* :P

Thanks a lot and sorry for the noise :)

Yes, streaming replication has made our hash indexes even worse. In the
past, I have suggested we issue a warning for the creation of hash
indexes, but did not get enough agreement.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#4)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Bruce Momjian <bruce@momjian.us> writes:

On Thu, May 15, 2014 at 05:20:35PM +0200, Olivier Macchioni wrote:

I guess my best bet is to replace it by another kind of indexes... and maybe one day PostgreSQL will be clever enough to issue a warning / error in such a case for the people like me who don't read *all the doc* :P

Yes, streaming replication has made our hash indexes even worse. In the
past, I have suggested we issue a warning for the creation of hash
indexes, but did not get enough agreement.

Mainly because it wouldn't be a very helpful message.

I wonder though if we could throw a flat-out error for attempts to use
a hash index on a hot standby server. That would get people's attention
without being mere nagging in other situations. It's not a 100% solution
because you'd still lose if you tried to use a hash index on a slave
since promoted to master. But it would help without being a large
sink for effort.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#6Olivier Macchioni
olivier.macchioni@wingo.ch
In reply to: Tom Lane (#5)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On May 15, 2014, at 21:07, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bruce Momjian <bruce@momjian.us> writes:

On Thu, May 15, 2014 at 05:20:35PM +0200, Olivier Macchioni wrote:

I guess my best bet is to replace it by another kind of indexes... and maybe one day PostgreSQL will be clever enough to issue a warning / error in such a case for the people like me who don't read *all the doc* :P

Yes, streaming replication has made our hash indexes even worse. In the
past, I have suggested we issue a warning for the creation of hash
indexes, but did not get enough agreement.

Mainly because it wouldn't be a very helpful message.

I wonder though if we could throw a flat-out error for attempts to use
a hash index on a hot standby server. That would get people's attention
without being mere nagging in other situations. It's not a 100% solution
because you'd still lose if you tried to use a hash index on a slave
since promoted to master. But it would help without being a large
sink for effort.

regards, tom lane

I am not aware of the technical implementation hurdles, but in this case any of the following behaviors would have been better for me:

- still perform the query, without using the index (preferably issuing a warning of some kind)

- Tom's solution - refuse to perform the query, with a clear error message (looking at the error, I thought I had a broken replication, it's clearly not a nice message to have)

- (possibly replace on-the-fly any hash index by another kind of indexes on the slave when setting up the replication or creating the index on the master? does this even make sense?)

- somehow disallow to setup of streaming replication when there is a hash index in any DB / and refuse the creation of a hash index on a master server, so we don't reach this situation

- I assume there is a usage for hash indexes and we don't simply want to deprecate them

Olivier

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: Olivier Macchioni (#6)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Olivier Macchioni <olivier.macchioni@wingo.ch> writes:

- I assume there is a usage for hash indexes and we don't simply want to deprecate them

There's a school of thought that says we *should* deprecate them as
long as we don't have WAL support, but that viewpoint is not universally
subscribed to.

One of the arguments against Bruce's proposal to print a warning at hash
index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by some
app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#8David G. Johnston
david.g.johnston@gmail.com
In reply to: Tom Lane (#5)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Tom Lane-2 wrote

Bruce Momjian &lt;

bruce@

&gt; writes:

On Thu, May 15, 2014 at 05:20:35PM +0200, Olivier Macchioni wrote:

I guess my best bet is to replace it by another kind of indexes... and
maybe one day PostgreSQL will be clever enough to issue a warning /
error in such a case for the people like me who don't read *all the doc*
:P

Yes, streaming replication has made our hash indexes even worse. In the
past, I have suggested we issue a warning for the creation of hash
indexes, but did not get enough agreement.

Mainly because it wouldn't be a very helpful message.

I wonder though if we could throw a flat-out error for attempts to use
a hash index on a hot standby server. That would get people's attention
without being mere nagging in other situations. It's not a 100% solution
because you'd still lose if you tried to use a hash index on a slave
since promoted to master. But it would help without being a large
sink for effort.

At least a promoted slave can "REINDEX" and get back to functioning with
minimal fuss.

Side question: if one were to do this intentionally is there a recommended
way to have the REINDEX run immediately upon the former slave becoming
promoted?

I have to presume there is some reason why we do not currently resolve;

base/56100265/57047884

into something more useful. It is obviously possible since oid2name exists.
I suspect some of it may just be "hasn't been worth the effort" and some of
it is "expensive to compute and if the error is happening repeatedly it
could bog down the system". But knowing what type of relation is affected,
and conditionally reporting additional diagnostic detail based upon that
type, has value since, as this case shows, when an error like this arises
the typical user is going into a state of panic and very little info is
immediately at hand to couch that for non-critical situations.

OpenERP should be more helpful in their own right since they know they are
using a feature with limitations; though given the lack of complaints we are
not that popular with them and/or people are not using hot-standby slaves
for queries on this particular table.

All that said I don't see how it would really hurt to issue a notice upon
creation of a hash index. Providing multiple opportunities for someone to
see the message, question its meaning, and learn why it is being issued
would decrease the chances of people being surprised; and I cannot imagine
the check for index type, and the resultant logging, would be considered
expensive relative to how long a CREATE INDEX typically would run.

David J.

--
View this message in context: http://postgresql.1045698.n5.nabble.com/BUG-10329-Could-not-read-block-0-in-file-base-56100265-57047884-read-only-0-of-8192-bytes-tp5804037p5804123.html
Sent from the PostgreSQL - bugs mailing list archive at Nabble.com.

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#9Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#7)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Thu, May 15, 2014 at 8:25 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

One of the arguments against Bruce's proposal to print a warning at hash
index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by some
app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

I suggested we make a GUC allow_unrecoverable_indexes and default it
to false. If you want to create hash indexes you need to set it to
true or else you just get errors.

A more general solution is to emit a WAL record the first time a
non-crashsafe index is touched after a checkpoint. On a slave that
record could just mark the index invalid.

--
greg

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#10Andres Freund
andres@anarazel.de
In reply to: Bruce Momjian (#9)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On 2014-05-16 12:58:57 +0100, Greg Stark wrote:

On Thu, May 15, 2014 at 8:25 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

One of the arguments against Bruce's proposal to print a warning at hash
index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by some
app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

I suggested we make a GUC allow_unrecoverable_indexes and default it
to false. If you want to create hash indexes you need to set it to
true or else you just get errors.

Fine with me.

A more general solution is to emit a WAL record the first time a
non-crashsafe index is touched after a checkpoint. On a slave that
record could just mark the index invalid.

Not trivially no. Recovery can't write to the catalog.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#11Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#9)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Greg Stark <stark@mit.edu> writes:

On Thu, May 15, 2014 at 8:25 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

One of the arguments against Bruce's proposal to print a warning at hash
index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by some
app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

I suggested we make a GUC allow_unrecoverable_indexes and default it
to false. If you want to create hash indexes you need to set it to
true or else you just get errors.

I still think this is throwing the error at the wrong place. People
will turn on the GUC the first time it gets in their way, and then
much later discover that the index doesn't work on a slave, and we'll
get a bug report exactly like this one. We need a check that is tightly
connected to actual unsafe usage, rather than basically-user-unfriendly
complaints at a point that's not doing anything unsafe. (Well, anything
more unsafe than it ever was.)

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#12Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#11)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Fri, May 16, 2014 at 2:45 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I still think this is throwing the error at the wrong place. People
will turn on the GUC the first time it gets in their way, and then
much later discover that the index doesn't work on a slave, and we'll
get a bug report exactly like this one. We need a check that is tightly
connected to actual unsafe usage, rather than basically-user-unfriendly
complaints at a point that's not doing anything unsafe. (Well, anything
more unsafe than it ever was.)

So there three cases where crash recovery is necessary:

1) A database crashes and does recovery when it starts up

2) A database has a hot backup taken and it's restored in a whole new place

3) A standby is replaying logs and has hot standby queries against it
or is activated and runs normally

The first two are actually safe as long as the index is quiescent.
That is, if the database crashes and there were no writes in progress
or the machine crashes and there were no writes since the last
checkpoint then the index should still be valid after a restart. A hot
backup should be fine if there were no writes to the index during the
hot backup. Obviously this is a precarious state of affairs but I
guess you're right that it's unchanged from the historical situation.

The index on the standby is a bit different. If the index has never
been touched since before the standby was created then it's ok (and
incidentally Heroku does see the occasional customer in that situation
-- they have hash indexes on archived partitions for data warehouse
queries). But if there's been a write ever since then even if it was a
long time ago the standby index is unusable. That's a lot more likely
than a backup taken while the index was being updated.

I take it you're arguing we should have an error in the standby if it
tries to use a hash index? That's seems reasonable. It might be good
to have a guc to override it (unfortunately we can't turn it into a
warning since it m

--
greg

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#13Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#11)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On 2014-05-16 09:45:02 -0400, Tom Lane wrote:

Greg Stark <stark@mit.edu> writes:

On Thu, May 15, 2014 at 8:25 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

One of the arguments against Bruce's proposal to print a warning at hash
index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by some
app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

I suggested we make a GUC allow_unrecoverable_indexes and default it
to false. If you want to create hash indexes you need to set it to
true or else you just get errors.

I still think this is throwing the error at the wrong place. People
will turn on the GUC the first time it gets in their way, and then
much later discover that the index doesn't work on a slave, and we'll
get a bug report exactly like this one.

If there's one index within many as in this case it's likely to be
accidental, right? So I think such a GUC might have helped to prevent
the problem.

We need a check that is tightly
connected to actual unsafe usage, rather than basically-user-unfriendly
complaints at a point that's not doing anything unsafe. (Well, anything
more unsafe than it ever was.)

I agree that that'd be nicer, but I haven't seen a nice proposal of how
to do that yet.
The best I can think of is to WAL LOG the removal of the entire relation
the first time a hash index is used in a session, replacing it with a
metapage that errors out when used. That should be doable.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#14Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#11)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Fri, May 16, 2014 at 09:45:02AM -0400, Tom Lane wrote:

Greg Stark <stark@mit.edu> writes:

On Thu, May 15, 2014 at 8:25 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

One of the arguments against Bruce's proposal to print a warning at hash
index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by some
app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

I suggested we make a GUC allow_unrecoverable_indexes and default it
to false. If you want to create hash indexes you need to set it to
true or else you just get errors.

I still think this is throwing the error at the wrong place. People
will turn on the GUC the first time it gets in their way, and then
much later discover that the index doesn't work on a slave, and we'll
get a bug report exactly like this one. We need a check that is tightly
connected to actual unsafe usage, rather than basically-user-unfriendly
complaints at a point that's not doing anything unsafe. (Well, anything
more unsafe than it ever was.)

Well, at this point we are decade into having crash-unsafe hash indexes,
and rather than getting better, the issue has gotten worse with
streaming replication. If we can't find the best way to warn people,
let's find _a_ way, at least.

I feel we are waiting for the calvary to come over the hill (and fix
hash indexes), except the calvary never arrives. At some point we have
to take ownership of the situation we are in and actively do something.

If someone today tried to add a crash-unsafe, replication-impotent
index, it would never be accepted, but because hash indexes came from
Berkeley, we go with a warning in the CREATE INDEX manual page and do
nothing more. I can't think of any other foot-gun feature that is
allowed to remain with so little user warning.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#15David G. Johnston
david.g.johnston@gmail.com
In reply to: Bruce Momjian (#14)
Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Fri, May 16, 2014 at 10:49 AM, Bruce Momjian [via PostgreSQL] <
ml-node+s1045698n5804229h23@n5.nabble.com> wrote:

On Fri, May 16, 2014 at 09:45:02AM -0400, Tom Lane wrote:

Greg Stark <[hidden email]<http://user/SendEmail.jtp?type=node&amp;node=5804229&amp;i=0&gt;&gt;

writes:

On Thu, May 15, 2014 at 8:25 PM, Tom Lane <[hidden email]<http://user/SendEmail.jtp?type=node&amp;node=5804229&amp;i=1&gt;&gt;

wrote:

One of the arguments against Bruce's proposal to print a warning at

hash

index creation is that it's a particularly ineffective form of
deprecation. In your example, since the hash index was created by

some

app not manually, I'll bet nobody would have seen/noticed the warning
even if there had been one.

I suggested we make a GUC allow_unrecoverable_indexes and default it
to false. If you want to create hash indexes you need to set it to
true or else you just get errors.

I still think this is throwing the error at the wrong place. People
will turn on the GUC the first time it gets in their way, and then
much later discover that the index doesn't work on a slave, and we'll
get a bug report exactly like this one. We need a check that is tightly
connected to actual unsafe usage, rather than basically-user-unfriendly
complaints at a point that's not doing anything unsafe. (Well, anything
more unsafe than it ever was.)

Well, at this point we are decade into having crash-unsafe hash indexes,
and rather than getting better, the issue has gotten worse with
streaming replication. If we can't find the best way to warn people,
let's find _a_ way, at least.

I feel we are waiting for the calvary to come over the hill (and fix
hash indexes), except the calvary never arrives. At some point we have
to take ownership of the situation we are in and actively do something.

If someone today tried to add a crash-unsafe, replication-impotent
index, it would never be accepted, but because hash indexes came from
Berkeley, we go with a warning in the CREATE INDEX manual page and do
nothing more. I can't think of any other foot-gun feature that is
allowed to remain with so little user warning.

​+1

Unless there is a convincing argument to be made why doing it at CREATE
INDEX is FRIGGIN' EVIL then who really cares if its not perfect.

NOTICE: This index IS NOT WAL LOGGED and cannot be used on SLAVE servers or
AFTER RECOVERY. See Documentation for Details!

The goal should be to communicate FUD to the uninformed.

I'm all for additional and improved warnings in other places but this one
at least seems to have the benefit of being relatively simple to implement
and non-obnoxious since it only is issued once per index creation.

As devil's advocate it isn't like anyone is likely to intentionally use
hash indexes without reading the documentation first - if only to know they
exist, how they work, and what syntax to use. If an application is
installing such indexes for the user then a warning at CREATE INDEX is only
a little better than a warning in the documentation - though both are
likely to never be seen.

But that argument doesn't hold any sway for me. If someone knows they are
using a hash index intentionally then the notice/warning will be understood
and ignored while if someone is seeing the notice/warning and are not aware
of the limitations of hash indexes - like the OP - this live/logged notice
will hopefully cause them to become better informed and able to evaluate
their specific situation. If the application doesn't point out it is using
hash indexes then the typical user will not be checking PostgreSQL
documentation for the same; but just maybe the notice that is raised will
end up visible to the end-user or cause the application developers to
further re-examine their usage and/or documentation of hash indexes.

David J.

--
View this message in context: http://postgresql.1045698.n5.nabble.com/BUG-10329-Could-not-read-block-0-in-file-base-56100265-57047884-read-only-0-of-8192-bytes-tp5804037p5804238.html
Sent from the PostgreSQL - bugs mailing list archive at Nabble.com.

#16Bruce Momjian
bruce@momjian.us
In reply to: David G. Johnston (#15)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Fri, May 16, 2014 at 08:27:33AM -0700, David G Johnston wrote:

Well, at this point we are decade into having crash-unsafe hash indexes,
and rather than getting better, the issue has gotten worse with
streaming replication. �If we can't find the best way to warn people,
let's find _a_ way, at least.

I feel we are waiting for the calvary to come over the hill (and fix
hash indexes), except the calvary never arrives. �At some point we have
to take ownership of the situation we are in and actively do something.

If someone today tried to add a crash-unsafe, replication-impotent
index, it would never be accepted, but because hash indexes came from
Berkeley, we go with a warning in the CREATE INDEX manual page and do
nothing more. �I can't think of any other foot-gun feature that is
allowed to remain with so little user warning.

Unless there is a convincing argument to be made why doing it at CREATE INDEX
is FRIGGIN' EVIL then who really cares if its not perfect.

NOTICE: This index IS NOT WAL LOGGED and cannot be used on SLAVE servers or
AFTER RECOVERY. �See Documentation for Details!

The goal should be to communicate FUD to the uninformed.

I'm all for additional and improved warnings in other places but this one at
least seems to have the benefit of being relatively simple to implement and
non-obnoxious since it only is issued once per index creation.

As devil's advocate it isn't like anyone is likely to intentionally use hash
indexes without reading the documentation first - if only to know they exist,
how they work, and what syntax to use. �If an application is installing such
indexes for the user then a warning at CREATE INDEX is only a little better
than a warning in the documentation - though both are likely to never be seen.
�

But that argument doesn't hold any sway for me. If someone knows they are using
a hash index intentionally then the notice/warning will be understood and
ignored while if someone is seeing the notice/warning and are not aware of the
limitations of hash indexes - like the OP - this live/logged notice will
hopefully cause them to become better informed and able to evaluate their
specific situation. �If the application doesn't point out it is using hash
indexes then the typical user will not be checking PostgreSQL documentation for
the same; but just maybe the notice that is raised will end up visible to the
end-user or cause the application developers to further re-examine their usage
and/or documentation of hash indexes.

Here is a patch which implements the warning during CREATE INDEX ...
HASH. If WAL-logging of hash indexes is ever implemented, we can remove
this warning.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

Attachments:

hash.difftext/x-diff; charset=us-asciiDownload+12-0
#17Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Bruce Momjian (#16)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Bruce Momjian wrote:

Here is a patch which implements the warning during CREATE INDEX ...
HASH. If WAL-logging of hash indexes is ever implemented, we can remove
this warning.

I think we should have CREATE UNLOGGED INDEX, and simply disallow any
hash index from being created unless it's marked as such.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#18Bruce Momjian
bruce@momjian.us
In reply to: Alvaro Herrera (#17)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Sat, Sep 6, 2014 at 09:42:45PM -0400, Alvaro Herrera wrote:

Bruce Momjian wrote:

Here is a patch which implements the warning during CREATE INDEX ...
HASH. If WAL-logging of hash indexes is ever implemented, we can remove
this warning.

I think we should have CREATE UNLOGGED INDEX, and simply disallow any
hash index from being created unless it's marked as such.

Wow, that sounds much more radical than we discussed. Seeing I got
push-back just for the warning, I don't see how disabling "logged" WAL
indexes is going to be accepted.

It is a good idea, though. :-)

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#19Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#16)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

On Sat, Sep 6, 2014 at 11:07:43AM -0400, Bruce Momjian wrote:

Here is a patch which implements the warning during CREATE INDEX ...
HASH. If WAL-logging of hash indexes is ever implemented, we can remove
this warning.

Applied, though I used the term "streaming standbys" to match our docs.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#20Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#19)
Re: Re: BUG #10329: Could not read block 0 in file "base/56100265/57047884": read only 0 of 8192 bytes

Bruce Momjian <bruce@momjian.us> wrote:

On Sat, Sep 6, 2014 at 11:07:43AM -0400, Bruce Momjian wrote:

Here is a patch which implements the warning during CREATE INDEX ...
HASH. If WAL-logging of hash indexes is ever implemented, we can remove
this warning.

Applied, though I used the term "streaming standbys" to match our docs.

Hmm. The wording of the warning doesn't seem to really indicate
the full scope of the limitation. Any a standby (warm or hot)
maintained by WAL file copying would also be affected (i.e.,
streaming replication as the WAL delivery mechanism is irrelevant),
and you also have problems after a database crash or PANIC. I'm
not sure how to state that concisely, though.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#21Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#20)
#22Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#21)
#23Stefan Kaltenbrunner
stefan@kaltenbrunner.cc
In reply to: Bruce Momjian (#18)
#24Andres Freund
andres@anarazel.de
In reply to: Stefan Kaltenbrunner (#23)
#25Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#22)
#26Bruce Momjian
bruce@momjian.us
In reply to: Andres Freund (#24)
#27Andres Freund
andres@anarazel.de
In reply to: Bruce Momjian (#26)
#28Stefan Kaltenbrunner
stefan@kaltenbrunner.cc
In reply to: Andres Freund (#27)
#29Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#27)
#30Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#29)