Bug #10432 failed to re-find parent key in index

Started by Joshua D. Drakealmost 11 years ago13 messages
#1Joshua D. Drake
jd@commandprompt.com

Hello,

We have a database that has run into this problem. The version is 9.1.15
on Linux. I note in this thread:

/messages/by-id/CAM-w4HP34ppweGTCwjBZnWhQ0CmU-Lxna62vjKU8QRTWLOBuHw@mail.gmail.com

That things appear to be fixed in 9.4 but they have not been
back-patched? What is the current status?

Sincerely,

jD
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Peter Geoghegan
pg@heroku.com
In reply to: Joshua D. Drake (#1)
Re: Bug #10432 failed to re-find parent key in index

On Mon, Mar 30, 2015 at 7:50 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

We have a database that has run into this problem. The version is 9.1.15 on
Linux. I note in this thread:

/messages/by-id/CAM-w4HP34ppweGTCwjBZnWhQ0CmU-Lxna62vjKU8QRTWLOBuHw@mail.gmail.com

That things appear to be fixed in 9.4 but they have not been back-patched?
What is the current status?

I believe that Heikki said he'd backpatch that when 9.4 was considered
very stable. I don't think that we've reached that level of confidence
in the invasive B-Tree bugfixes that went into 9.4 yet.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Joshua D. Drake
jd@commandprompt.com
In reply to: Peter Geoghegan (#2)
Re: Bug #10432 failed to re-find parent key in index

Hello,

Just wondering if what Peter said was the last word on this?

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Peter Geoghegan (#2)
Re: Bug #10432 failed to re-find parent key in index

On 03/30/2015 09:57 PM, Peter Geoghegan wrote:

On Mon, Mar 30, 2015 at 7:50 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

We have a database that has run into this problem. The version is 9.1.15 on
Linux. I note in this thread:

/messages/by-id/CAM-w4HP34ppweGTCwjBZnWhQ0CmU-Lxna62vjKU8QRTWLOBuHw@mail.gmail.com

That things appear to be fixed in 9.4 but they have not been back-patched?
What is the current status?

I believe that Heikki said he'd backpatch that when 9.4 was considered
very stable. I don't think that we've reached that level of confidence
in the invasive B-Tree bugfixes that went into 9.4 yet.

I have no intention to backpatch the changes. Too big, too invasive.
Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Joshua D. Drake
jd@commandprompt.com
In reply to: Heikki Linnakangas (#4)
Re: Bug #10432 failed to re-find parent key in index

On 03/31/2015 04:20 AM, Heikki Linnakangas wrote:

I believe that Heikki said he'd backpatch that when 9.4 was considered
very stable. I don't think that we've reached that level of confidence
in the invasive B-Tree bugfixes that went into 9.4 yet.

I have no intention to backpatch the changes. Too big, too invasive.

I can certainly appreciate that.

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Andres Freund
andres@2ndquadrant.com
In reply to: Joshua D. Drake (#5)
Re: Bug #10432 failed to re-find parent key in index

On 2015-03-31 10:49:06 -0700, Joshua D. Drake wrote:

On 03/31/2015 04:20 AM, Heikki Linnakangas wrote:

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

Man.

Easy for you to say. You're not doing the work (which would be
significant in this case). You're not going to be blamed if the backport
breaks more things than it fixed.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Robert Haas
robertmhaas@gmail.com
In reply to: Joshua D. Drake (#5)
Re: Bug #10432 failed to re-find parent key in index

On Tue, Mar 31, 2015 at 1:49 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

I don't agree. When a bug fix is really invasive, there is a
considerable risk that it will itself have bugs. We've got to balance
the risk of fixing things for users who are currently having problems
with the risk of creating problems for people who currently aren't
having any.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Josh Berkus
josh@agliodbs.com
In reply to: Joshua D. Drake (#1)
Re: Bug #10432 failed to re-find parent key in index

On 03/31/2015 04:20 AM, Heikki Linnakangas wrote:

On 03/30/2015 09:57 PM, Peter Geoghegan wrote:

On Mon, Mar 30, 2015 at 7:50 PM, Joshua D. Drake
<jd@commandprompt.com> wrote:

We have a database that has run into this problem. The version is
9.1.15 on
Linux. I note in this thread:

/messages/by-id/CAM-w4HP34ppweGTCwjBZnWhQ0CmU-Lxna62vjKU8QRTWLOBuHw@mail.gmail.com

That things appear to be fixed in 9.4 but they have not been
back-patched?
What is the current status?

I believe that Heikki said he'd backpatch that when 9.4 was considered
very stable. I don't think that we've reached that level of confidence
in the invasive B-Tree bugfixes that went into 9.4 yet.

I have no intention to backpatch the changes. Too big, too invasive.
Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

We could use somewhere for users to find out about this kind of issue.
That is, for users to know that they can fix it by upgrading to 9.4, but
not otherwise. Ideas?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Joshua D. Drake
jd@commandprompt.com
In reply to: Andres Freund (#6)
Re: Bug #10432 failed to re-find parent key in index

On 03/31/2015 10:51 AM, Andres Freund wrote:

On 2015-03-31 10:49:06 -0700, Joshua D. Drake wrote:

On 03/31/2015 04:20 AM, Heikki Linnakangas wrote:

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

Man.

Easy for you to say. You're not doing the work (which would be
significant in this case). You're not going to be blamed if the backport
breaks more things than it fixed.

I understand that. I am not picking on anyone. I am just saying that
looking at the problem this way is poor, which it is. We are saying as a
community: Your option to remove this data loss bug is to upgrade. That
is generally not how we approach things.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Joshua D. Drake
jd@commandprompt.com
In reply to: Robert Haas (#7)
Re: Bug #10432 failed to re-find parent key in index

On 03/31/2015 10:58 AM, Robert Haas wrote:

On Tue, Mar 31, 2015 at 1:49 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

I don't agree. When a bug fix is really invasive, there is a
considerable risk that it will itself have bugs.

Absolutely.

We've got to balance
the risk of fixing things for users who are currently having problems
with the risk of creating problems for people who currently aren't
having any.

We are not in disagreement.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Joshua D. Drake
jd@commandprompt.com
In reply to: Josh Berkus (#8)
Re: Bug #10432 failed to re-find parent key in index

On 03/31/2015 11:05 AM, Josh Berkus wrote:

I have no intention to backpatch the changes. Too big, too invasive.
Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

We could use somewhere for users to find out about this kind of issue.
That is, for users to know that they can fix it by upgrading to 9.4, but
not otherwise. Ideas?

That is a good idea. It also opens up the ability for us to be more
proactive about upgrades. The more complex we get, the more likely this
type of problem is going to arise. Some type of deficiency to upgrade
matrix (similar to our feature matrix) might be a good idea.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#7)
Re: Bug #10432 failed to re-find parent key in index

Robert Haas <robertmhaas@gmail.com> writes:

On Tue, Mar 31, 2015 at 1:49 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

I don't agree. When a bug fix is really invasive, there is a
considerable risk that it will itself have bugs. We've got to balance
the risk of fixing things for users who are currently having problems
with the risk of creating problems for people who currently aren't
having any.

It should also be noted that there is very little reason to assume that
whatever issue JD saw on his 9.1.15 system would have been prevented by
Heikki's changes. We've seen many instances of "failed to re-find parent
key" errors over the years, with widely varying root causes (when we were
able to find the root cause).

Personally I'm fine with Heikki's opinion that the costs/risks of
backporting outweigh the likely benefits. I'm certainly on board with
the idea that we wouldn't do it for another year or so ... by which
time 9.1 will be out of support or nearly so.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Joshua D. Drake (#9)
Re: Bug #10432 failed to re-find parent key in index

On 03/31/2015 09:19 PM, Joshua D. Drake wrote:

On 03/31/2015 10:51 AM, Andres Freund wrote:

On 2015-03-31 10:49:06 -0700, Joshua D. Drake wrote:

On 03/31/2015 04:20 AM, Heikki Linnakangas wrote:

Perhaps we could consider it after a year or two, once 9.4 is indeed
very stable, but at that point you have to wonder if it's really worth
the trouble anymore. If someone has runs into that issue frequently, he
probably should just upgrade to 9.4.

Ouch. That is a really poor way to look at this.

Man.

Easy for you to say. You're not doing the work (which would be
significant in this case). You're not going to be blamed if the backport
breaks more things than it fixed.

I understand that. I am not picking on anyone. I am just saying that
looking at the problem this way is poor, which it is. We are saying as a
community: Your option to remove this data loss bug is to upgrade. That
is generally not how we approach things.

Hmm, I've never considered this to be a data loss bug. I guess you can
view it that way: if you have a standby following the master, and the
master fails so that you fail over to the standby, the standby will
refuse to start up because of this, so you can't access the data.
However, the table itself is OK, it's just the index that's corrupt.
You'll need some hackery to force the system out of standby mode, but
it's not like the data has been overwritten and lost forever.

Greg Stark suggested downgrading the error to warning during recovery
mode, so that the error would not prevent you from starting up the
system. That makes a lot of sense, I think we should do that in the
back-branches.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers