pgsql: Protect GIST logic that assumes penalty values can't be negative
Protect GIST logic that assumes penalty values can't be negative.
Apparently sane-looking penalty code might return small negative values,
for example because of roundoff error. This will confuse places like
gistchoose(). Prevent problems by clamping negative penalty values to
zero. (Just to be really sure, I also made it force NaNs to zero.)
Back-patch to all supported branches.
Alexander Korotkov
Branch
------
REL8_3_STABLE
Details
-------
http://git.postgresql.org/pg/commitdiff/ca76a3956c80f97eb4ea0cc8fc2a54435f6a573b
Modified Files
--------------
doc/src/sgml/gist.sgml | 2 ++
src/backend/access/gist/gistutil.c | 14 +++++++++++---
2 files changed, 13 insertions(+), 3 deletions(-)
On Tue, May 31, 2011 at 2:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Prevent problems by clamping negative penalty values to
zero. (Just to be really sure, I also made it force NaNs to zero.)
Do gistchoose et al expect the triangle function to obey the triangle
inequality? If so isn't it possible treating NaNs as zero would fail
that? I'm not sure there's any safe assumption for NaN
--
greg
Greg Stark <gsstark@mit.edu> writes:
On Tue, May 31, 2011 at 2:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
�Prevent problems by clamping negative penalty values to
zero. �(Just to be really sure, I also made it force NaNs to zero.)
Do gistchoose et al expect the triangle function to obey the triangle
inequality?
Don't think so.
If so isn't it possible treating NaNs as zero would fail
that? I'm not sure there's any safe assumption for NaN
Well, leaving it as NaN is almost certain to not work desirably.
regards, tom lane
On Tue, May 31, 2011 at 4:11 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Do gistchoose et al expect the triangle function to obey the triangle
inequality?Don't think so.
I guess it was obvious but that was "expect the *penalty* function to
obey the triangle inequality"
--
greg
On Wed, Jun 1, 2011 at 3:57 AM, Greg Stark <gsstark@mit.edu> wrote:
I guess it was obvious but that was "expect the *penalty* function to
obey the triangle inequality"
Actually, penalty function for boxes is even not commutative. Fox example:
A = ((0,0)-(1,1))
B = ((0,0)-(2,2))
penalty(A,B) = 3
penalty(B,A) = 0
------
With best regards,
Alexander Korotkov.