pgsql: Avoid gratuitous inaccuracy in numeric width_bucket().

Started by Tom Lanealmost 6 years ago1 messagescomitters
Jump to latest
#1Tom Lane
tgl@sss.pgh.pa.us

Avoid gratuitous inaccuracy in numeric width_bucket().

Multiply before dividing, not the reverse, so that cases that should
produce exact results do produce exact results. (width_bucket_float8
got this right already.) Even when the result is inexact, this avoids
making it more inexact, since only the division step introduces any
imprecision.

While at it, fix compute_bucket() to not uselessly repeat the sign
check already done by its caller, and avoid duplicating the
multiply/divide steps by adjusting variable usage.

Per complaint from Martin Visser. Although this seems like a bug fix,
I'm hesitant to risk changing width_bucket()'s results in stable
branches, so no back-patch.

Discussion: /messages/by-id/6FA5117D-6AED-4656-8FEF-B74AC18FAD85@brytlyt.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/7538708394e7a70105a4e601e253adf80f47cca8

Modified Files
--------------
src/backend/utils/adt/numeric.c | 31 ++++++++++++++-------------
src/test/regress/expected/numeric.out | 40 +++++++++++++++++++++++++++++++++++
src/test/regress/sql/numeric.sql | 9 ++++++++
3 files changed, 65 insertions(+), 15 deletions(-)