BUG #5975: Incorrect result from mod function with cast
The following bug has been logged online:
Bug reference: 5975
Logged by: John Dickson
Email address: jdickson@tnsi.com
PostgreSQL version: 8.3.11
Operating system: RedHat EL 5.6
Description: Incorrect result from mod function with cast
Details:
The mod function (and the % operator) can return an incorrect result when
combined with a cast().
The following SQL (executed on 8.3.11) shows a result one higher than the
maximum permissible remainder when combined with a cast:
-----
select 1129590 % 66,
cast(1129590 as numeric(21, 0)) % 66;
?column? | ?column?
----------+----------
0 | 66
(1 row)
-----
When executed on Postgres 8.2.5 (on Centos 4.6), the results are:
-----
?column? | ?column?
----------+----------
0 | 0
(1 row)
-----
We're using the cast here to demonstrate the bug, but we actually strike the
problem when operating on values from a numeric(21, 0) column rather than
using cast().
We haven't tested this on later releases of 8.3, but didn't see any likely
fixes in the changelog.
"John Dickson" <jdickson@tnsi.com> writes:
The following SQL (executed on 8.3.11) shows a result one higher than the
maximum permissible remainder when combined with a cast:
select 1129590 % 66,
cast(1129590 as numeric(21, 0)) % 66;
?column? | ?column?
----------+----------
0 | 66
This is basically a roundoff issue. It's fixed in 8.4 and up, as a
result of this patch:
Author: Tom Lane <tgl@sss.pgh.pa.us>
Branch: master Release: REL8_4_BR [a0fad9762] 2008-04-04 18:45:36 +0000
Re-implement division for numeric values using the traditional "schoolbook"
algorithm. This is a good deal slower than our old roundoff-error-prone
code for long inputs, so we keep the old code for use in the transcendental
functions, where everything is approximate anyway. Also create a
user-accessible function div(numeric, numeric) to provide access to the
exact result of trunc(x/y) --- since the regular numeric / operator will
round off its result, simply computing that expression in SQL doesn't
reliably give the desired answer. This fixes bug #3387 and various related
corner cases, and improves the usefulness of PG for high-precision integer
arithmetic.
We felt at the time that it was inappropriate to back-patch such a
behavioral change, and I doubt that decision is going to change now.
I'd suggest considering an update to 8.4 if this is a critical issue
for you.
regards, tom lane