Numeric modulo operator is incorrect.

Started by Ken-ichi Yamasakiabout 25 years ago2 messagesbugs
Jump to latest
#1Ken-ichi Yamasaki
yama@yuzu.gr.jp

Dear sir,

I build PostgreSQL 7.0.3 with jumbo-20010318.patch.gz on Vine Linux2.1.5.

Following result are incorrect. Am I under an illusion?

test=# select '34666624082'::numeric % '34666671'::numeric;
?column?
----------
-46918
(1 row)

test=# select '34666624082'::numeric - '34666671'::numeric
* trunc('34666624082'::numeric / '34666671'::numeric);
?column?
----------
34619753
(1 row)

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Ken-ichi Yamasaki (#1)
Re: Numeric modulo operator is incorrect.

Ken-ichi Yamasaki <yama@yuzu.gr.jp> writes:

I build PostgreSQL 7.0.3 with jumbo-20010318.patch.gz on Vine Linux2.1.5.

Following result are incorrect. Am I under an illusion?

test=# select '34666624082'::numeric % '34666671'::numeric;
?column?
----------
-46918
(1 row)

This is a bug; looks like mod_var() is using a rather silly estimate
of how many digits of precision it needs to keep in the division step.

I have a fix but I'm not sure if it will get into 7.1 or be held
for 7.1.1 ...

regards, tom lane