modulo operator bug?

Kemal Bayram <kemal@corenetworks.co.uk> writes:

from psql:
kemal=# select 0.99 % 0.50;
?column?
----------
0.04
(1 row)

Am I missing something fundemental here or is this indeed a bug?

Looks like a bug to me too. mod_var shouldn't be using tmp.rscale,
I think.

regards, tom lane

Kemal Bayram <kemal@corenetworks.co.uk> writes:

from psql:
kemal=# select 0.99 % 0.50;
?column?
----------
0.04
(1 row)

Am I missing something fundemental here or is this indeed a bug?

Looks like a bug to me too. mod_var shouldn't be using tmp.rscale,
I think.

OK, fixed for 7.1. If you need a fix in 7.0.*, the attached patch should
apply to 7.0.*, although the line numbers will be different.

regards, tom lane

*** src/backend/utils/adt/numeric.c.orig	Wed Dec  6 21:47:35 2000
--- src/backend/utils/adt/numeric.c	Wed Mar 14 11:50:37 2001
***************
*** 3355,3370 ****
  	init_var(&tmp);

/* ----------
! * We do it by fiddling around with global_rscale and truncating
! * the result of the division.
* ----------
*/
save_global_rscale = global_rscale;
global_rscale = var2->rscale + 2;

div_var(var1, var2, &tmp);
tmp.rscale = var2->rscale;
- tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + tmp.rscale + 1));

  	global_rscale = var2->rscale;
  	mul_var(var2, &tmp, &tmp);
--- 3355,3373 ----
  	init_var(&tmp);

/* ----------
! * We do this using the equation
! * mod(x,y) = x - trunc(x/y)*y
! * We fiddle a bit with global_rscale to control result precision.
* ----------
*/
save_global_rscale = global_rscale;
global_rscale = var2->rscale + 2;

  	div_var(var1, var2, &tmp);
+ 
+ 	/* do trunc() by forgetting digits to the right of the decimal point */
+ 	tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + 1));
  	tmp.rscale = var2->rscale;

global_rscale = var2->rscale;
mul_var(var2, &tmp, &tmp);