Re: modulo operator bug?

> Kemal Bayram <kemal@corenetworks.co.uk> writes:
>> from psql:
>> kemal=# select 0.99 % 0.50;
>> ?column?
>> ----------
>> 0.04
>> (1 row)

>> Am I missing something fundemental here or is this indeed a bug?

> Looks like a bug to me too.  mod_var shouldn't be using tmp.rscale,
> I think.

OK, fixed for 7.1.  If you need a fix in 7.0.*, the attached patch should
apply to 7.0.*, although the line numbers will be different.

            regards, tom lane


*** src/backend/utils/adt/numeric.c.orig    Wed Dec  6 21:47:35 2000
--- src/backend/utils/adt/numeric.c    Wed Mar 14 11:50:37 2001
***************
*** 3355,3370 ****
      init_var(&tmp);

      /* ----------
!      * We do it by fiddling around with global_rscale and truncating
!      * the result of the division.
       * ----------
       */
      save_global_rscale = global_rscale;
      global_rscale = var2->rscale + 2;

      div_var(var1, var2, &tmp);
      tmp.rscale = var2->rscale;
-     tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + tmp.rscale + 1));

      global_rscale = var2->rscale;
      mul_var(var2, &tmp, &tmp);
--- 3355,3373 ----
      init_var(&tmp);

      /* ----------
!      * We do this using the equation
!      *        mod(x,y) = x - trunc(x/y)*y
!      * We fiddle a bit with global_rscale to control result precision.
       * ----------
       */
      save_global_rscale = global_rscale;
      global_rscale = var2->rscale + 2;

      div_var(var1, var2, &tmp);
+
+     /* do trunc() by forgetting digits to the right of the decimal point */
+     tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + 1));
      tmp.rscale = var2->rscale;

      global_rscale = var2->rscale;
      mul_var(var2, &tmp, &tmp);