Re: WIP: rewrite numeric division

Gregory Stark <stark@enterprisedb.com> writes:
> "Tom Lane" <tgl@sss.pgh.pa.us> writes:
>> Well, this doesn't take a lot of numerical methods background: the
>> fundamental problem is that the existing code generates an *approximate*
>> answer, whereas people who are doing div and mod on large integers tend
>> to expect an *exact* answer.  Approximate doesn't cut it --- there will
>> always be cases where an off-by-one-in-the-last-internal-place error can
>> carry far enough to the left to be visible to the user.

> So does your code behave differently for this or does it round off to the
> arbitrary division precision before hitting trunc?

> postgres=# select trunc(99999999999999999999::numeric / 1000000000::numeric);
>     trunc
> --------------
>  100000000000
> (1 row)

No, my proposed patch doesn't change that.  It might be that we should
provide an "integer division" operator for NUMERIC, so that you can get
at the exact result of trunc(x/y).

            regards, tom lane