Re: Incorrect results from numeric round() and trunc()

On Mon, 8 Jul 2024 at 00:40, Joel Jacobson <joel@compiler.org> wrote:
>
> On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote:
> > I've also tidied up a bit by replacing all instances of SHRT_MAX with
> > a new constant NUMERIC_WEIGHT_MAX, whose name more accurately
> > describes the limit, as used in various other overflow checks.
>
> Having thought a bit more on this, I think we probably need a
> DEC_DIGITS sensitive definition of NUMERIC_WEIGHT_MAX,
> since per spec the max range for numeric is 0x20000 (131072)
> decimal digits.
>

No, the maximum weight is determined by the use of int16 to store the
weight. Therefore if you did reduce DEC_DIGITS to 1 or 2, the number
of decimal digits allowed before the decimal point would be reduced
too.

Regards,
Dean