Re: Incorrect results from numeric round() and trunc()
От | Joel Jacobson |
---|---|
Тема | Re: Incorrect results from numeric round() and trunc() |
Дата | |
Msg-id | c65c7129-50c6-4850-872b-b561701f99bf@app.fastmail.com обсуждение исходный текст |
Ответ на | Incorrect results from numeric round() and trunc() (Dean Rasheed <dean.a.rasheed@gmail.com>) |
Список | pgsql-hackers |
On Sun, Jul 7, 2024, at 13:28, Dean Rasheed wrote: > The numeric round() and trunc() functions clamp the scale argument to > the range between +/- NUMERIC_MAX_RESULT_SCALE, which is +/- 2000. > That's a long way short of the actual allowed range of type numeric, > so they produce incorrect results when rounding/truncating more than > 2000 digits before or after the decimal point. For example, > round(1e-5000, 5000) returns 0 instead of 1e-5000. > > Attached is a patch fixing that, using the actual documented range of > type numeric. > > I've also tidied up a bit by replacing all instances of SHRT_MAX with > a new constant NUMERIC_WEIGHT_MAX, whose name more accurately > describes the limit, as used in various other overflow checks. > > In doing so, I also noticed a comment in power_var() which claimed > that ln_dweight could be as low as -SHRT_MAX (-32767), which is wrong. > It can only be as small as -NUMERIC_DSCALE_MAX (-16383), though that > doesn't affect the point being made in that comment. > > I'd like to treat this as a bug-fix and back-patch it, since the > current behaviour is clearly broken. Fix seems straightforward to me. I agree it should be back-patched. Regards, Joel
В списке pgsql-hackers по дате отправления: