Re: BUG #19340: Wrong result from CORR() function

Поиск
Список
Период
Сортировка
От Tom Lane
Тема Re: BUG #19340: Wrong result from CORR() function
Дата
Msg-id 931689.1764807066@sss.pgh.pa.us
обсуждение исходный текст
Ответ на Re: BUG #19340: Wrong result from CORR() function  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-bugs
I wrote:
> Poking at this, I soon found a test case where even with the separate
> sqrt() calls we'd produce a result slightly outside [-1, 1] (running
> this test over more values of x is sufficient).  So now I think we
> should do both the separate sqrt and the clamp.

Per CI results, on some platforms the roundoff error is different from
what I observe, producing a value just less than 1 rather than just
more.  That doesn't invalidate needing the clamp, but it does mean
that we can't use that test case just like that.  I'm inclined to
remove the change of extra_float_digits, but keep the test case.

            regards, tom lane



В списке pgsql-bugs по дате отправления: