Re: BUG #17334: Assert failed inside computeDistance() on gist index scanning

Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
> FWIW I can reproduce this on master too. The failure happens because of
> NaN value in the index:

Man, what a pain those are.

> I'm not sure if the issue is in allowing the NaN to be added to the
> index, or not handling it correctly during the index scan.

Surely omitting the entry from the index would lead to incorrect
answers.  Without any index, we get

regression=# CREATE TABLE point_tbl(f1 point);
CREATE TABLE
regression=# INSERT INTO point_tbl SELECT ('0,0') FROM generate_series(1, 2);
INSERT 0 2
regression=# INSERT INTO point_tbl VALUES ('0,NaN');
INSERT 0 1
regression=# SELECT f1, f1 <-> point '(0,0)' AS dist FROM point_tbl ORDER BY dist;
   f1    | dist
---------+------
 (0,0)   |    0
 (0,0)   |    0
 (0,NaN) |  NaN
(3 rows)

You can argue about where the NaN distance should sort, but
not about whether the row should appear at all.

> It's interesting btree_gist does not have issues (for NaN in float8
> columns). It seems not to store NaN in the index,  It seems to replace
> them with tiny values, at least according to pageinspect.

Yipes, that's even worse, if true.

            regards, tom lane