BUG #6399: knngist sometimes returns tuples in incorrect order

The following bug has been logged on the website:

Bug reference:      6399
Logged by:          YAMAMOTO Takashi
Email address:      yamt@mwd.biglobe.ne.jp
PostgreSQL version: Unsupported/Unknown
Operating system:   NetBSD
Description:=20=20=20=20=20=20=20=20

9.2devel
(01d83ffdcae92f75dbfd41de0b4213d241edd394)

knngist seems to assume that any distances can be represented with float8.
at least distances between int8 values can not.

the following example uses btree_gist extension.
results should be the same regardless of the existance of the index.

create temp table t (a int8);
insert into t values (9223372036854775806),(9223372036854775807);
select *,0<->a as dist from t order by dist;
create index on t using gist (a);
set enable_seqscan=3Doff;
select *,0<->a as dist from t order by dist;


CREATE TABLE
INSERT 0 2
          a          |        dist=20=20=20=20=20=20=20=20=20
---------------------+---------------------
 9223372036854775806 | 9223372036854775806
 9223372036854775807 | 9223372036854775807
(2 rows)

CREATE INDEX
SET
          a          |        dist=20=20=20=20=20=20=20=20=20
---------------------+---------------------
 9223372036854775807 | 9223372036854775807
 9223372036854775806 | 9223372036854775806
(2 rows)