Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2

At 05:50 8/08/00 -0700, Don Baccus wrote:
>
>The simplest thing is to realize that using float4 leaves you with
>just over 7 significant digits, and to only print out 7 digits.
>Then you'll get the answer you expect (10.100000).
>

You may have missed the point; my suggestions are only aimed at changing
the results of float4/float8 conversions & comparisons.

My (very vague) recollections of this stuff is that the machine
representation is only guaranteed to be within a certain machine/language
accuracy, so the stored value is within +/-(machine error) of the 'real
value'. Further, my recollection is that one or more bits are usually used
to provide rounding information so that, eg., the 7 digit representations
are consistent.

Given this, I have assumed that printf etc use these least significant bits
to determine the 'correct' representation. The idea is to do exactly the
same in converting float4 to float8, so that:

   '4.1'::float4 = '4.1'::float8

will be true.

Maybe my recollection is false...




----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/