Re: Re: [Oledb-dev] double precision error with pg linux server, but not with windows pg server

Tom Lane wrote:
> Shachar Shemesh <shachar@shemesh.biz> writes:
>   
>> Before you explode at me (again :), I'm not arguing that you can do
>> binary based calculations of decimal numbers without having rounding
>> errors that come to bite you. I know you can't. What I'm saying is that
>> we have two cases to consider. In one of them the above is irrelevant,
>> and in the other I'm not so sure it's true.
>>     
>
> You're setting up a straw-man argument, though.
I was answering your claim that it's impossible to convert decimal to
binary based floats without rounding errors.
>   The real-world problem
> cases here are not decimal, they are non-IEEE binary floating
> arithmetic.  The typical difference from IEEE is slightly different
> tradeoffs in number of mantissa bits vs number of exponent bits within a
> 32- or 64-bit value.
I answered that elsewhere while suggesting a different format that would
address that. These numbers do not appear to be a concern in our
situation, however.
>   I seem to recall also that there are machines that
> treat the exponent as power-of-16 not power-of-2.
I'm pretty sure I don't understand this. Maybe I misunderstood the
format, but wouldn't that actually lose you precision with, at most,
marginal gain in range? As far as I can see, the moment you no longer
work in base 2 you lose the implicit bit, which means you have a one bit
less starting point than base 2 notations (all number are denormalized).
>   So depending on which
> way the tradeoffs went, the other format will have either more precision
> or more range than IEEE.
>   
Again, should that be a real concern, see my message at
http://archives.postgresql.org/pgsql-hackers/2007-05/msg00892.php for
details about what the suggestion actually is. Just be sure to read
"IEEE" there as meaning "IEEE like". I allowed different sizes for the
fields.
>             regards, tom lane
>   
Shachar