[LCP]Rounding questions

Chuck Martin nrocinu at myrealbox.com
Mon Sep 9 19:06:26 UTC 2002


On Fri, Sep 06, 2002 at 06:02:30PM -0400, Mike & Penny wrote:
> a general problem in computing. The first thing you have to understand is
> that your programs shouldn't depend on EXACT tests but instead on some
> "fuzz" and the order in which calculations are performed should be assumed
> to matter. In other words, two calculations with different order of
> operations that should give the same result according to the rules of
> Mathematics may not do so when performed by the computer (finite
> approximations).

Yes, but how can you use "fuzz" when determining whether to round up or
down, when that determination is based on whether a result is greater
than or equal to 0.5?  It seems likely that it has something to do with
the differing precision between a 64 bit double in memory and an 80 bit
result on the FPU stack, where the program has no way of determining
which of those the value in question is at the moment.

Chuck





More information about the linuxCprogramming mailing list