Strange math behaviour... could someone explain this ?

[Alex Tweedly]

Lynch, Jonathan wrote:

>But still...
>
>Answer trunc((36-34.2)*100) should return 180, not 179. 
>
No it shouldn't.

>I mean, the
>underlying code should work to return an accurate value. 
>
It does.

>Perhaps it is just a matter of opinion, but to me, if the software returns a wrong
>value in a calculation, it is a bug.
>
>  
>
It's not a matter of opinion, and it's not a wrong answer - it's a 
correct answer. The binary double precision representation of 34.2 is 
inexact, and hence the binary double precision representation of 36-34.2 
is similarly inexact - so instead of exactly 180, it's about 1x2**-40 
less than that. 

And then when you use trunc() it does what you ask.

It's NOT a Rev bug - it's an artifact of double precision binary 
arithmetic (or, if you like, an artifact of the IEEE format used by 
Intel (and everyone else)).

Here's the equivalent C program and its output (Intel P4, WinXP, 
Bloodshed C++ IDE):

> #include <stdio.h>
> #include <stdlib.h>
> #include <math.h>
>
> int main(int argc, char *argv[])
> {
>   double f1, f2;
>   double t1, t2;
>   int i;
>   f1 = 36.0;
>   f2 = 34.2;
>   printf("%6.4f %6.4f\n", f1, f2);
>   t1 = 100.0*(f1-f2);
>   t2 = trunc(t1);
>   printf("%6.4f gives %6.4f\n", t1, t2);
>  
>   system("PAUSE");    
>   return 0;
> }

36.0000 34.2000
180.0000 gives 179.0000

>I use trunc in a calendar that calculates things like the number of
>weeks in a month, or which days belong in which week of a given month.
>An error of this sort could put a day into the wrong week.
>
>  
>
Users of trunc() (whichever language they use it in :-) should be wary 
of the dangers.

-- 
Alex Tweedly       http://www.tweedly.net



-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.344 / Virus Database: 267.11.13/123 - Release Date: 06/10/2005