Strange math behaviour... could someone explain this ?

Cubist at aol.com Cubist at aol.com
Sun Oct 9 11:42:14 EDT 2005


sez kray at sonsothunder.com:
>I'm still trying to figure out how something as simple as ((36-34.2)*100)
>can be anything but 180, or how (as Alex put it) "the binary double
>precision representation of 34.2 is inexact"...
   Perhaps a few details will help clear up this question.

   .1 binary = .5 decimal
   .01 binary = .25 decimal
   .001 binary = .125 decimal
   .0001 binary = .0625 decimal
   .00001 binary = .03125 decimal
   .000001 binary = .015625 decimal
   etc

   Try it out and you'll discover that while .2 is a nice, inoffensive 
decimal number, it has a binary representation that turns out to be an infinitely 
repeating sequence, not unlike .111.... for 1/9 or .142857... for 1/7. 
Specifically, .2 decimal = .0011... binary.

>and if it *is*, isn't there
>any other form that could represent 34.2 exactly that's not binary double
>precision?
   Yes. The cryptic abbreviation "BCD" stands for Binary Coded Decimal. It's 
what IBM based its character set EBCDIC on; if you check out the EBCDIC entry 
in the Jargon File, you may see one of the reasons why BCD has never caught on 
amongst hackers.



More information about the use-livecode mailing list