Strange math behaviour... could someone explain this ?
Cubist at aol.com
Cubist at aol.com
Sun Oct 9 11:42:14 EDT 2005
sez kray at sonsothunder.com:
>I'm still trying to figure out how something as simple as ((36-34.2)*100)
>can be anything but 180, or how (as Alex put it) "the binary double
>precision representation of 34.2 is inexact"...
Perhaps a few details will help clear up this question.
.1 binary = .5 decimal
.01 binary = .25 decimal
.001 binary = .125 decimal
.0001 binary = .0625 decimal
.00001 binary = .03125 decimal
.000001 binary = .015625 decimal
etc
Try it out and you'll discover that while .2 is a nice, inoffensive
decimal number, it has a binary representation that turns out to be an infinitely
repeating sequence, not unlike .111.... for 1/9 or .142857... for 1/7.
Specifically, .2 decimal = .0011... binary.
>and if it *is*, isn't there
>any other form that could represent 34.2 exactly that's not binary double
>precision?
Yes. The cryptic abbreviation "BCD" stands for Binary Coded Decimal. It's
what IBM based its character set EBCDIC on; if you check out the EBCDIC entry
in the Jargon File, you may see one of the reasons why BCD has never caught on
amongst hackers.
More information about the use-livecode
mailing list