Math issue, isn't it?

Colin Holgate coiin at rcn.com
Mon May 11 16:09:07 EDT 2009


On May 11, 2009, at 3:40 PM, Jan Schenkel wrote:

> Scott Raney - the original developer of Metacard, the underlying  
> engine for Revolution - opted for the better speed of CPU-native  
> numbers, instead of the byte arithmetic algorithm as implemented in  
> HyperCard

While that is interesting, HyperCard also has the same math issue. So  
does Javascript (I know that's not like Java, but you might think that  
its math was on the same lines):

on mouseUp
set the numberformat to "#.0000000000000000"
answer 283.67-150.00-133.67
end mouseUp

shows an answer of .0000000000000284. This:

<html><script type="text/javascript">alert(283.67-150.00-133.67)</ 
script></html>

alerts an answer of 2.842170943040401e-14. So it seems like everyone  
gets it wrong!

For completeness, I tried Java too:

println((283.67-150.00)-133.67);

shows as 1.5258789E-5, so Java is off on its own level of error.





More information about the use-livecode mailing list