MOD function does not work! So be warned!

Michael D Mays michael at GreppyDreppies.com
Sun Mar 31 14:36:01 EST 2002


Richard Gaskin of ambassador at FourthWorld.com wrote the following on 3/31/02
10:00 AM

> Interestingly, when you type  "put 5 mod 5" (or "255 md 5" or other number
> divisible by five that fails when inside a loop) into the Message Box the
> result is 0.
> 
> Stranger still, if you rewrite the line like this so that the loop var is
> evaluated outside of the rest of the statement:
> 
> do "put loop &tab&"& loop &&"mod 5 & return after msg"
> 
> ...you get 0.

The reason it works in the msg box is because you are using a different
number. Inside the loop the number is
254.9999999999988631316227838397 not the 255 you typed.

If you set the numberFormat large enough you will see that loop is never
255. And also your do statement in the loop will fail.

The situation is that when you convert .1 to binary and add it to 255 it is
actually adding a binary number whose value is a little less than the value
of decimal .10. .1 has no exact binary representation. if the increment were
some inverse power of two everything would 'work' the way we expect it to.

When you were using the do statement you are forcing loop to be converted to
a string at the precision of the default numberFormat (since you didn't set
it to anything else). This string is 255. When you increase the precision of
the numberFormat to max precision the string is
254.9999999999988631316227838397.

michael




More information about the use-livecode mailing list