Troubles with inequality

William Prothero prothero at earthednet.org
Wed Feb 4 15:28:18 EST 2015


Colin:
But: I don’t see this. Say your target accuracy is six figures. You would us:
put round(myNum1,6) into num1
put round(myNum2,6) into num2
So, for example, 3.1234569 would round to 3.123457
3.1234563 would round to 3.123456, which would be unequal, but 3.1234567 would round up to 3.123457 and show as equal.
If you use too many digits, you will run into the limits of the binary number accuracy. If you do round(1.9,1) and round(2.0), you should be comparing 1.9 to 2.0. which would show as unequal.

Bill

> On Feb 4, 2015, at 8:55 AM, Colin Holgate <coiin at verizon.net> wrote:
> 
> Round has a problem in that rounding could change the nature of something. For example, you want to see if 1.9995 is similar to 1.9994, but you end up checking 1.9 against 2.0. One nice addition would be to give Trunk the same second parameter that Round has. Then you would keep the nature of the string.
> 
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode





More information about the use-livecode mailing list