Math issue, isn't it?
Colin Holgate
coiin at rcn.com
Mon May 11 12:36:56 EDT 2009
On May 11, 2009, at 12:11 PM, Bob Sneidar wrote:
> I think we are mincing words here. It's a simple matter of changing
> the way the math function works.
I understand that it's an oddity that the computer doesn't quite see
how simple the numbers are, but that happens for humans too. Take a
simple case of expressing 2/3 as a floating point decimal number, you
can't actually type out the exact value, even with unlimited monkeys
to do the typing for you. That's a case where decimal has its
limitations. Computers have different limitation cases, because
they're working in binary.
That said, having a mathPrecision equivalent to numberFormat would
solve the issue too, any tiny amounts below the mathPrecision value
would be rounded up or down.
More information about the use-livecode
mailing list