Using Arrays - math precision

Mark Wieder mwieder at ahsoftware.net
Sat Jun 4 19:01:27 EDT 2011


Jerry-

Saturday, June 4, 2011, 3:52:03 PM, you wrote:

> LC uses IEEE double-precision binary floating point numbers for
> calculations. These are 64 bits: 52 for mantissa, 11 for exponent
> and 1 for sign. So exponent range won't be a problem, neither will 6
> decimal places, as long as the number of significant digits is less
> than 15. Be sure to round to that least digit, because calcs are in
> binary.

Thanks - good to know. But I thought the default numberFormat was to
display only six digits?

-- 
-Mark Wieder
 mwieder at ahsoftware.net





More information about the use-livecode mailing list