Using Arrays - math precision
Jerry J
jhj at jhj.com
Sat Jun 4 19:21:23 EDT 2011
On Jun 4, 2011, at 4:01 PM, Mark Wieder wrote:
> Jerry-
>
> Saturday, June 4, 2011, 3:52:03 PM, you wrote:
>
>> LC uses IEEE double-precision binary floating point numbers for
>> calculations. These are 64 bits: 52 for mantissa, 11 for exponent
>> and 1 for sign. So exponent range won't be a problem, neither will 6
>> decimal places, as long as the number of significant digits is less
>> than 15. Be sure to round to that least digit, because calcs are in
>> binary.
>
> Thanks - good to know. But I thought the default numberFormat was to
> display only six digits?
I think thats true. Now as for why... ??
--Jerry
More information about the use-livecode
mailing list