Small valued numbers or scientific computing
john at onechip.com
Sun Jun 23 22:46:06 EDT 2013
With an insanely large number of "#" in number format I can get this to work: 1234567890123e-10 * 1e-170. I get the result: 1.2345678901230e-168.
It appears that there is a bit more going on as they are floating point values.
On Jun 23, 2013, at 7:35 PM, Craig Newman <dunbarx at aol.com> wrote:
> I may be old fashioned, in that I always thought that the Apple SANE limits, 19 digits, was still the law of the land. This is OS9 stuff, though and I could just be doddering. But it would mean that setting the number format to such an outrageous string would be futile, most of that string ignored by the OS. But I could be wrong.
> Craig Newman
> Sent from my iPhone
> On Jun 23, 2013, at 10:20 PM, John <john at onechip.com> wrote:
>> It seems odd that I haven't stumbled into this before and I am sure I am doing something wrong, I just can't seem to find a better way around the issue of working with small numbers.
>> Suppose I want to multiply two small numbers such as: 1.23456e-41 * 1e-20. Unless I am happy with a result of 0, I have to use a really odd numberFormat statement such as:
>> set the numberformat to "0.##########...############" where " …" represents sixty or so '#'
>> This seems really odd. Surely there is a better way to do this. Can someone help?
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
More information about the Use-livecode