Small valued numbers or scientific computing
dunbarx at aol.com
Sun Jun 23 22:35:55 EDT 2013
I may be old fashioned, in that I always thought that the Apple SANE limits, 19 digits, was still the law of the land. This is OS9 stuff, though and I could just be doddering. But it would mean that setting the number format to such an outrageous string would be futile, most of that string ignored by the OS. But I could be wrong.
Sent from my iPhone
On Jun 23, 2013, at 10:20 PM, John <john at onechip.com> wrote:
> It seems odd that I haven't stumbled into this before and I am sure I am doing something wrong, I just can't seem to find a better way around the issue of working with small numbers.
> Suppose I want to multiply two small numbers such as: 1.23456e-41 * 1e-20. Unless I am happy with a result of 0, I have to use a really odd numberFormat statement such as:
> set the numberformat to "0.##########...############" where " …" represents sixty or so '#'
> This seems really odd. Surely there is a better way to do this. Can someone help?
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
More information about the Use-livecode