Small valued numbers or scientific computing

John john at onechip.com
Sun Jun 23 22:46:06 EDT 2013


Craig,

   With an insanely large number of "#" in number format I can get this to work: 1234567890123e-10 * 1e-170. I get the result: 1.2345678901230e-168.

   It appears that there is a bit more going on as they are floating point values.

Thanks,
John

On Jun 23, 2013, at 7:35 PM, Craig Newman <dunbarx at aol.com> wrote:

> I may be old fashioned, in that I always thought that the Apple SANE limits, 19 digits, was still the law of the land. This is OS9 stuff, though and I could just be doddering. But it would mean that setting the number format to such an outrageous string would be futile, most of that string ignored by the OS. But I could be wrong.
> 
> Craig Newman
> 
> Sent from my iPhone
> 
> On Jun 23, 2013, at 10:20 PM, John <john at onechip.com> wrote:
> 
>>  It seems odd that I haven't stumbled into this before and I am sure I am doing something wrong, I just can't seem to find a better way around the issue of working with small numbers.
>> 
>>  Suppose I want to multiply two small numbers such as: 1.23456e-41 * 1e-20.  Unless I am happy with a result of 0, I have to use a really odd numberFormat statement such as:
>> 
>> set the numberformat to "0.##########...############" where " …" represents sixty or so '#'
>> 
>> This seems really odd.  Surely there is a better way to do this.  Can someone help?
>> 
>> Thanks,
>> John
>> 
>> 
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode





More information about the Use-livecode mailing list