Small valued numbers or scientific computing

John john at onechip.com
Sun Jun 23 22:42:22 EDT 2013


Mike,

   Thanks for the tip.  I am using "format" for display purposes so that I get correctly formatted numbers but, unless I use "numberFormat" first, there is not enough precision available from the result of the multiplication and the result is 0.

   It appears that "numberFormat" sets the number of decimal places LC uses to do the math and "Format" formats the results for display.

   Any other thoughts?

Thanks,
John




On Jun 23, 2013, at 7:31 PM, Mike Bonner <bonnmike at gmail.com> wrote:

> look at the format function, specifically the "%#.##E" (or e) incantation.
> 
> 
> On Sun, Jun 23, 2013 at 8:20 PM, John <john at onechip.com> wrote:
> 
>>   It seems odd that I haven't stumbled into this before and I am sure I
>> am doing something wrong, I just can't seem to find a better way around the
>> issue of working with small numbers.
>> 
>>   Suppose I want to multiply two small numbers such as: 1.23456e-41 *
>> 1e-20.  Unless I am happy with a result of 0, I have to use a really odd
>> numberFormat statement such as:
>> 
>>  set the numberformat to "0.##########...############" where " …"
>> represents sixty or so '#'
>> 
>> This seems really odd.  Surely there is a better way to do this.  Can
>> someone help?
>> 
>> Thanks,
>> John
>> 
>> 
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode





More information about the use-livecode mailing list