Small valued numbers or scientific computing

[Dar Scott]

Arithmetic takes (for each parameter) either a string that is a numeral or it takes the result of arithmetic (or sometimes binaryDecode).

The result of arithmetic is a "double precision" floating point number, an internal representation.  That essentially means you get about 15 digits of resolution in scientific notation with an exponent of about + or - 300.  The internal form is mostly hidden.

Thus, you can use the result of arithmetic in other arithmetic.  You can get double in and double out.  All of that (about) 15 digits of resolution is there.

When you do something that needs a string, the value is converted to a string based on numberformat.  The process loses information.  

If you do more arithmetic on the result after that, the string--that is, the numeral, possibly zero if all significant digits were lost--is converted to floating point and the arithmetic uses that.  

That can be a handy way to round a number, though round() is more explicit.  

So with a default numberFormat ...

get ( (1e-20 * 2e-30) & empty) * 1

...results in exactly zero.

The & forces a loss of information, it essentially rounds.

There was a time that 'send' would force the use of numberFormat.  Perhaps there are other surprising times still lurking.

Dar


On Jun 23, 2013, at 8:20 PM, John wrote:

>   It seems odd that I haven't stumbled into this before and I am sure I am doing something wrong, I just can't seem to find a better way around the issue of working with small numbers.
> 
>   Suppose I want to multiply two small numbers such as: 1.23456e-41 * 1e-20.  Unless I am happy with a result of 0, I have to use a really odd numberFormat statement such as:
> 
>  set the numberformat to "0.##########...############" where " …" represents sixty or so '#'
> 
> This seems really odd.  Surely there is a better way to do this.  Can someone help?
> 
> Thanks,
> John
> 
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode