64 bit LC

[Kay C Lan]

Surprised that no one has replied to this so I'll just offer a bit of
advice.  Whilst on the surface your assumption is correct, 64bit will
allow more accuracy than 32 bit, what you need to be aware of is the
same gotchas still apply to 64 bit LC as 32 bit LC.  Try this in the
msg box:

put 283.67-150.00-133.67

you should get 0. Now try this in the msg box:

put 283.67-150.00-133.67=0

You will get 'false' when you know you wanted true;  whether in 32bit
or 64 bit LC (Python, JBScript and some other languages) in some cases
you don't get the answer you know is correct.

The answer can be found in the Dictionary entry for NumberFormat:

Note: Since LiveCode does not use decimal numbers for its internal
calculations (for reasons of speed), the decimal representation of a
number is sometimes slightly off the correct number. For example,
10^-1 is equal to 0.1, but is calculated (to eighteen decimal places)
as 0.100000000000000006. Because of this, setting the numberFormat to
specify many decimal places after the decimal point may produce
unexpected results in a statement that tests for an exact number. To
prevent this, either avoid setting the numberFormat to a value more
precise than you need, or use the abs function instead of the =
operator to test equality:

put abs(283.67-150.00-133.67)=0

but we still don't get the answer we are expecting.

NumberFormat is not the only LC property/command/function that is
effected by the use of 'decimal representation'; trunc is another that
comes to mind.  But as can be seen by my example ALL math within LC is
effected by the way LC treats decimal values.

If you do a search of this mailing list for 'decimal floating point'
you will find plenty of examples of people who have been caught out by
this 'computing gotcha' - again it is not just LC that suffers from
this.  For any and all 'precision' math within LC, be it 32 or 64 bit,
special attention will need to be made to account for these fringe
cases.

Good luck.

On Mon, Jul 16, 2018 at 4:44 AM, Tom Glod via use-livecode
<use-livecode at lists.runrev.com> wrote:
> Hi folks, am I correct to assume that a 64 bit build of livecode will
> enable the engine to do more precise math?  I want to build a tool that
> requires precise division  with really small decimals and multiplication of
> large numbers I want to know the number of digits is can reliably count on
> on both sides of 0, and get 100% accurate result every time.
>
> Thank you for any clarification on this subject. I defer to your greater
> wisdom.
>
> I can't test LC linux on right now, and I don't have a mac ..or i would
> test this myself. :)
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode