Rev math

Dar Scott dsc at
Sun Mar 16 10:18:01 EST 2003

On Saturday, March 15, 2003, at 10:54 PM, curry wrote:

> From trying it out with some operations, it looks like about 16 digits 
> (whole or decimal) can be trusted. Using the 16th digit as a rounder, 
> that would mean 15 digits usable answer.
> My question is, can we trust about that number of digits for all types 
> of calculations?

Yes.  Use 14.  (49 bits floating point)  Use less if you need exact 

In many cases one can think of all values except arrays as strings.  
However, when you press limits like this, that may not be sufficient.  
The result of arithmetic is stored internally as floating point.  It is 
still floating point when used in subsequent arithmetic, but a copy 
converted to a string is used in non-arithmetic operations.

A consequence is that you may also lose digits when a number is 
converted to string based on the numberFormat property.  This happens 
when an operation needs a string.  This includes concatenation (&) as 
well as a few operations that you might not realize are converting 
(such as send).

You might also see some interesting rounding in some arithmetic.  This 
is because the rounding is based on a binary point.  This is mitigated 
somewhat with a (standard) laxness in numerical equality.  However, 
that laxness may also cause surprises.

Note that in some operations (such as base conversion and in 
bit-logical operations, I think), you are limited to 31 or 32 bits.

> And are there any plans to change the way Revolution handles this?

I have no inside knowledge of this, so my comment is pure speculation.

I do not expect this will change any time soon.  One reason is the 
natural inertia of such things.  It works just fine for people who use 
Rev arithmetic, a group self-selected because the arithmetic does what 
they want.  Another is compatible semantics among platforms; this is 
implemented using hardware (where I have used it) and the change 
(advance) should work across the primary platforms.

There are three directions a change might occur.  The simplest is 
simply more precision.  I think this most likely should a change occur. 
  I wouldn't be surprised if representation changed from 8 byte fp to 16 
byte fp in the next five years.  The second is the use of a decimal 
point.  This is not likely to occur if it involves a decrease in 
precision or a multifold decrease in speed.  (It is commonly assumed 
that this requires a decimal internal representation such as BCD; this 
is not true.)  The third is a paradigm shift such as indefinite 
precision.  I doubt this will occur.

Since you can safely work with a 12 digit integer (say), you can use 
that as a building block to build other types of numbers.  I would do 
this rather than wait for a change.

Dar Scott

More information about the use-livecode mailing list