Tentative Guidelines for Numbers

[Dar Scott]

On Monday, April 1, 2002, at 02:22 PM, Dar Scott wrote:

> 3.
> Equality for numbers is tolerant of small differences.
>
> Use = for numbers in 2 above only after a couple calculations.  
> Accumulated errors may go outside of what = will consider equal.
>
> Use < or < before <= or >=.  Use any before =.  (Actually, I'm 
> guessing about <= or >=.  Some tests need to be run to see if they 
> have the same tolerance as =.)
>

Michael Mays described this as a characteristic of PPC numerics.  
On my OS X on my Blue & White G3 I found = to be tolerant up to 
plus or minus 9 times the weight of the LSB.  Clearly, somebody had 
decimal in mind.  The lack of transitivity could be a problem in 
some cases, but the use of = for numbers described in #2 (see 
original email, 4/1) should be avoided in the case of a chain of 
calculations, anyway.

I also noticed that =, > and < seem to be distinct.  A number has 
to be greater by 9 counts to be considered greater.

So, I amend my suggestion for #3:
**************
3.
Equality for numbers may tolerant of very small differences on some 
computers.  Nevertheless, even in this case, if a number is > or < 
a number, it is not = to the number.

Equality might work after a couple calculations on some computers 
and fail on others.  For general scripting, continue to avoid =, if 
possible.  This does not apply to whole numbers in #1; this applies 
to only large numbers or numbers with general fractional parts as 
described in #2.
***************

Optionally, #3 could be dropped and the "15 digits" in #1 changed 
to "14 digits".  This can simplify the guideline.

I would also add this paragraph to #2:
***************
Functions that have a distinct step in value, such as round(), 
trunc() and mod should be used with care when used with these 
numbers.  Normally, one does not care about the exact point of 
rounding with round(), but the step point can be crucial with 
trunc() or mod.
***************

Dar Scott