Math issue, isn't it?

[Mick Collins]

It seems to me that significant digits might be a better way to go  
than mathPrecision.  Problem is in comparing something like
3^-6 - 3^-7  = 0   or   3^-26 - 3^-27  = 0.  If that is what you  
intend you would likely have a problem with the latter being true,  
when for your purposes it isn't, if you are using a precision based  
on number of digits, rather than number of significant digits.  Or am  
I misunderstanding something here?

Any comments on my function from a couple days ago?
****************************
Until the matter is finished being debated and a plan of action is
decided on, a function could be used such as

function numsAreEqual n1, n2, epsilon
     return abs(n1 - n2) < epsilon
end numsAreEqual

If your testing of the immediate comparison indicates to you that
there are no roundoff problems, then use =, otherwise use this
function or your own variant with an epsilon that seems prudent.
******************************




> Date: Mon, 11 May 2009 09:45:48 -0700
> From: Brian Yennie <briany at qldlearning.com>
> Subject: Re: Math issue, isn't it?
> To: How to use Revolution <use-revolution at lists.runrev.com>
> Message-ID: <C4D20BF6-2DC8-47DE-A32B-038F023516D7 at qldlearning.com>
> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>
> Agree on the mathPrecision value. While it is true that floating point
> values are an issue in any programming language, Rev should NOT be
> failing on simple comparisons with two decimal places. Computers are
> limited in floating point math, but they are perfectly capable of
> handling 2 decimals places!!
>
> An alternative would be to delve into significant digits, but I'm
> afraid that would cause more confusion than anything...
>
>>
>> On May 11, 2009, at 12:11 PM, Bob Sneidar wrote:
>>
>>> I think we are mincing words here. It's a simple matter of changing
>>> the way the math function works.
>>
>> I understand that it's an oddity that the computer doesn't quite see
>> how simple the numbers are, but that happens for humans too. Take a
>> simple case of expressing 2/3 as a floating point decimal number,
>> you can't actually type out the exact value, even with unlimited
>> monkeys to do the typing for you. That's a case where decimal has
>> its limitations. Computers have different limitation cases, because
>> they're working in binary.
>>
>> That said, having a mathPrecision equivalent to numberFormat would
>> solve the issue too, any tiny amounts below the mathPrecision value
>> would be rounded up or down.
>
>
>