Math issue, isn't it?

[Bob Sneidar]

I was thinking of a new property called MathPrecision or something.  
Set the mathprecision to -2 would round to 2 decimal places for the  
result of any math equasion. Set the mathprecision to 2 would round to  
the nearest 100. The default could be -9 in which case the prior error  
in floating point math that started this thread would not have occurred.

Bob Sneidar
IT Manager
Logos Management
Calvary Chapel CM

On May 11, 2009, at 9:45 AM, Brian Yennie wrote:

> Agree on the mathPrecision value. While it is true that floating point
> values are an issue in any programming language, Rev should NOT be
> failing on simple comparisons with two decimal places. Computers are
> limited in floating point math, but they are perfectly capable of
> handling 2 decimals places!!
>
> An alternative would be to delve into significant digits, but I'm
> afraid that would cause more confusion than anything...
>
>>
>> On May 11, 2009, at 12:11 PM, Bob Sneidar wrote:
>>
>>> I think we are mincing words here. It's a simple matter of changing
>>> the way the math function works.
>>
>> I understand that it's an oddity that the computer doesn't quite see
>> how simple the numbers are, but that happens for humans too. Take a
>> simple case of expressing 2/3 as a floating point decimal number,
>> you can't actually type out the exact value, even with unlimited
>> monkeys to do the typing for you. That's a case where decimal has
>> its limitations. Computers have different limitation cases, because
>> they're working in binary.
>>
>> That said, having a mathPrecision equivalent to numberFormat would
>> solve the issue too, any tiny amounts below the mathPrecision value
>> would be rounded up or down.
>
>
>
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your  
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution