Math problems?

[Dar Scott]

On May 19, 2006, at 1:04 PM, Peter T. Evensen wrote:

> put 10.27 into tNumber
> put the trunc of tNumber into tInteger
> put tNumber - tInteger into tDecimal
> put 100 * tDecimal into tNew Decimal
> put the trunc of tNewDecimal
>
> and I get 26 instead of 27.
>
> Am I doing something wrong?  Is this a bug?

This is not a bug.

The result of arithmetic is stored internally as a floating point  
number with a binary point.

tDecimal is really this (about):

0.26999999999999957367435854393988847732543945312...

Before tNumber can be have the trunc applied, it must be converted to  
the same form.  The decimal numeral you entered cannot be represented  
exactly.  Here is a close approximation:

10.26999999999999957367435854393988847732543945312...

Some operations such as =, fudge a little to accommodate this.   
Apparently trunc does not.

Though this is not a bug, it is a surprise for many, and I believe a  
shortcoming of Revolution.  However, others might feel that the  
increased execution time of (say) decimal point arithmetic is not  
worth it.

> Is there another way to isolate the decimal portion?

If you just want to create a numeral with 2 digits to the right, then  
set the numberFormat to "#.00".  If you actually want the round of  
the two digits, use round() instead of trunc().  If you want those  
two digits you can pick off the last two characters after formatting.

Dar Scott