# Strange math behaviour... could someone explain this ?

Lynch, Jonathan bnz2 at cdc.gov
Fri Oct 7 09:37:13 CDT 2005

```But still...

Answer trunc((36-34.2)*100) should return 180, not 179. I mean, the
underlying code should work to return an accurate value. Perhaps it is
just a matter of opinion, but to me, if the software returns a wrong
value in a calculation, it is a bug.

I use trunc in a calendar that calculates things like the number of
weeks in a month, or which days belong in which week of a given month.
An error of this sort could put a day into the wrong week.

-----Original Message-----
From: use-revolution-bounces at lists.runrev.com
[mailto:use-revolution-bounces at lists.runrev.com] On Behalf Of Alex
Tweedly
Sent: Friday, October 07, 2005 10:20 AM
To: How to use Revolution
Subject: Re: Strange math behaviour... could someone explain this ?

Lynch, Jonathan wrote:

>Even just this simple line produces the same error:
><snip>
>Somehow, Rev is performing 36-34.2, and even though it displays that
>number as 1.8, it must be processing it internally as
>1.799999999999999999999999999 or something like that.
>
>Very disturbing - This could affect a program of mine.
>
>It is easily worked around, with a function like this:
>
>function trueTrunc pNumber
>  set the itemdelimiter to "."
>  return item 1 of pNumber
>end trueTrunc
>
>
>
That won't necessarily work. trueTrunc(179.9999) ==> 179
You need to be very careful how you test things like this .... given
half a chance, Transcript will use a string representation rather than
an arithmetic one.

set the itemDel to "."
put <something> into myT
put item 1 of myT && trunc(myT)

Gives the following results
"something"     output
179.99999999    179 179  (it used the string repr)
179.99999999+0  180 179
(the addition forced arithmetic representation - and then got
rounding; trunc didn't get any rounding).

>But still, trunc should work properly. That makes me wonder if any
other
>math functions might have some underlying weirdness.
>
>
(see the little snippet above :-)

As I understand it (though I can't now find it in the docs), when doing
arithmetic Transcript represents numbers in the most suitable format -
either integer or double-prec floating point. In the example here,
because it has "34.2" the suitable precision is double. Since many
"simple" decimal floating values can't be exactly represented in binary
floating point format, there is always a chance of minor discrepancies
between the "obvious" value and the precise one used by Transcript.

Transcript does a good job of "magically" rounding as needed - but the
issue is still there.

trunc(x) is always dangerous because it will truncate down to the lower
integral part (e.g. 179.9999999  --> 179) if there is even the tiniest
discrepancy. Much safer would be to trunc(x+delta), where delta is the
level of accuracy required - say typically delta = 0.000005

trueTrunc() will be slightly "safer" because it uses Transcripts "magic"

conversion, and hence rounds off at some nominal level - probably 6
decimal places or something like that -- but if it were me I'd rather
take control and determine for myself what number of digits to do the
rounding at.

I do not believe this is a bug - merely a trap for the unwary.

--
Alex Tweedly       http://www.tweedly.net

--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.344 / Virus Database: 267.11.13/123 - Release Date:
06/10/2005

_______________________________________________
use-revolution mailing list
use-revolution at lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution

```