Strange math behaviour... could someone explain this ?

[Mark Waddingham]

Hi JB,

The solution to the issue you have here is to always add a small
'epsilon' value to your numbers before truncating.

What is manifesting itself here is the innate approximation methods used
by floating-point arithmetic, combined with the fact the trunc function
does not shield you from these (indeed, trunc is there so you can use it
as a basis for your own rounding methods and shouldn't really be used
raw).

Any floating-point number should not be considered a single number at
all, but a *range* of numbers (alternatively, you can view it as a
single number +/- an error). So:
  34.2 is actually: 34.2 - <epsilon> to 34.2 + <epsilon>
  36 is actually: 36 - <epsilon> to 36 + <epsilon>

Therefore, when you get to (36 - 34.2) * 100, what you actually have is
the range:
  180 - epsilon to 180 + epsilon

The actual representative stored in the computer can be any number
within this range, and it just turns out in the case in question, the
representative is:
  179.99999999999971578291

All the trunc function does is chop off the fractional part, leaving
179.

Therefore, in order to get the 'expected' answer, you need to ensure
that you correct for maximum deviation *below* an integer that can occur
before truncating. i.e. You need to add an epsilon value.

As a general rule of thumb, if you require n decimal places of
precision, you should use an epsilon value of 1e-(n+1).

So, let's say that your calculations need no more precision than 7
decimal places, then
  epsilon = 0.00000001

And your code becomes:
  trunc((36 - 34.2) * 100 + 1e-8)
Which does, indeed, return 180.

I should point out that this is probably the only instance where you
would get bitten by this precisely because the trunc function does
nothing to correct for error in the floating-point approximation as it
does not know the precision with which the number you passed to it was
calculated.

Warmest Regards,

Mark.

On Fri, 2005-10-07 at 13:16 +0200, jbv wrote:
> Hi list,
> 
> example 1 :
> 
>   put 1.8 into myA
>   put trunc(myA * 100) into myA
> 
>   in that case, myA=180    which is OK
> ------------------
> 
> example 2 :
> 
>   put 34.2 into myA
>   put 36 into myT
> 
>   if myA > 0 then
>     if myA < myT then
>       get myA
>       put myT - myA into myA
>       put it into myT
>     else
>       put 0 into myA
>     end if
>   end if
> 
>   put trunc(myA * 100) into myA
> 
>   in that case, myA=179    ?????????????????????
> 
> 
> Best,
> JB
> 
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution

------------------------------------------------------------------
 Mark Waddingham ~ 36degrees at runrev.com ~ http://www.runrev.com
       Runtime Revolution ~ User-Centric Development Tools