Strange math behaviour... could someone explain this ?

[Mark Waddingham]

Hi JB,

>   get ((36-34.2)*100)
>   put it & cr & trunc(it)

Revolution has a 'numberFormat' property that determines to how many
decimal places are used when numbers are converted to a string.

The difference you are observing here is that the first line is the
string representation of 'it' to (IIRC) 6 decimal places and the second
line is the string representation of trunc(it) to 6 decimal places.

As 'it' contains the number:  
  179.99999999999971578291
When rounded to 6 decimal places you get 180

However, trunc(it) contains 179 and so is just 179 when rounded to 6
decimal places.

If you do:
  set the numberFormat to ".###############"
  get ((36-34.2)*100)
  put it & return & trunc(it)

You get:
  179.999999999999716
  179

Which is more inline with what you see in Javascript.

Warmest Regards,

Mark.

------------------------------------------------------------------
 Mark Waddingham ~ 36degrees at runrev.com ~ http://www.runrev.com
       Runtime Revolution ~ User-Centric Development Tools