Trunc Error

Schwartz, Jonathan L. jschwart at mitre.org
Tue Mar 2 11:33:23 EST 2010


I just found the following result.

(1.2 - 0.8)/0.2              = 2
trunc( (1.2 - 0.8)/0.2)  = 1

--
I have previously reported the following:

on mouseUp
   repeat with n = 0 to 1 step 0.1
      put n&"," after aList
   end repeat
   put aList
end mouseUp

returns:  0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1,1.1,
It should stop at 1

Also 0 to 0.8 step 0.1 stops at 0.9   ... there are others

The use of the increment is consistent with the documentation.


Are there other similar problems that users are aware of ?

When applied properly, these can produce false results and make analysis using runRev questionable.


I have reported these in the Rev Quality Control Center.






More information about the use-livecode mailing list