Troubles with inequality
livfoss at mac.com
Wed Feb 4 10:02:42 EST 2015
Using LC 7.0.2-rc-2, I have two numbers, each in its own variable ('temp' and 'aa' below) that may differ at say the eleventh decimal place but are otherwise identical. This bit of code:
set the numberformat to "0.######" -- to get 6 dec place precision just for the comparison
put temp*1 into temp -- force a calculation to make a character string of the correct length
put aa*1 into aa -- force a calculation to make a character string of the correct length
if temp <> aa then
answer error "No! aa must be" && aa && "length=" & (number of chars of aa) & ", but it is actually" && temp && "length=" & (number of chars of temp)
always results in an error even though their length as strings is the same and the actual strings are by inspection identical, i.e. the numbers are being seen as unequal even though their representation as strings is undoubtedly equal.
If I code
if length (aa) <> length (temp) or (char 1 to length(aa) of temp) <> (char 1 to length(aa) of aa)
Then I get the expected result and the error message is not shown.
So it looks as if either I have made a massive mistake (always possible, to put it mildly) or the comparison is for the numeric values of the original quantities and not the string representations which I have tried to achieve. This goes against common sense - where are these more precise numbers being stored?
Sadly I can't (yet) reduce this to a simple demonstration to report it as a bug, but I shall continue to try.
Just wondered if anyone else had seen anything at all like this.
Still trying, but puzzled.
More information about the use-livecode