math, bigly

Mark Wieder ahsoftware at sonic.net
Wed Aug 9 12:14:20 EDT 2017


On 08/09/2017 02:36 AM, hh via use-livecode wrote:
>> Mark wrote:
>> but I'm annoyed that the IDE doesn't give me an overflow warning if I try
>> to perform an operation on a number that's bigger than 0xFFFFFFFF.
>> Instead, it just happily trims it down to an unsigned integer and proceeds
>> to give me a wrong result.
> 
> Hey Mark.
> Don't complain because *you* are easily able to work around your
> "not-reading-the-dictionary" ;-))
> 
>>From the baseconvert entry of the dictionary:
> baseConvert(number, originalBase, destinationBase)
> The number must be an integer between zero and 4,294,967,295 (2^32 - 1).

Oops. OK - point sheepishly taken. Is that the definition of integer in 
the dictionary? The "integer" entry just says "if the value contains 
digits (and an optional minus sign) and no other characters, it is an 
integer", which is as simplisitic a definition as the entry for "hex".

But what annoys me isn't the fact that baseconvert, etc (bitXor is the 
one that bit me) only work on integers less than 2^32. It's more the 
fact that the input data was crunched down to something the functions 
could handle, and then give me the wrong result with no warning.

-- 
  Mark Wieder
  ahsoftware at gmail.com




More information about the use-livecode mailing list