math, bigly

[Bob Sneidar]

Hi Mark. 

I remember an old trick to determine if a math function was overflowing. Run the equasion backwards and see if that result equals the initial value. This will reveal rounding errors too. 

Bob S


> On Aug 8, 2017, at 18:38 , Mark Wieder via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Phooey...
> 
> put baseconvert("100000001",16,10)
> 
> ...and yes, I understand what's going on under the hood, but I'm annoyed that the IDE doesn't give me an overflow warning if I try to perform an operation on a number that's bigger than 0xFFFFFFFF. Instead, it just happily trims it down to an unsigned integer and proceeds to give me a wrong result. I came across this yesterday handing two large numbers to the bitXor operand which threw no errors and went on with the rest of the calculation. I had to backtrack to figure out what was going on, and finally ended up writing my own bitwise operators to deal with this.
> 
> -- 
> Mark Wieder
> ahsoftware at gmail.com