hexadecimal calculations

Dar Scott dsc at swcp.com
Thu Jan 27 16:12:50 CST 2005


On Jan 26, 2005, at 10:00 PM, Paul Salyers wrote:

> will the computer (REV) do math calculations with hexadecimal without 
> converting it to decimal?

Internally Revolution uses a standard floating point representation 
with a binary point.

When math is presented with a value that is the result of arithmetic, 
it uses it as is.  If it is a numeral (text meaning a number), it 
converts first.  It can handle a wide range of numerals.  The numerals 
are assumed to be decimal (base 10) unless prefixed with "0x" which 
means hex.  Either kind of numeral may use "E" notation, a computereze 
version of scientific notation.

Most people don't put quote marks around numerals, but it does not hurt 
to do that.

If the result of arithmetic is used as a string it is converted using 
the current local value of the numberFormat property.  It will convert 
the result of arithmetic to a decimal numeral, but since the number is 
stored internally with a binary point and not a decimal point, the 
fractional part may be off by a tiny bit.

The result of arithmetic (or any numberal) can be converted to a string 
with the format() function which allows hexidecimal formatting, but it 
does not work for the fractional part.

Also, the baseConvert() function can be used to convert numerals.  This 
uses a 32-bit whole number intermediate value, so conversions are 
limited by that.

For example:

on mouseUp
   put "0xA.8" into a
   put (a+2) && format("%x",a + 17)
end mouseUp

==>

12.5 1c


Dar

**********************************************
     DSC (Dar Scott Consulting & Dar's Lab)
     http://www.swcp.com/dsc/
     Programming Services and Software
**********************************************



More information about the use-livecode mailing list