numberformat question

[Bob Sneidar]

The ingenuity of people on this list never ceases to amaze me. 

Bob S


> On Mar 22, 2021, at 6:41 AM, Craig Newman via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Check out this thread in the forum:
> 
> https://forums.livecode.com/viewtopic.php?f=7&t=25132&p=130985&hilit=longadd#p130985 <https://forums.livecode.com/viewtopic.php?f=7&t=25132&p=130985&hilit=longadd#p130985>
> 
> Craig
> 
>> On Mar 21, 2021, at 7:34 AM, jbv via use-livecode <use-livecode at lists.runrev.com> wrote:
>> 
>> Paul, Sean,
>> 
>> Thank you both for your answers, although it doesn't really solve my problem.
>> The most surprising thing is that javascript runs these calculations without
>> problem :
>> (-0.0000007347880794884118999 / 1000000000) returns -7.347880794884119e-16
>> (-0.0000007347880794884118999 / 1000000000000) returns -7.347880794884119e-19
>> 
>> Best
>> 
>> 
>> Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :
>>> Livecode can handle floating point numbers up to 2^52. But then
>>> calculations would have to fit within that range in a floating point
>>> perspective too. So, once you add that ninth zero to the end of your
>>> divider, it struggles with the calculation and returns, effectively, number
>>> out of range.
>>> Fractions in FP binary are tremendously complicated. Calculation in it are
>>> even more so.
>>> https://www.h-schmidt.net/FloatConverter/IEEE754.html
>>> On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>> I don't have an answer, other than to say there has been a LOT of
>>>> discussion over the years about the limits of number precision in
>>>> LiveCode and a number of enhancement requests and 3rd party libraries
>>>> that have attempted to provide a framework for high precision math. You
>>>> might search the list archives or search the forum on the LC web site
>>>> for past discussions about number precision.
>>>> On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
>>>>> Hi list,
>>>>> I am a bit lost :
>>>>> 
>>>>>   set numberformat to ".#########################"
>>>>>   put -0.0000007347880794884118999 / 1000
>>>>> -- returns -0.0000000007347880794884119  which is ok
>>>>> 
>>>>>   put -0.0000007347880794884118999 / 1000000
>>>>> -- returns -0.0000000000007347880794884 -> also ok
>>>>> 
>>>>>   put -0.0000007347880794884118999 / 100000000
>>>>> -- returns -0.0000000000000073478807949 -> still ok
>>>>> 
>>>>>   put -0.0000007347880794884118999 / 1000000000
>>>>> -- returns 0 -> ?????
>>>>> 
>>>>> Any explanations ?
>>>>> Thanks in advance.
>>>>> Best,
>>>>> jbv
>>>>> 
>>>>> _______________________________________________
>>>>> use-livecode mailing list
>>>>> use-livecode at lists.runrev.com
>>>>> Please visit this url to subscribe, unsubscribe and manage your
>>>>> subscription preferences:
>>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> use-livecode at lists.runrev.com
>>>> Please visit this url to subscribe, unsubscribe and manage your
>>>> subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode