numberformat question

[jbv at souslelogo.com]

Hi Craig,

Thank you for the link, but it doesn't help much :

bigDivide(-0.0000007347880794884118999,1000000000)
returns 0 , -0.000001

what I'm actually trying to do is to transpose in LC
an algo for solving cubic equations that runs perfectly
in javascript.
So my solution will finally be to multiply data by 10^6,
run the algo, and then divide the results by 10^6 by
treating them as strings and moving the dot...
It's not really elegant and especially quite slow, as I
need to process large amounts of data and am not sure
that I'll be able to use LC for that project (which is
so far only a prototype).

I don't mean to start another endless thread on this topic,
but one thing puzzles me though : I understand that LC doesn't
use decimal numbers for internal calculations for reasons
of speed, but then how come does js perform high precision
calculations without any speed issues ?

Best

Le 2021-03-22 09:41, Craig Newman via use-livecode a écrit :
> Check out this thread in the forum:
> 
> https://forums.livecode.com/viewtopic.php?f=7&t=25132&p=130985&hilit=longadd#p130985
> <https://forums.livecode.com/viewtopic.php?f=7&t=25132&p=130985&hilit=longadd#p130985>
> 
> Craig
> 
>> On Mar 21, 2021, at 7:34 AM, jbv via use-livecode 
>> <use-livecode at lists.runrev.com> wrote:
>> 
>> Paul, Sean,
>> 
>> Thank you both for your answers, although it doesn't really solve my 
>> problem.
>> The most surprising thing is that javascript runs these calculations 
>> without
>> problem :
>> (-0.0000007347880794884118999 / 1000000000) returns 
>> -7.347880794884119e-16
>> (-0.0000007347880794884118999 / 1000000000000) returns 
>> -7.347880794884119e-19
>> 
>> Best
>> 
>> 
>> Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :
>>> Livecode can handle floating point numbers up to 2^52. But then
>>> calculations would have to fit within that range in a floating point
>>> perspective too. So, once you add that ninth zero to the end of your
>>> divider, it struggles with the calculation and returns, effectively, 
>>> number
>>> out of range.
>>> Fractions in FP binary are tremendously complicated. Calculation in 
>>> it are
>>> even more so.
>>> https://www.h-schmidt.net/FloatConverter/IEEE754.html
>>> On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>> I don't have an answer, other than to say there has been a LOT of
>>>> discussion over the years about the limits of number precision in
>>>> LiveCode and a number of enhancement requests and 3rd party 
>>>> libraries
>>>> that have attempted to provide a framework for high precision math. 
>>>> You
>>>> might search the list archives or search the forum on the LC web 
>>>> site
>>>> for past discussions about number precision.
>>>> On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
>>>> > Hi list,
>>>> > I am a bit lost :
>>>> >
>>>> >    set numberformat to ".#########################"
>>>> >    put -0.0000007347880794884118999 / 1000
>>>> > -- returns -0.0000000007347880794884119  which is ok
>>>> >
>>>> >    put -0.0000007347880794884118999 / 1000000
>>>> > -- returns -0.0000000000007347880794884 -> also ok
>>>> >
>>>> >    put -0.0000007347880794884118999 / 100000000
>>>> > -- returns -0.0000000000000073478807949 -> still ok
>>>> >
>>>> >    put -0.0000007347880794884118999 / 1000000000
>>>> > -- returns 0 -> ?????
>>>> >
>>>> > Any explanations ?
>>>> > Thanks in advance.
>>>> > Best,
>>>> > jbv
>>>> >
>>>> > _______________________________________________
>>>> > use-livecode mailing list
>>>> > use-livecode at lists.runrev.com
>>>> > Please visit this url to subscribe, unsubscribe and manage your
>>>> > subscription preferences:
>>>> > http://lists.runrev.com/mailman/listinfo/use-livecode
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> use-livecode at lists.runrev.com
>>>> Please visit this url to subscribe, unsubscribe and manage your
>>>> subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your 
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode