[Gambas-user] Calculation Bug?

[R. Stormo]

Pablo Vera wrote:
> 
> The problem is that your calculation is made up of Integers, so the 
> partial results are also integers and this causes rounding errors.
> 
> To solve it, start with a floating number, so that all the calculations 
> are done as a Float, like this:
> 
>    fTmp1 =  1.0 * 618 * 2 * 60 * 1000000 / (618 * 2) / 360
> 
> Which results in: 166666.666666667
> 
> Saludos,
> Pablo Vera
> 

Superb , Thanks alot


> James Linden wrote:
>> Gambas does not appear to know how to handle 8 digit numbers...
>> 
>> Dim fTmp1 As Float
>> fTmp1 = 60 * 1000000
>> Message.Info( CStr( fTmp1 ) )
>> 
>> -- James
>> 
>> PS: Isn't your formula a bit redundant -- you multiply by (618*2), then 
>> divide by (618*2)... ?
>  
Yes , but if the real formala was shown with variables you would not have
seen the problem. And the hole
formula is a little bit longer ;)


Regards Rohnny
       My Gambas Community http://gambasforum.tk


-- 
View this message in context: http://www.nabble.com/Calculation-Bug--tf2714348.html#a7573460
Sent from the gambas-user mailing list archive at Nabble.com.