[Gambas-user] Calculation Bug?

Pablo Vera pvera at ...729...
Tue Nov 28 00:54:30 CET 2006


The problem is that your calculation is made up of Integers, so the 
partial results are also integers and this causes rounding errors.

To solve it, start with a floating number, so that all the calculations 
are done as a Float, like this:

   fTmp1 =  1.0 * 618 * 2 * 60 * 1000000 / (618 * 2) / 360

Which results in: 166666.666666667

Saludos,
Pablo Vera

James Linden wrote:
> Gambas does not appear to know how to handle 8 digit numbers...
> 
> Dim fTmp1 As Float
> fTmp1 = 60 * 1000000
> Message.Info( CStr( fTmp1 ) )
> 
> -- James
> 
> PS: Isn't your formula a bit redundant -- you multiply by (618*2), then 
> divide by (618*2)... ?
> 
> R. Stormo wrote:
>> I'm doing a small calculation for a CDI box to a 4 wheel bike. And when I
>> putting the function togheter it does not calculate the right values
>> according to the calculator.
>>
>> >From Debug.
>> Dim fTmp1 as Float
>> fTmp1 =  618 * 2 * 60 * 1000000 / (618 * 2) / 360 =2574.51449119022
>>
>>
>> The calculator gives the value
>>
>> 166666,666666667
>>
>> Spreadsheet gives
>> 166666,666666667
>>
>> So there must be something wrong in gambas when calculation is done.
>>
>> Because if I take one zero from the 1000000 it gives me a total different
>> value
>> Dim fTmp1 as Float
>> fTmp1 =  618 * 2 * 60 * 100000 / (618 * 2) / 360 =-2638.29241280115
>>
>> Should have been 16666,6666666667
>>
>>
>>
>> Regards R. Stormo
>>
>>   My Gambas Community  http://gambasforum.tk
>>
>>
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Gambas-user mailing list
> Gambas-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-user
> 




More information about the User mailing list