[Gambas-user] Issue 165 in gambas: Arithmetic problem with float

nando nando_f at ...951...
Thu Dec 22 17:19:53 CET 2011


Personally, I do not see this as a problem for a float doing this calculation.
Since it is transcendental function, this will happen.
Plus, float is not 64 bits of precision, it is 52 bits.
I suggest use INT or LONG
This type of round-off error goes back to the beginning of time
-Fernando

---------- Original Message -----------
From: gambas at ...2524...
To: gambas-user at lists.sourceforge.net
Sent: Wed, 21 Dec 2011 19:12:21 +0000
Subject: [Gambas-user] Issue 165 in gambas: Arithmetic problem with float

> Status: New
> Owner: ----
> Labels: Version Type-Bug Priority-Medium OpSys-Any Dist-Any Arch-Any  
> Desktop-Any GUI-Any
> 
> New issue 165 by jussi.la... at ...626...: Arithmetic problem with float
> http://code.google.com/p/gambas/issues/detail?id=165
> 
> 1) Describe the problem.
> 
> Dim ff As Float = 2 ^ 23
> 
> Print ff
> 
> Gambas gives result: 8388607.99999999 while expected is 8388608.
> Floating point (64bit) precision should be enough for accurate result.
> 
> 2) GIVE THE FOLLOWING INFORMATIONS (if they are appropriate):
> 
> Version: TRUNK
> Revision: r4297
> Operating system: Linux
> Distribution: Xubuntu
> Architecture: x86_64
> GUI component: QT4 and GTK+
> Desktop used: XFCE
> 
> 3) Provide a little project that reproduces the bug or the crash.
> 
> Attached.
> 
> Attachments:
> 	ArithmeticError-0.0.1.tar.gz  4.6 KB
> 
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create 
> new or port existing apps to sell to consumers worldwide. Explore the 
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Gambas-user mailing list
> Gambas-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-user
------- End of Original Message -------





More information about the User mailing list