[Gambas-user] Issue 165 in gambas: Arithmetic problem with float
nando
nando_f at ...951...
Thu Dec 22 17:19:53 CET 2011
Personally, I do not see this as a problem for a float doing this calculation.
Since it is transcendental function, this will happen.
Plus, float is not 64 bits of precision, it is 52 bits.
I suggest use INT or LONG
This type of round-off error goes back to the beginning of time
-Fernando
---------- Original Message -----------
From: gambas at ...2524...
To: gambas-user at lists.sourceforge.net
Sent: Wed, 21 Dec 2011 19:12:21 +0000
Subject: [Gambas-user] Issue 165 in gambas: Arithmetic problem with float
> Status: New
> Owner: ----
> Labels: Version Type-Bug Priority-Medium OpSys-Any Dist-Any Arch-Any
> Desktop-Any GUI-Any
>
> New issue 165 by jussi.la... at ...626...: Arithmetic problem with float
> http://code.google.com/p/gambas/issues/detail?id=165
>
> 1) Describe the problem.
>
> Dim ff As Float = 2 ^ 23
>
> Print ff
>
> Gambas gives result: 8388607.99999999 while expected is 8388608.
> Floating point (64bit) precision should be enough for accurate result.
>
> 2) GIVE THE FOLLOWING INFORMATIONS (if they are appropriate):
>
> Version: TRUNK
> Revision: r4297
> Operating system: Linux
> Distribution: Xubuntu
> Architecture: x86_64
> GUI component: QT4 and GTK+
> Desktop used: XFCE
>
> 3) Provide a little project that reproduces the bug or the crash.
>
> Attached.
>
> Attachments:
> ArithmeticError-0.0.1.tar.gz 4.6 KB
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Gambas-user mailing list
> Gambas-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-user
------- End of Original Message -------
More information about the User
mailing list