[Gambas-user] Issue 165 in gambas: Arithmetic problem with float

[Randall Morgan]

Now you guys come up with samples of floating point rounding hehehehe....

A couple months ago I was looking for such rounding demos for my tutorials
I am writing. Note that this is just a part of programming. Many real
numbers do not have a direct representation in base 2 (binary). So to
represent those numbers the floating point library estimates the numbers.
The way to deal with this issue is to always define the precision needed
and use the most appropriate data type then limit the precision to
something less. This is a common issue with floating point math and why you
will find many specialized math libraries for science and engineering with
use much more precise methods to represent real numbers.

Thanks for the demos I may use them in my tutorials if that's ok with you?

Randall

On Fri, Dec 23, 2011 at 10:10 AM, <gambas at ...2524...> wrote:

>
> Comment #6 on issue 165 by jussi.la... at ...626...: Arithmetic problem with
> float
> http://code.google.com/p/gambas/issues/detail?id=165
>
> That is kind of expected result,
> since with doubles (float in Gambas) the actual value is (5.025 - 5):
> 0.0250000000000003552713678800500929355621337890625000
> And Gambas rounds it to:
> 0.0250000000000004
>
> See this for further details:
> http://en.wikipedia.org/wiki/Floating_point
>
>
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Gambas-user mailing list
> Gambas-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>



-- 
If you ask me if it can be done. The answer is YES, it can always be done.
The correct questions however are... What will it cost, and how long will
it take?