[Gambas-user] Issue 165 in gambas: Arithmetic problem with float

[Jussi Lahtinen]

Here is real sample for you:

Dim sngA As Single = 4.1
Dim sngB As Single = 2.2

Print sngA - sngB

1.8999999

Jussi




On Fri, Dec 23, 2011 at 20:33, Jussi Lahtinen <jussi.lahtinen at ...626...>wrote:

> You can use the samples. BUT some of them are because of bug of Gambas,
> not the general problem with floating points!
> Example, Log2(536870912) = 29 is true with double precision floating
> points but not with Gambas.
> Also 8388608.0 is 8388608.0 in double precision floating point, but in
> Gambas *shows* 8388607,99999999.
>
> If you use these all as samples you will give wrong information.
>
> Jussi
>
>
>
>
>
>
> On Fri, Dec 23, 2011 at 20:22, Randall Morgan <rmorgan62 at ...626...> wrote:
>
>> Now you guys come up with samples of floating point rounding hehehehe....
>>
>> A couple months ago I was looking for such rounding demos for my tutorials
>> I am writing. Note that this is just a part of programming. Many real
>> numbers do not have a direct representation in base 2 (binary). So to
>> represent those numbers the floating point library estimates the numbers.
>> The way to deal with this issue is to always define the precision needed
>> and use the most appropriate data type then limit the precision to
>> something less. This is a common issue with floating point math and why
>> you
>> will find many specialized math libraries for science and engineering with
>> use much more precise methods to represent real numbers.
>>
>> Thanks for the demos I may use them in my tutorials if that's ok with you?
>>
>> Randall
>>
>> On Fri, Dec 23, 2011 at 10:10 AM, <gambas at ...2524...> wrote:
>>
>> >
>> > Comment #6 on issue 165 by jussi.la... at ...626...: Arithmetic problem
>> with
>> > float
>> > http://code.google.com/p/gambas/issues/detail?id=165
>> >
>> > That is kind of expected result,
>> > since with doubles (float in Gambas) the actual value is (5.025 - 5):
>> > 0.0250000000000003552713678800500929355621337890625000
>> > And Gambas rounds it to:
>> > 0.0250000000000004
>> >
>> > See this for further details:
>> > http://en.wikipedia.org/wiki/Floating_point
>> >
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Write once. Port to many.
>> > Get the SDK and tools to simplify cross-platform app development. Create
>> > new or port existing apps to sell to consumers worldwide. Explore the
>> > Intel AppUpSM program developer opportunity.
>> appdeveloper.intel.com/join
>> > http://p.sf.net/sfu/intel-appdev
>> > _______________________________________________
>> > Gambas-user mailing list
>> > Gambas-user at lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/gambas-user
>> >
>>
>>
>>
>> --
>> If you ask me if it can be done. The answer is YES, it can always be done.
>> The correct questions however are... What will it cost, and how long will
>> it take?
>>
>> ------------------------------------------------------------------------------
>> Write once. Port to many.
>> Get the SDK and tools to simplify cross-platform app development. Create
>> new or port existing apps to sell to consumers worldwide. Explore the
>> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
>> http://p.sf.net/sfu/intel-appdev
>> _______________________________________________
>> Gambas-user mailing list
>> Gambas-user at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/gambas-user
>>
>
>