[Gambas-user] Inconsistency with floating point notation.

Jussi Lahtinen jussi.lahtinen at ...626...
Mon Jan 3 21:24:17 CET 2011


Little problem with floating points... as you can see (below) -2E-16 is non
zero,
but if you write same value with other notation (-0.0000000000000002) it
will be interpreted as zero.

If -0.000000000000000166533453693773 > -2E-16 Then
Debug "First is bigger."
Endif

If -0.000000000000000166533453693773 > -0.0000000000000002 Then
Debug "Not anymore..."
Endif

If -0.000000000000000166533453693773 = -0.0000000000000002 Then
Debug "Now it is equal."
Endif

If -0.0000000000000002 <> -2E-16 Then
Debug "They are not equal?"
Endif

Debug Format$(-2E-16,
".#####################################################")
Debug Format$(-0.0000000000000002,
".#####################################################")


Result:
mTest.DoTests.573: First is bigger.
mTest.DoTests.581: Now it is equal.
mTest.DoTests.585: They are not equal?
mTest.DoTests.588: -.0000000000000002
mTest.DoTests.589: 0


Gambas 3 rev 3436 @ Ubuntu 10.10 64bit
Jussi



More information about the User mailing list