[Gambas-user] Possible bug in round() function

Jussi Lahtinen jussi.lahtinen at ...626...
Thu Jan 30 19:24:54 CET 2014


>
> I wouldn't say magical. Disillusioning maybe :-)
>
> 0.21 is not exactly representable with a finite number of bits. That's
> why you should expect errors when computing with it. When you multiply,
> the absolute error can get relatively large, obviously.
>

This probably has something to do with floating point errors, but not sure
it's that simple.

This is the exact presentation of value 59.535 with 64 bits:
59.534999999999996589394868351519107818603515625

And...

? round(59.534999999999996589394868351519107818603515625, -2)
59,54

? round(59.53499999999999, -2)
59,54

This seems to be where Gambas cut off:
? round(59.5349999999999, -2)
59,53


If I have understand right Gambas does cut off to the presentation when
they are shown, but the whole accuracy is intact in memory. So maybe cut
off is done in wrong order in Round().


Jussi



More information about the User mailing list