[Gambas-user] Float hidden precision?

Benoît Minisini g4mba5 at gmail.com
Tue Nov 17 14:06:02 CET 2020


Le 17/11/2020 à 13:47, KKing a écrit :
> Is it possible for a float variable to have unwanted hidden distortion 
> in a held value?
> 
> I have a conundrum buried in a lot of code but essentially I have two 
> monetary values which originally arrive as strings in a csv file.
> 
> I then convert these to float values for other processing but by 
> debugging I have come across the following issue where essentially the 
> value has been distorted.
> 
> I've summarised the code below, and as a standalone test it works as 
> expected but as part of much larger project I get the result below.
> 
> The question is how can a float value in debugger show 429761 and when 
> converted to integer ends up as 429760 ?
> 

Never use floating point for storing currencies. Always use integers.

Floating point is binary, and a number with a finite number of decimal 
digits in base 10 often has an infinite number of decimal digits in 
binary, and so you have rounding error by converting between the two bases.

-- 
Benoît Minisini


More information about the User mailing list