[Gambas-user] sqlite3 float value errors

Doriano Blengino doriano.blengino at ...1909...
Tue Mar 16 11:26:17 CET 2010


Jorge Carrión ha scritto:

>I use integer for money fields without decimals. I store cents... in display
>just field / 100. Hope it's valid for you
>
>2010/3/16 Bill-Lancaster <bill-lancaster at ...2231...>
>
>  
>
>>Yes - that IS much simpler but I have found that some of my SQL fields are
>>not exact money values (like 10.23 say) but have tiny extra decimal values.
>>
>>These display as normal in a gridview, tableview or whatever.
>>
>>By converting SQL field to float called fTempVal
>>then  fTempVal=fTempVal * 100.   fTempVal - Int(fTempVal) gives a very
>>small decimal value.
>>
>>    
>>
If you can not use integers (because your  database is already in 
production, or you have too much code to change), then you should round 
the values before storing them in the database, or, at least, while 
retrieving them.

If you multiply a float by 100, you do nearly nothing (the decimal point 
moves a little). You should multiply, then take the integer part, and 
then divide again:

    goodvalue = int(badvalue * 100 + 0.5) / 100

The above should round to the cent: change "100" with "1000" or "10000" 
to get more precision.
The "+ 0.5" is a quick (and erroneous but acceptable) way to round off. 
Take care for negative values.

I am too lazy to try (why should I, when you have everything set up to 
do it? :-)), but you can also try to use different multipliers - powers 
of 2. Instead of multiply and divide for 100, you could use 128, which 
is nearer to the way the machine stores floats. The only doubt is what 
to add: if using 100, you add 0.5 -  if you use 128, you should add 
0.64... you can try to see if it works better.

Regards,
Doriano






More information about the User mailing list