[Gambas-user] sqlite3 float value errors
Doriano Blengino
doriano.blengino at ...1909...
Tue Mar 16 11:26:17 CET 2010
Jorge Carrión ha scritto:
>I use integer for money fields without decimals. I store cents... in display
>just field / 100. Hope it's valid for you
>
>2010/3/16 Bill-Lancaster <bill-lancaster at ...2231...>
>
>
>
>>Yes - that IS much simpler but I have found that some of my SQL fields are
>>not exact money values (like 10.23 say) but have tiny extra decimal values.
>>
>>These display as normal in a gridview, tableview or whatever.
>>
>>By converting SQL field to float called fTempVal
>>then fTempVal=fTempVal * 100. fTempVal - Int(fTempVal) gives a very
>>small decimal value.
>>
>>
>>
If you can not use integers (because your database is already in
production, or you have too much code to change), then you should round
the values before storing them in the database, or, at least, while
retrieving them.
If you multiply a float by 100, you do nearly nothing (the decimal point
moves a little). You should multiply, then take the integer part, and
then divide again:
goodvalue = int(badvalue * 100 + 0.5) / 100
The above should round to the cent: change "100" with "1000" or "10000"
to get more precision.
The "+ 0.5" is a quick (and erroneous but acceptable) way to round off.
Take care for negative values.
I am too lazy to try (why should I, when you have everything set up to
do it? :-)), but you can also try to use different multipliers - powers
of 2. Instead of multiply and divide for 100, you could use 128, which
is nearer to the way the machine stores floats. The only doubt is what
to add: if using 100, you add 0.5 - if you use 128, you should add
0.64... you can try to see if it works better.
Regards,
Doriano
More information about the User
mailing list