[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Gambas-bugtracker] Bug #2921: Postgresql 19.900000000000055 is 20 in Gambas


http://gambaswiki.org/bugtracker/edit?object=BUG.2921&from=L21haW4-

Comment #4 by zxMarce:

Pardon me buttin' in.

DISCLAIMER: I do not intend to sound condescending; I just want to go to the root of the issue and maybe I over-explain things a bit. My fault.

This problem is caused most probably because floating-point numbers (float, double) are an approximate, but not precise, representation of Real Numbers.
For example, another "impossible" value with which I test my financial code is 4.85: It cannot be represented in either floating-point representation.

That is why there is at least another representation, focused particularly in financials and precision scientific values: Fixed-point numbers.
It normally consists of a bunch-o-bytes that is internally scaled, giving you a huge range (although not as big as floating-point), but this notation can deal with such little differences.

In PostgreSQL, the fixed-point datatypes are called "decimal" or "numeric", while the floating-point ones are "real" and "double precision". The PostgreSQL datatypes page even says which is "exact" (fixed-point) and "inexact" (floating-point). Link: https://www.postgresql.org/docs/current/datatype-numeric.html

As far as I can tell, Gambas lacks a fixed-point datatype (https://gambaswiki.org/wiki/lang/type?nh); thus making reading from and writing to fixed-point database fields lossy, as the precision is already lost at the time of writing an "inexact" floating-point in memory to an "exact" field. The field will contain the already inexact value... exactly.

Conclusion: Writing float values to fixed fields may cause rounding error, because the internal representation of the number in question may be already off-target from the moment (for example) a textbox numeric string value is converted to the float in memory to write to the database.

The actual fix: Introduce a native fixed-point datatype in Gambas, or develop a (fast!) fixed-point class in Gambas.

HTH,
zxMarce


----[ Gambas bugtracker-list is hosted by https://www.hostsharing.net ]----