[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gambas-user] rounding a half


Le 09/12/2024 à 21:27, Jussi Lahtinen a écrit :
Hmmm... no this seems like a fundamental problem in floating points.
You could do something like this:

RETURN->_float.value = floor((PARAM->_float.value / power + 0.5) * t) / t * power;

Where t is double with a large value. This would hide the problem into smaller decimal places. Not sure it's worth it. Probably implementing arbitrary precision datatype would be the right way. Benoit?


Jussi


The reason is that Round() rounds in binary, not in decimal.

1.015 and 1.055 do not have an exact representation in binary.

1.015 = 1.000000(11110101110000101000)*
1.055 = 1.0000(11100001010001111010)*

Moreover, Round() sometimes uses a multiplication (when its second argument is positive) or a division (when its second argument is negative) to round.

So, when the decimal digit where we want to round is "5", sometimes the number is rounding down, sometimes up.

Regards,

--
Benoît Minisini.


Follow-Ups:
Re: [Gambas-user] rounding a halfBenoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx>
References:
Re: [Gambas-user] rounding a halfJussi Lahtinen <jussi.lahtinen@xxxxxxxxx>
Re: [Gambas-user] rounding a halfJussi Lahtinen <jussi.lahtinen@xxxxxxxxx>