[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gambas-user] rounding a half
[Thread Prev] | [Thread Next]
- Subject: Re: [Gambas-user] rounding a half
- From: Benoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx>
- Date: Wed, 11 Dec 2024 16:05:43 +0100
- To: user@xxxxxxxxxxxxxxxxxxxxxx
Le 09/12/2024 à 21:27, Jussi Lahtinen a écrit :
Hmmm... no this seems like a fundamental problem in floating points. You could do something like this:RETURN->_float.value = floor((PARAM->_float.value / power + 0.5) * t) / t * power;Where t is double with a large value. This would hide the problem into smaller decimal places. Not sure it's worth it. Probably implementing arbitrary precision datatype would be the right way. Benoit?Jussi
The reason is that Round() rounds in binary, not in decimal. 1.015 and 1.055 do not have an exact representation in binary. 1.015 = 1.000000(11110101110000101000)* 1.055 = 1.0000(11100001010001111010)*Moreover, Round() sometimes uses a multiplication (when its second argument is positive) or a division (when its second argument is negative) to round.
So, when the decimal digit where we want to round is "5", sometimes the number is rounding down, sometimes up.
Regards, -- Benoît Minisini.
Re: [Gambas-user] rounding a half | Benoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx> |
Re: [Gambas-user] rounding a half | Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx> |
Re: [Gambas-user] rounding a half | Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx> |