[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gambas-user] rounding a half
[Thread Prev] | [Thread Next]
- Subject: Re: [Gambas-user] rounding a half
- From: Benoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx>
- Date: Wed, 11 Dec 2024 16:24:54 +0100
- To: user@xxxxxxxxxxxxxxxxxxxxxx
Le 11/12/2024 à 16:05, Benoît Minisini a écrit :
Le 09/12/2024 à 21:27, Jussi Lahtinen a écrit :Hmmm... no this seems like a fundamental problem in floating points. You could do something like this:RETURN->_float.value = floor((PARAM->_float.value / power + 0.5) * t) / t * power;Where t is double with a large value. This would hide the problem into smaller decimal places. Not sure it's worth it. Probably implementing arbitrary precision datatype would be the right way. Benoit?JussiThe reason is that Round() rounds in binary, not in decimal. 1.015 and 1.055 do not have an exact representation in binary. 1.015 = 1.000000(11110101110000101000)* 1.055 = 1.0000(11100001010001111010)*Moreover, Round() sometimes uses a multiplication (when its second argument is positive) or a division (when its second argument is negative) to round.So, when the decimal digit where we want to round is "5", sometimes the number is rounding down, sometimes up.Regards,
I just introduce a little change in Round(): it now always use positive powers of ten for rounding. It leads to less errors, and return the expected result in the previous specific case.
But for sure we will find other numbers where Round() won't round the way we think.
Regards, -- Benoît Minisini.
Re: [Gambas-user] rounding a half | Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx> |
Re: [Gambas-user] rounding a half | Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx> |
Re: [Gambas-user] rounding a half | Benoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx> |