[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gambas-user] rounding a half
[Thread Prev] | [Thread Next]
- Subject: Re: [Gambas-user] rounding a half
- From: Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx>
- Date: Mon, 9 Dec 2024 22:27:05 +0200
- To: wig@xxxxxxxxx, Gambas Mailing List <user@xxxxxxxxxxxxxxxxxxxxxx>
Hmmm... no this seems like a fundamental problem in floating points. You could do something like this: RETURN->_float.value = floor((PARAM->_float.value / power + 0.5) * t) / t * power; Where t is double with a large value. This would hide the problem into smaller decimal places. Not sure it's worth it. Probably implementing arbitrary precision datatype would be the right way. Benoit? Jussi On Mon, Dec 9, 2024 at 5:24 PM Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx> wrote: > Interesting... this is a floating point presentation problem. Should be > fixable. > It is due to the internally used floor. And the problem occurs in Gambas > as well: > > Print (1.055 / 0.01 + 0.5);; Floor((1.055 / 0.01 + 0.5)) > 106 105 > > > Jussi > > > On Mon, Dec 9, 2024 at 2:19 PM wig <wig@xxxxxxxxx> wrote: > >> >> >> I would think that rounding a half would always react the same, either >> up or down; >> >> f = 1.015 >> >> Print "Round(" & f & ", -2) = " & Round(f, -2) >> >> Result: >> >> Round(1.015, -2) = 1.02 >> >> f = 1.055 >> >> Print "Round(" & f & ", -2) = " & Round(f, -2) >> >> Round(1.055, -2) = 1.05 >> >> What should I expect? >> >> I used Gambas 3.18.4 >> >> System info see attach >> >> Kind greetings, >> >> Wig >> >> _______________________________________________ >> Gambas-user mailing list >> Gambas-user@xxxxxxxxxxxxxxxxxxxxx >> https://lists.sourceforge.net/lists/listinfo/gambas-user >> >
Re: [Gambas-user] rounding a half | Benoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx> |
Re: [Gambas-user] rounding a half | Jussi Lahtinen <jussi.lahtinen@xxxxxxxxx> |