[Gambas-user] floating point and single datatype precision...

Jose J. Rodriguez joe1962 at ...626...
Sat Aug 5 09:28:52 CEST 2006


On 8/5/06, sbungay <sbungay at ...981...> wrote:
>   Um... yeah... right...
>   So exactly HOW does the storage mechanism work such that it returns an
> incorrect result? If one was to store the value 22.79 into a variable of
> type single in C would one expect it to return a value other than 22.79?
> This means variables of type single and float can not be relied upon to
> return the value we store to them. IMHO that is not a good thing.
>

It's been that way since computers started storing numbers in binary
fashion. There's loads of info on the subject on the net, a quick
search (numeric precision computers) turned up this nice one:

http://www.vectorsite.net/tsfloat.html

Regards,
Joe1962




More information about the User mailing list