[Gambas-user] floating point and single datatype precision...
sbungay
sbungay at ...981...
Sat Aug 5 06:28:56 CEST 2006
Um... yeah... right...
So exactly HOW does the storage mechanism work such that it returns an
incorrect result? If one was to store the value 22.79 into a variable of
type single in C would one expect it to return a value other than 22.79?
This means variables of type single and float can not be relied upon to
return the value we store to them. IMHO that is not a good thing.
Fabien Bodard wrote:
> Le Samedi 5 Août 2006 02:03, sbungay a écrit :
>
>> Anybody know Why it is that when a value like 22.79 is stored in a
>>variable of type single, it represents it internally as 22.7900009155227?
>> Just curious.
>
> because of the binary storing...
>
>>-------------------------------------------------------------------------
>>Take Surveys. Earn Cash. Influence the Future of IT
>>Join SourceForge.net's Techsay panel and you'll get the chance to share
>>your opinions on IT & business topics through brief surveys -- and earn
>>cash
>>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>>_______________________________________________
>>Gambas-user mailing list
>>Gambas-user at lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/gambas-user
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Gambas-user mailing list
> Gambas-user at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gambas-user
>
More information about the User
mailing list