[Gambas-user] Arithmetic general question
BB
adamnt42 at gmail.com
Tue Jan 24 21:48:04 CET 2023
On 25/1/23 12:35 am, T Lee Davidson wrote:
> On 1/23/23 23:53, BB wrote:
>> Var1 and Var2 are defined as Variants. Say Var1 is an integer value
>> 1300 and Var2 is a float value 1285.44 I want to get the difference
>> between the two _as an integer_. (Var2 can be bigger or smaller than
>> Var1)
>>
>> If I just use Var1 - Var2, I get (printed as an integer) 14, but is
>> that arithmetically correct? Similarly, if Var2 is 1318.55 I get -18
>>
>> Should I be converting or rounding the values before doing the
>> subtraction?
>
> No. Doing that would give inaccurate results. If anything, you would
> round or truncate _after_ getting the difference.
>
> In your example, the conversion to integer is obviously truncating the
> fractional part. Maybe that is what leads you to question its
> correctness. "Var1 - Var2" correctly expresses the difference between
> two variable numbers regardless of their sign or the sign of the result.
>
>
Thanks Lee!
More information about the User
mailing list