[Gambas-user] Coming back to gb.test

[Benoît Minisini]

Le 03/04/2020 à 13:11, Tobias Boege a écrit :
> On Fri, 03 Apr 2020, Christof Thalhofer wrote:
>> Am 03.04.20 um 08:09 schrieb Christof Thalhofer:
>>
>>> I did not write this test just for a joke.
>>
>> And by the way. While developing this feature of gb.test I found a
>> failure in Gambas' Error.Clear, which did not clear the error number
>> correctly. This bug is meanwhile fixed.
>>
>> To find errors, to detect crashes, to be able to control situations
>> where a program crashes and to test it is an important part to ensure
>> stability of code.
>>
>> For instance your gb.openssl or any other security relevant program: I
>> can imagine that there are situations, where you would prefer to stop
>> the program immediately. Then you would be glad if you are able to test it.
>>
> 
> I fully agree with the necessity of testing abnormal conditions.
> If you want to test whether your library throws errors appropriately,
> then write an actual test for it in your test code:
> 
>    Try MyLibrary.Method("invalid")
>    Assert.Ok(Error, "error happened as expected")
>    Assert.Match(Error.Text, "Method cannot process 'invalid' input", "error message is correct")
> 
> That's all that's being debated here: move the blanket Try *around*
> a test method *into* the test method. Test specifically for errors
> that you know should be thrown under this and this condition.
> 
> If gb.test has to catch errors on the programmer's code paths, then
> that means that they are *unexpected* errors which slip through the
> programmer's test methods. Errors from the library which the test code
> does not explicitly anticipate. To me that means that the test code
> is buggy (expects things to work that don't) or that the library is
> buggy (doesn't work when it should).
> 
> To a certain extent that's ok. Test code can have bugs too which need
> to be debugged when they occur. I feel like your catching of all errors
> in gb.test is a way to deal with unintentionally broken tests -- tests
> which don't catch all the errors they provoke -- and get some information
> about what happened in order to fix it. But as I said, letting errors
> escape and crash the interpreter will allow you to run the debugger on it
> and get more information directly from the source of the error than
> catching it in gb.test can give.
> 
> The norm should be that test methods don't raise errors of their own:
> 
>    - If a test decides that it needs to stop immediately because
>      something is really bad, then it has the option of BailOut
>      in the protocol.
> 
>    - Test code should also not provoke errors in the tested code,
>      except for the purpose of catching them and making Assert'ions
>      about them.
> 
> Regards,
> Tobi
> 

Maybe I will say rubbish, but if both can be useful, couldn't we make 
that an option ? "Let gb.test catch all errors" or "let the interpreter 
crash".

-- 
Benoît Minisini