[Gambas-user] Questions about gb.test and gb.test.tap

[Christof Thalhofer]

Am 28.02.20 um 10:33 schrieb Christof Thalhofer:

> afaics the work in branch gb.test.tap is finished, it can be rebased or
> merged into master. Are there any complaints?

Oh-kay! Now I see sth:

The behavior before was: The module TestFailures should run until end
and report errors and failures as not ok.

Instead it bailouts at "Assert.Equals(oOne, oTwo)" in
"TestEqualsFailure()" with

"Bail out! Unittest stopped with error "Type mismatch: wanted String,
got Object instead" caused by TestFailures.TestEqualsFailure."

*This is definitely wrong!* There are two different objects involved,
not a string ...

But! This can only be seen if "TestDoACrash" is renamed to
"TestZDoACrash" because this bailouts also, but earlier.

The last one could be a meaningful decision. But my decision before was:

----------------------------------------------------------

Failures and errors in testmethods or tested code are reported as "not
ok" and get a diagnosis note. It cannot be distinguished between test
code and tested code. Both is written by the programmer, both can
contain failures.

This happens in TestSuite.Run() and in TestCase.Run() by encapsulating
Object.Call() with Try and investigating errors or crashes afterwards.

But a crash, or error in gb.test itself leads to a BailOut. Or – if the
programmer wants to Bailout this can be done with Assert.Bailout().

A BailOut (in my opinion) is a situation, where there is no answer
possible any more. An Error instead is a defined response of the system.
Maybe one doesn't like it, but it is a defined answer.

But now an error in a testmethod or a tested method always leads to a
BailOut. I do not like this decision. If I want to see "not ok" on
errors I cannot do this any more ... but it was possible before!

For example: I never would have found out that Assert.Equals(oOne, oTwo)
with two objects says something wrong about having detected a string ...
because the system bailed out before.

I am aware that I am now saying something different than in a previous
mail. I can only say that I did not have a complete overview of this
before. Only when something is suddenly missing, which you did a year
ago, you understand why you did it.

----------------------------------------------------------

As I said in an earlier mail: I have finetuned the testing over a long
time in different situations, while working with it. Especially the
behavior how it handles errors and crashes.

Now a meaningful behavior has disappeared. I definetely do not like this!

----------------------------------------------------------

To restore the (better) behavior of the system before (and also satisfy
aesthetics):

Every "Object.Call" in TestCase and TestSuite must have a leading Try
and if an error happened it has to be printed out by Assert in gb.test.tap.

Would you do this, Tobi, or should I?


Alles Gute

Christof Thalhofer

-- 
Dies ist keine Signatur

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gambas-basic.org/pipermail/user/attachments/20200228/e3f55d76/attachment.sig>