[Gambas-user] Questions about gb.test and gb.test.tap

[Christof Thalhofer]

Am 24.02.20 um 23:20 schrieb Tobias Boege:

> gb.test tries to do all of that in one process at the moment and this is
> a lack of separation that facilitates it doing "impure" things. Concretely
> I mean this
> 
>   https://gitlab.com/gambas/gambas/-/blob/3752088d37/comp/src/gb.test/.src/TestSuite/TestCase.class#L48
> 
>   Try Object.Call($MyTestModule, Me.name)
>   If Error Then
>     Inc Track.Counter
>     Track.NOKs.Add(Track.Counter)
>     Print "not ok " & Track.Counter & " " &
>     Track.TestModuleName & ":" & Track.TestName & " Raised error: " & Error.Text
>     Error.Clear
>   Endif
> 
> If a *test method* raises an uncaught error, then a *test harness*-like
> piece of code catches it and spoofs a test failure in the TAP stream.
> Arguably that is not correct, semantically. If an error is raised and not
> caught, then the *test code* is buggy, not the *tested* code. Furthermore,
> this trick isn't always helpful: it only inserts one test failure when
> the entire test method aborted due to an error at any point, what if the
> test method had a couple more tests planned? Then the TAP stream will be
> subsequently shifted and parts will be missing but it continues to run.

I like your criticism very much! It is wonderful if someone takes a
close look at the code and the decisions in it.

Regarding this case:

I think that is not really of interest (that the testmodule failed in an
early state, and subsequent testmethods in it may have reported Oks
later ...).

At this point I also could have done a BailOut and stop the entire
testing process instead of just report that an error happened in the
testmodule. Earlier I think there was a BailOut and that was a bit
better. But gb.test reports here, where the error happened, in
"Track.TestModuleName & ":" & Track.TestName" and that's all the
programmer has to know about. I don't give a damn what happens after that!

If there is one error/crash the program is buggy and this is, what I
want to see as a programmer. For sure, the testcode could also be buggy,
but who cares? The programmer has to fix!
Either the testee or the tester. That's all. That's the only thing that
interests me as a programmer.

The real reason I did that in that way you don't like, was:

With the current gb.test I can step through the test code and the tested
code with the debugger *because the processes are not separated*! This
is very convenient for daily work as a programmer! You step through and
see: "Oh, my testcode was buggy" or "Oh, my tested code does a silly thing".

Its the matter of how to work fluently with testing and programming. The
current gb.test supports my way to program best and it's not because I
like spaghetti.

If the testing was done according to your ideas, I could only see that
something is broken, but I couldn't step through. I would really miss that.

You complain about that other test systems you know also do not follow
your idea of "pureness" and "semantically correct" separation.

I bet, their programmers faced the same problem and also came to the
same decision as I did here.

Btw, this is something we discussed a year ago and our positions have
not changed.

:-)

Alles Gute

Christof Thalhofer

-- 
Dies ist keine Signatur

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gambas-basic.org/pipermail/user/attachments/20200225/8ab91bd8/attachment.sig>