[Gambas-user] New component gb.test

[Tobias Boege]

On Mon, 21 May 2018, Adrien Prokopowicz wrote:
> There's something I just noticed in your implementation: why is a "test" (in
> TAP terminology) mapped to a single assertion, and not to a whole test
> method ?
> 

There are two things that might be called a "test". One is a single
assertion, which produces an "ok" or "not ok" line, and the other is
multiple assertions bundled into a file -- that file would be called
a test, too, if you ask me.

> The problem with this approach is that all assertions share state, which
> should not be the case with tests: ideally a test should have no influence
> on subsequent tests, otherwise a test following a failure could be either a
> false positive or a false negative, and you can't really test things in
> isolation.
> 

I think that depends. If you create an object and want to test two methods,
these two assertions should share state. Or what do you mean?

> After reading the TAP/Perl spec a bit more thoroughly, it seems they also
> use this approch, which I find really wierd. Admittedly i've never ever
> encountred Perl, but of all the testing tools I've seen and used (I've
> looked into C, Rust, Ruby, Java, C# and Javascript for my testing) I've
> never seen this approach before.
> 

I, on the other hand, am not familiar with those. For me, naively, testing
is executing scripts that probe the API of your project and report their
results. How you write those scripts is entirely up to you, as long as
their output follows the protocol.

And that is about the level at which I worked. I just implemented TAP
because Christof mentioned it, I recognised it and thought it was a
good idea. I don't know if that was clear to you, but the "higher-level"
stuff was mostly duct-taped on top to be able to run something that
produces and consumes TAP and I took inspiration for that from perl.
That part is very much intended to evolve. [ Maybe my deleting all
those COMUnit classes gave away the impression that I replaced them,
but I really just didn't want to spend time porting them to the new
TAP base, and instead implemented a fresh and small TestHarness class
to have something presentable. Sorry, Christof :-) ]

It is kind of hard at the moment to see where the design of gb.test is
supposed to go and I definitely lack synapses in the usability department,
which many of the recent remarks were about. If you have experience with
mature testing methods, please take the lead.

However, despite your comment about state sharing above, or because
I didn't understand the problem about it, I still think TAP is a good
format, regardless of how tests are presented to the user in the end.
I can't really imagine that the TAP architecture can get this much
in the way of testing workflows basically everywhere non-Perl.

Regards,
Tobi

PS: I should also note that I have to gradute in a few months and can't
spend much time researching testing and evaluating proposals. The frame
is really tight this time.

-- 
"There's an old saying: Don't change anything... ever!" -- Mr. Monk