[Gambas-user] New component gb.test

[Adrien Prokopowicz]

Le 21/05/2018 à 16:55, Christof Thalhofer a écrit :
> Am 20.05.2018 um 20:04 schrieb Tobias Boege:
> 
>> If you don't Setup, the TapPrinter will keep track of the number of tests
>> you ran and spoof a plan at the end. Currently you have to call the
>> Finish() method for it to do that. Given how the Assert module is used,
>> maybe that should be made automatic somehow.
>>
>> [ Actually, I think the TapParser doesn't throw an error yet if it doesn't
>>    encounter a plan line, which is a nuance I have likely overlooked. ]
> 
> A premature ending is not reported.
> 
> If I insert
> 
> "Assert.IsType(1 / 0, gb.Integer, "Crash")"
> 
> in TestAsserts as the last line in Main(), then the execution
> 
> gbx3 -s TestAsserts
> /home/christof/programming/gambas/gambas/comp/src/gb.test
> 
> outputs:
> 
> 1..10
> ok 1 - True is ok
> ok 2 - False is not ok
> ok 3 - Multiplication
> ok 4 - Gambas is sufficiently fast
> ok 5 - Float absolute error
> not ok 6 - Float relative error # SKIP Intentionally wrong
> ok 7 - Float relative error
> ok 8 - CStr returns a String
> ok 9 - LIKE email address
> ok 10 - MATCH email address
> ok 11 - not MATCH email address
> TestAsserts.Main.21: #26: Division by zero
> TestAsserts.Main.21
> 
> Tester outputs:
> 
> Test # 2: TestAsserts
> Planned tests: 0
> Exit code:     1
> Run tests:     0
> Duration:      0s
> --------------------
> Passed tests:  0
> Failed tests:  0
> Skipped tests: 0
> To-do tests:   0
> Bonus:         0
> Status:        FAILED
> ****************************************
> 
> Thats not good. Just the exit code says, that sth crashed, but
> TestHarness.Current neither knows where it crashed nor what crashed nor
> contains the number of passed tests.
> 
> Shouldn't it?
> 
> Alles Gute
> 
> Christof Thalhofer

There's something I just noticed in your implementation: why is a "test" 
(in TAP terminology) mapped to a single assertion, and not to a whole 
test method ?

The problem with this approach is that all assertions share state, which 
should not be the case with tests: ideally a test should have no 
influence on subsequent tests, otherwise a test following a failure 
could be either a false positive or a false negative, and you can't 
really test things in isolation.

After reading the TAP/Perl spec a bit more thoroughly, it seems they 
also use this approch, which I find really wierd. Admittedly i've never 
ever encountred Perl, but of all the testing tools I've seen and used 
(I've looked into C, Rust, Ruby, Java, C# and Javascript for my testing) 
I've never seen this approach before.

-- 
Adrien Prokopowicz