[Gambas-user] New component gb.test

[Tobias Boege]

On Mon, 21 May 2018, Christof Thalhofer wrote:
> Am 20.05.2018 um 20:04 schrieb Tobias Boege:
> 
> > If you don't Setup, the TapPrinter will keep track of the number of tests
> > you ran and spoof a plan at the end. Currently you have to call the
> > Finish() method for it to do that. Given how the Assert module is used,
> > maybe that should be made automatic somehow.
> > 
> > [ Actually, I think the TapParser doesn't throw an error yet if it doesn't
> >   encounter a plan line, which is a nuance I have likely overlooked. ]
> 
> A premature ending is not reported.
> 
> If I insert
> 
> "Assert.IsType(1 / 0, gb.Integer, "Crash")"
> 
> in TestAsserts as the last line in Main(), then the execution
> 
> gbx3 -s TestAsserts
> /home/christof/programming/gambas/gambas/comp/src/gb.test
> 
> outputs:
> 
> 1..10
> ok 1 - True is ok
> ok 2 - False is not ok
> ok 3 - Multiplication
> ok 4 - Gambas is sufficiently fast
> ok 5 - Float absolute error
> not ok 6 - Float relative error # SKIP Intentionally wrong
> ok 7 - Float relative error
> ok 8 - CStr returns a String
> ok 9 - LIKE email address
> ok 10 - MATCH email address
> ok 11 - not MATCH email address
> TestAsserts.Main.21: #26: Division by zero
> TestAsserts.Main.21
> 
> Tester outputs:
> 
> Test # 2: TestAsserts
> Planned tests: 0
> Exit code:     1
> Run tests:     0
> Duration:      0s
> --------------------
> Passed tests:  0
> Failed tests:  0
> Skipped tests: 0
> To-do tests:   0
> Bonus:         0
> Status:        FAILED
> ****************************************
> 
> Thats not good. Just the exit code says, that sth crashed, but
> TestHarness.Current neither knows where it crashed nor what crashed nor
> contains the number of passed tests.
> 
> Shouldn't it?
> 

Definitely. The problem is that

  Exec ["gbx3", "-s", Test, Project] To sOutput

returns me an empty sOutput in this case, so the parser gets no note of the
passed tests at all. I don't know why but it has been a TODO item (shortly
above that line) to not buffer all the output into a String variable anyway
but to use the Process object as a stream. Maybe then I'll get the lines
which were printed before the crash, too. Someone should reproduce that
behaviour with "Exec To stringvar" and file a bug as well...

While I'm at it, I should also capture stderr and convert it into diagnostic
messages. Doing that in Gambas always requires this weird mix of event-driven
and synchronous programming, but I just saw that there is a Process.Wait()
method, which should come in handy.

Regards,
Tobi

-- 
"There's an old saying: Don't change anything... ever!" -- Mr. Monk