[Gambas-user] gb.test and gb.test.tap

[Benoît Minisini]

Le 10/03/2020 à 17:34, Tobias Boege a écrit :
> On Fri, 06 Mar 2020, Christof Thalhofer wrote:
>> Am 06.03.20 um 09:52 schrieb Benoît Minisini:
>>
>>> Is there a reason why 'gb.test.tap' is compiled before 'gb.test'? (see
>>> 'comp/src/order')
>>>
>>> It seems not logical, unless these two components are completely
>>> independent?
>>
> 
> I'm just coming back, slowly, from vacation and a flu. Please bear with me.
> 
> gb.test.tap is meant to be common code for printing and parsing TAP. It is
> what gb.test should use so that gb.test.tap is a transitive dependency of
> every project in "test mode". There will be a global Assert object that the
> test code and gb.test can use together to print a coherent TAP stream.
> This is why gb.test.tap must be compiled before gb.test -- it is a dependency.

Ah OK, I thought gb.test.tap was just for parsing TAP, not generating it.

> 
> Furthermore, gbt3 and the IDE should use gb.test.tap for parsing TAP output
> from test programs.

OK. I still don't really understand why there should be a 'gbt3' program.

> 
>> We will have to rethink dependencies ...
>> (please forget my Flowchart)
>>
>> This is what I think now, IMO:
>>
>> * gb.test should contain Assert.
>>
>> * Assert should be a module not a class
>>    (we do not need an object being Assert).
>>
> 
> Ok, why not.
> 
>> * gb.test.tap should contain gb.test as component
>>    and should provide its Unittest() like that:
>>
>>    Unittest(Tests as String, Optional Values ... )
>>
>>    The variable "Tests" is a comma separated string
>>    of testmodules and testmethods.
>>
>> Because:
>>
>> Maybe we later want to have another output module for testing instead of
>> gb.test.tap? Imagine things like that:
>>
>> gb.test.db -> writes test output into a database
>> gb.test.log -> hands over to syslog
>> gb.test.monitor -> hands over to a monitoring system
>> gb.test.csv -> writes tests output to a csv file
>>
>> Or someone can write it's own testing system.
>>
> 
> How does turning around the dependency I described above achieve this?
> gb.test.tap is a generic component for reading and writing TAP. It is
> only tangentially related to testing in Gambas. gb.test.tap is an
> implementation of TAP just like gb.web contains an implementation
> of JSON. gb.test on the other hand is the testing framework component
> shipped with Gambas, it should organize, setup and drive the testing.
> People can write their own testing framework (potentially), but gb.test
> is the one officially provided by Gambas.
> 
> If you want gb.test not to output TAP but CSV files, then make gb.test
> recognize some kind of configuration, maybe via environment variables,
> to load instead of gb.test.tap the gb.test.csv component. That component
> must provide an Assert module with the same interface as gb.test.tap.
> Then that Assert module can do with the assertions whatever it wants.
> If Assert becomes part of gb.test, how will you handle the conversion
> of high-level assertions to different output formats?
> 
> IMO, gb.test may or may not use gb.test.tap, but gb.test.tap should not
> depend on gb.test.
> 
> Regards,
> Tobias

All that seems logical to me.

'gb.test' does the testing, and pass the result of each test to 
'gb.test.tap' that prints it in TAP format.

You must agree on the interface the Assert class, and be sure that it is 
somewhat complete.

But I have a few remarks about what I see at the moment:

1) ASSERT is already the name of an instruction. But it is not a big 
deal, as there is no ambiguity between the instruction and the class.

2) When writing my test modules, I should only call methods in the 
'gb.test' component. In other words, I suggest having the Assert class 
in 'gb.test', and overriding it in 'gb.test.tap'.

The Assert class in 'gb.test' could emit nothing, or just an internal 
format for debugging purpose, or detect that no 'gb.test.*' module has 
been loaded and abort.

The Assert class in 'gb.test.tap' will do the real formatting job.

Note that the inheritance thing works only if Assert is a dynamic class 
with "CREATE STATIC" option, and all its methods being dynamic.

Assert methods in 'gb.test' could provide all the common stuff so that 
there is just printing and formatting in 'gb.test.tap'.

3) Could we rename "UnitTest" into "Test" directly? After all, I may 
want to do some tests that are not "unit tests".

Regards,

-- 
Benoît Minisini