[Gambas-devel] Suggestion : Gambas Unit Testing

Benoît Minisini gambas at ...1...
Wed Dec 10 08:20:24 CET 2014


Le 10/12/2014 04:08, Adrien Prokopowicz a écrit :
> Le Tue, 09 Dec 2014 22:45:01 +0100, Benoît Minisini
> <gambas at ...1...> a écrit:
>
>> I like the Tobias' idea of "just" comparing the result of a unit test
>> with a text file, like in Python. I wrote "just" because there must be
>> some pattern matching magic in the comparison.
>>
>> But I prefer the Adrian way of coding all the tests inside the project.
>
> I like Tobias' idea too, it can be handy to write fast little tests, even
> if it is quite limited compared to writing actual code.
>
>> I can add a new special hidden directory for storing the "UnitTests". It
>> will be compiled like the rest of the code, and will be put inside the
>> executable. It will be run by a special option of gbx3 / gbr3.
>>
>> For example, if you have a "gb.abcd" component, you will run something
>> like:
>>
>> $ gbr3 -t gb.abcd.gambas
>>
>> to execute the unit tests and check the result.
>
> I do not like the idea of having the unit tests in the final executable.
> To me, unit testing should be used for development purposes only, not
> something the end user should see.
>
> Plus, if a project has a lot of unit tests (100+ is a reasonable number),
> and optionally a good amount of test data (in the case of gb.xml, for
> example, I would put a few hundreds of XML/HTML files to test the parser),
> it would end up wasting a noticeable amount of the user's disk space.

As unit test classes will be stored in their own directories, it should 
be easy for the archiver not to include them.

>
>> Moreover, we can make "UnitTest" a new "form" type in the IDE. We could
>> store the code to test in the class, that class automatically inheriting
>> the UnitTest class. And the form file would be just the textual result
>> that must be emitted by the tests.
>>
>> What do you think about that?
>
> A test's result should not rely on its standard output IMHO. There are
> too many ways this could go wrong for the test writer (an extra space
> here, a little typo there...).

This is why there will be pattern matching magic. i.e. some part of the 
expected outputs will be 'jokers'.

> Additionally, it would not be very
> indicative about why/where the test failed (no stack trace or error
> message), and I think that maintaining two files for a test would quickly
> become a pain. Relying only on the Assert() methods is the best solution
> to me.
>

This is a better argument. I can't tell what is the best, as I never use 
these kinds of things a lot.

> For IDE integration, what I had in mind was, as you said, creating a
> new "form" type, so you could simply do New>Unit Test..., give it a name
> and immediately start implementing the pre-generated _Run() method.

Why an hidden method?

> Then, starting a test (either automatically after compilation or manually
>   from a menu entry) would open a tab containing the full report.
> I can work on that if you want.
>
> In the meantime, I will start working on a class to manage and retrieve
> the results from the testing process, so it can directly be used by the
> IDE (and maybe other projects).
>

OK, so the tests results should be a text file format easily transformed 
into a nice table by the IDE. Please don't use XML. :-)

Regards,

-- 
Benoît Minisini




More information about the Devel mailing list