[Gambas-devel] Suggestion : Gambas Unit Testing

[Adrien Prokopowicz]

Le Tue, 09 Dec 2014 22:45:01 +0100, Benoît Minisini
<gambas at ...1...> a écrit:

> I like the Tobias' idea of "just" comparing the result of a unit test
> with a text file, like in Python. I wrote "just" because there must be
> some pattern matching magic in the comparison.
>
> But I prefer the Adrian way of coding all the tests inside the project.

I like Tobias' idea too, it can be handy to write fast little tests, even
if it is quite limited compared to writing actual code.

> I can add a new special hidden directory for storing the "UnitTests". It
> will be compiled like the rest of the code, and will be put inside the
> executable. It will be run by a special option of gbx3 / gbr3.
>
> For example, if you have a "gb.abcd" component, you will run something  
> like:
>
> $ gbr3 -t gb.abcd.gambas
>
> to execute the unit tests and check the result.

I do not like the idea of having the unit tests in the final executable.
To me, unit testing should be used for development purposes only, not
something the end user should see.

Plus, if a project has a lot of unit tests (100+ is a reasonable number),
and optionally a good amount of test data (in the case of gb.xml, for
example, I would put a few hundreds of XML/HTML files to test the parser),
it would end up wasting a noticeable amount of the user's disk space.

> Moreover, we can make "UnitTest" a new "form" type in the IDE. We could
> store the code to test in the class, that class automatically inheriting
> the UnitTest class. And the form file would be just the textual result
> that must be emitted by the tests.
>
> What do you think about that?

A test's result should not rely on its standard output IMHO. There are
too many ways this could go wrong for the test writer (an extra space
here, a little typo there...). Additionally, it would not be very
indicative about why/where the test failed (no stack trace or error
message), and I think that maintaining two files for a test would quickly
become a pain. Relying only on the Assert() methods is the best solution
to me.

For IDE integration, what I had in mind was, as you said, creating a
new "form" type, so you could simply do New>Unit Test..., give it a name
and immediately start implementing the pre-generated _Run() method.
Then, starting a test (either automatically after compilation or manually
 from a menu entry) would open a tab containing the full report.
I can work on that if you want.

In the meantime, I will start working on a class to manage and retrieve
the results from the testing process, so it can directly be used by the
IDE (and maybe other projects).

-- 
Adrien Prokopowicz