[Gambas-devel] Suggestion : Gambas Unit Testing

Adrien Prokopowicz adrien.prokopowicz at ...176...
Wed Dec 10 10:35:38 CET 2014


Le Wed, 10 Dec 2014 08:20:24 +0100, Benoît Minisini  
<gambas at ...1...> a écrit:

> Le 10/12/2014 04:08, Adrien Prokopowicz a écrit :
>> Le Tue, 09 Dec 2014 22:45:01 +0100, Benoît Minisini
>> <gambas at ...1...> a écrit:
>>>
>>> [...]
>>>
>>> Moreover, we can make "UnitTest" a new "form" type in the IDE. We could
>>> store the code to test in the class, that class automatically  
>>> inheriting
>>> the UnitTest class. And the form file would be just the textual result
>>> that must be emitted by the tests.
>>>
>>> What do you think about that?
>>
>> A test's result should not rely on its standard output IMHO. There are
>> too many ways this could go wrong for the test writer (an extra space
>> here, a little typo there...).
>
> This is why there will be pattern matching magic. i.e. some part of the
> expected outputs will be 'jokers'.
>

Oh, I didn't catch the "pattern matching magic" part here. I thought it
was only for Tobias' idea, my bad.
Although this doesn't change much my point of view (see below).

>> Additionally, it would not be very
>> indicative about why/where the test failed (no stack trace or error
>> message), and I think that maintaining two files for a test would  
>> quickly
>> become a pain. Relying only on the Assert() methods is the best solution
>> to me.
>>
>
> This is a better argument. I can't tell what is the best, as I never use
> these kinds of things a lot.
>

I see a 'basic test case' like this :

	'If we use standard output comparison
	myObject = giveMeMyObject()
	Print myObject.value1
	Print myObject.value2	
	-- MyOtherFile.txt -- 8< --
	"expected value 1"
	"expected value 2"

      -- vs. --

	'If we use the assets() methods
	myObject = giveMeMyObject()
	Me.Assert(myObject.value1 = "expected value 1")
	Me.Assert(myObject.value2 = "expected value 2")

I don't know about you, but to me the second option looks way clearer and
easier to maintain. :-)

>> For IDE integration, what I had in mind was, as you said, creating a
>> new "form" type, so you could simply do New>Unit Test..., give it a name
>> and immediately start implementing the pre-generated _Run() method.
>
> Why an hidden method?
>

Because calling this method would run the test, but not update the
UnitTest's internal state variables, nor catch the (potential) errors
correctly. You should use the public Start() method for that.

Another (cleaner) possibility would be switching this method's
accessibility to protected (bumping Tobias' thread here [0]). ;)

>> Then, starting a test (either automatically after compilation or  
>> manually
>>   from a menu entry) would open a tab containing the full report.
>> I can work on that if you want.
>>
>> In the meantime, I will start working on a class to manage and retrieve
>> the results from the testing process, so it can directly be used by the
>> IDE (and maybe other projects).
>>
>
> OK, so the tests results should be a text file format easily transformed
> into a nice table by the IDE. Please don't use XML. :-)

I thought about using a text file only for importing/exporting test  
results,
because I would like the IDE's report to be updated in (almost) real-time.
For example, if only 30 of the 100 tests are done, it would be nice to show
them already, rather than just having a waiting cursor.

But don't worry, neither of the two will be implemented using XML. :-)
The only thing I'm wondering about the file format if it should be
human-readable or not...

-- 
Adrien Prokopowicz

[0] http://sourceforge.net/p/gambas/mailman/message/32993566/




More information about the Devel mailing list