[Gambas-user] Test Module, First try

[Brian G]

Hi,

Been busy the last few days so a long delay,

I posted a Gambas bug report regarding case sensitivity before I sent the email from which you fixed the issue. Sorry about any confusion.

So attached is a project with the ideas from bellow, removed any reference in tests to text output from the compiler/interpreter error messages.

Internationalized the error checks for the assert.error parts from the scripter(this works well).

Removed all extra notes, only print output from Compare when the test result fails.

It makes for a much more tidy output for sure.

Perhaps now this is a fair to middling use of the test harness.

Still learning of course!!!

Thank You
Brian G

----- Original Message -----
From: "Christof Thalhofer" <chrisml at deganius.de>
To: "Gambas mailing list" <user at lists.gambas-basic.org>
Sent: Tuesday, July 21, 2020 9:38:14 AM
Subject: Re: [Gambas-user] Test Module, First try

Am 20.07.20 um 14:40 schrieb Brian G:

> Thanks, That was a great explanation of what happens when you remove
> a test module, Maybe that could be in the documentation?

Maybe it lacks an explanation what gb.test really does:

It runs the tests (Assert.*) which can access all the public available
interfaces of the project at runtime *from inside the project*.

The tests are organized as little programs (testmethods) which again can
be organized in testmodules and testsuites at the end.

A testsuite is nothing other that a string which contains the names of
the testmodules and testmethods to be executed. gb.test takes this
string and executes all tests which are defined by it. By default (if
the string is empty) gb.test executes all testmodules and the included
testmethods it finds in the project. The execution order is always
alphabetically by the names of the testmodules and testmethods.

So it is natural, that a deleted (or nonexistent) testmethod or
testmodule which is called by the testsuite crashes the system.

> Ok so your previous email talks about not monitoring string returned
> when testing for errors. I am retrofitting the tests to a system
> which only returns string as errors, or raises and error when they
> happen.
> 
> If I want to test that the internal modules are returning the correct
> error in different situations, what would/should be the correct model
> to use when using the test harness to test this App. And I would like
> to emphasize check that the correct error is returned.
>
> Should the modules return an error code, then print the actual text
> thru another function in order to separate the module functionality
> from the displayed text? This would imply that it's difficult to
> throw errors anymore as error.raise does not seem to allow a code to
> be passed. I don't really understand how to re-implement error
> handling such that it will fit nicely with the test suite.

As long as you control the errors inside the tested project in this way:

Error.Raise("My error description")

... IMO* it is no problem, it is fully reliable. You can test with:

' create the error situation and then test it:
Assert.Error("My error description")

* I said IMO because I am unsure when the translation of errors happen
if the project is internationalized and has translated error strings. We
would have to ask Benoît for that or try it out. In that case you had to
test with:

Assert.Error(("My error description"))

But if you rely on error message strings of the interpreter or any other
component or program, as you did in the case I mentioned**, you have to
be aware that it could have been altered or translated.

** You tried to test the non-existence of a component by testing for an
error string of the interpreter, as I remember. But in reality *you
tested the error string of the interpreter* and not *whether or not the
component existed*.

> I had a great Sunday, I was off hiking in the mountains, did a 16
> mile section of the high Sierra trail, so cool the whole thing is
> above 10,000 feet(3048 meters).

Wow, cool, that sounds beautiful! :-)

Alles Gute

Christof Thalhofer

-- 
Dies ist keine Signatur



----[ http://gambaswiki.org/wiki/doc/netiquette ]----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gbs3-3.15.90.tar.gz
Type: application/x-compressed-tar
Size: 39755 bytes
Desc: not available
URL: <https://lists.gambas-basic.org/pipermail/user/attachments/20200723/60ca23dc/attachment-0001.bin>