[Gambas-user] Questions about gb.test and gb.test.tap

[Tobias Boege]

On Tue, 25 Feb 2020, Christof Thalhofer wrote:
> The real reason I did that in that way you don't like, was:
> 
> With the current gb.test I can step through the test code and the tested
> code with the debugger *because the processes are not separated*! This
> is very convenient for daily work as a programmer! You step through and
> see: "Oh, my testcode was buggy" or "Oh, my tested code does a silly thing".
> 

Thanks for explaining. I'll address it below.

> You complain about that other test systems you know also do not follow
> your idea of "pureness" and "semantically correct" separation.
> 
> I bet, their programmers faced the same problem and also came to the
> same decision as I did here.

I don't think I complained about other test systems. I use the `prove`
system pretty much daily in one language or another, even for verifying
SQL databases, and it works as in the flowchart.

On Tue, 25 Feb 2020, Benoît Minisini wrote:
> Le 25/02/2020 à 08:30, Christof Thalhofer a écrit :
> > I do not want to run the "full project in test mode" while I am
> > programming. I have projects that need two minutes to test. So I just
> > want to test one single testmethod while I work.
> 
> So, in test mode, the IDE must provide an interface to choose which test we
> want to run.
> 
> Can we assume that a "test" is a full module, or do we have to be able to
> choose one method in the test module?
> 

The latter, and gb.test currently does this if you give it the right
arguments. You can select a test module and a test method and it will
Object.Call() just this method. It seems like a valuable feature to me.

But separating tester and testee into two processes does not prevent
running, and hence debugging, only isolated test methods! Here is how
it goes:

  - The IDE determines the "test set" to run, a collection of test modules
    with test methods inside, by some "last used" setting or from a topical
    grouping of tests derived from tags in their help comments, or... let
    your imagination run free. This may be as few as one test method in
    one test module.

  - It starts gbx3 in test mode and passes the test set.

  - gb.test's Test.Main() takes over, receives the test modules and methods
    and does its thing. The result is TAP printed on stdout.

  - Oops, a test fails or the program crashes. The IDE sees this in the
    TAP stream. Pinpointing the test method and test module where the
    crash happened is currently not but can be implemented by a diagnostic
    message when starting a test method. (In fact, it sort of already is
    in my branch because I start a subtest for each test method and give
    it a description containing test module and method.)

  - The IDE starts gbx3 in debug mode and test mode. The Gambas debugger
    is a two-party protocol as well and requires the separate process
    anyway. The only difference is that the program being debugged starts
    inside gb.test's Test.Main() which dispatches to the chosen test
    methods. TAP is still printed, but while you debug.

You see: as long as the test modules and the tested code are inside the
project, you can step through the process even as it runs tests. Debugging
code and running test methods are independent of each other, so can be
freely mixed.

Regards,
Tobias

-- 
"There's an old saying: Don't change anything... ever!" -- Mr. Monk