[Gambas-user] Test Module, First try

Christof Thalhofer chrisml at deganius.de
Sun Jul 19 09:51:32 CEST 2020


Am 19.07.20 um 00:58 schrieb Brian G:

> I attach the final...hopefully acceptable style wise version!

On my machine some tests fail, I attach the TAP output. It is because
tests search for language agnostic strings which is not a really good idea:

>   # ------------- Expected -------------
>   # gbs3: Dim i as integer:Print i:inc i.0: Unknown component: gb.inotifyt
>   #
>   # ---------------- Got ---------------
>   # gbs3: Dim i as integer:Print i:inc i.0: Unbekannte Komponente: gb.inotifyt
>   # ------------------------------------

Also you do not test for the existence of the bad component but instead
for a string message. Apart from the language problem your test will
fail if someone decides to change the message later, for instance
'Unknown component' to 'Component not found', which is not what you want!

I did not inspect all of your tests but please ask yourself:

"Do I really test the state of my program that I want to test? Or do I
test something other which is influenced by the state that I really want
to test."

In the latter case you cannot rely on your tests!

> When I delete a test module, the default or any other test suite does
> not recalculate itself, It will crash BailOut , because the test is
> missing, It is required to use shift-F4 to enter the default plan 
> then exit it. This corrects the issue.
>
> Is it possible to fix this so that it is automatic and it detects
> that a test is missing and give opportunity to fix or bail?
>
> Its not intuitive from the message what needs to be done to fix the
> issue.

IMO this is not a bug. The test system has to bail out if a test is
called which does not exist. It says to you that sth is wrong and you
have to fix it. This is the real job the test system has to do.

One thing I saw in the TAP output I personally would avoid:

> Test TbCommandLinePrograms:ExecuteACommandLineProgram
>   # Application Output
>   # We say Hello World
>   # 0
>   # 1
>   # 2
>   # 
>   ok 1 - Check the output from the Executed command line script
>   1..1
> ok 2 - TbCommandLinePrograms:ExecuteACommandLineProgram

These notes are nice if you are debugging your program as well as the
test code. But if it is done and the test reports 'ok', get rid of it:

> Test TbCommandLinePrograms:ExecuteACommandLineProgram
>   ok 1 - Check the output from the Executed command line script
>   1..1
> ok 2 - TbCommandLinePrograms:ExecuteACommandLineProgram

Sometimes you want to read the TAP output (if you are looking for
failures) and then you will be focused on searching notes beginning with
"# " which will tell you what failed.

Then notes which tell sth obvious will annoy you:

> Test TbCommandLinePrograms:Precompile
>   # Testing PreCompile
>   ok 1 - Test precompile short forms I
>   ok 2 - Test precompile short forms F

Have a nice sunday! :-)

Alles Gute

Christof Thalhofer

-- 
Dies ist keine Signatur
-------------- next part --------------
1..10
# Initialize the component collection
#
# When Gambas ide creates a library with no vendor using make executable then an (unknown) directory is created in the users .local library
# This differs from create installation which forces a vendor name before continuing

Test TbCommandLinePrograms:CreateProgramFromCommandLineArgs
  not ok 1 -  Test For bad Component 
  #
  # ------------- Expected -------------
  # gbs3: Dim i as integer:Print i:inc i.0: Unknown component: gb.inotifyt
  #
  # ---------------- Got ---------------
  # gbs3: Dim i as integer:Print i:inc i.0: Unbekannte Komponente: gb.inotifyt
  # ------------------------------------
  #
  # Compare size a=6, b=6
  # 0001 = "Public.Sub.Main()"                    | "Public.Sub.Main()"                   
  # 0002 = "Dim.i.As.Integer"                     | "Dim.i.As.Integer"                    
  # 0003 = "i=9"                                  | "i=9"                                 
  # 0004 = "Print.i"                              | "Print.i"                             
  # 0005 = "inc.i"                                | "inc.i"                               
  # 0006 = "End"                                  | "End"                                 
  ok 2 - Verify program is formatted Correctly from command Line
  1..2
not ok 1 - TbCommandLinePrograms:CreateProgramFromCommandLineArgs

Test TbCommandLinePrograms:ExecuteACommandLineProgram
  # Application Output
  # We say Hello World
  # 0
  # 1
  # 2
  # 
  ok 1 - Check the output from the Executed command line script
  1..1
ok 2 - TbCommandLinePrograms:ExecuteACommandLineProgram

Test TbCommandLinePrograms:Precompile
  # Testing PreCompile
  ok 1 - Test precompile short forms I
  ok 2 - Test precompile short forms F
  ok 3 - Test precompile short forms L
  not ok 4 - Test for error
  #
  # ------------- Expected -------------
  # gbs3: {l}Acar=45.1: ReDefinition of variable Dim Acar As Long From Dim Acar As Integer Not Allowed
  #
  # ---------------- Got ---------------
  # gbs3: {l}Acar=45.1: Neudefinition der Variablen Dim Acar As Long Von Dim Acar As Integer Nicht erlaubt
  # ------------------------------------
  #
  1..4
not ok 3 - TbCommandLinePrograms:Precompile

Test TcMMainFunctions:CheckLibraryVersions
  ok 1 -  Veryify we can get the correct library n.n.x
  ok 2 -  Veryify we can get the correct library n.x.x
  ok 3 -  Veryify we can get the correct library n.n.n
  ok 4 -  Veryify we can get the correct library n.x.x
  ok 5 -  Veryify we can get the correct library x.x.x
  ok 6 -  Veryify we can get the correct library bad directory name
  ok 7 -  Veryify we can get the correct library Bad Lib Name
  1..7
ok 4 - TcMMainFunctions:CheckLibraryVersions

Test TcMMainFunctions:CheckProcessLibrary
  # Verify Library search and versioning
  #
  not ok 1 - Verify that error is generated when not found
  #
  # ------------- Expected -------------
  # gbs3: ~/projects.1: Unknown Library: DummyLibName
  #
  # ---------------- Got ---------------
  # gbs3: ~/projects.1: Unbekannte Bibliothek: DummyLibName
  # ------------------------------------
  #
  ok 2 - Test for absolute path in library name
  ok 3 - Test for auto select from system library n.n.x
  ok 4 - Test for auto select from User .local library n.n.n
  ok 5 - Test for auto select from User .local library n.n.n
  ok 6 - Test for auto select from User .local library n.n.x
  ok 7 - Test for auto select from User .local library n.x.x
  ok 8 - Test for auto select from User .local unknown library x.x.x
  1..8
not ok 5 - TcMMainFunctions:CheckProcessLibrary

Test TcMMainFunctions:Components
  # Testing Needed Component
  ok 1 - Check if form is correctly set on init
  ok 2 - Forms need gui first
  1..2
ok 6 - TcMMainFunctions:Components

Test TcMMainFunctions:FullReadCompileCheck
  # Compare size a=32, b=32
  # 0001 = "Class.Testit.Defined.{"               | "Class.Testit.Defined.{"              
  # 0002 = "...public.a.as.integer"               | "...public.a.as.integer"              
  # 0003 = "public.sub.GetA().as.integer"         | "public.sub.GetA().as.integer"        
  # 0004 = "...return.a"                          | "...return.a"                         
  # 0005 = "end"                                  | "end"                                 
  # 0006 = "}"                                    | "}"                                   
  # 0007 = "Project.file.{"                       | "Project.file.{"                      
  # 0008 = "#.Gambas.Project.File.3.0"            | "#.Gambas.Project.File.3.0"           
  # 0009 = "Title=Gambas.Script"                  | "Title=Gambas.Script"                 
  # 0010 = "Startup=MMain"                        | "Startup=MMain"                       
  # 0011 = "Component=gb.eval"                    | "Component=gb.eval"                   
  # 0012 = "Component=gb.eval.highlight"          | "Component=gb.eval.highlight"         
  # 0013 = "Component=gb.pcre"                    | "Component=gb.pcre"                   
  # 0014 = "}"                                    | "}"                                   
  # 0015 = "'.Gambas.module.file"                 | "'.Gambas.module.file"                
  # 0016 = "Public.x.as.string.=.\"hello\""       | "Public.x.as.string.=.\"hello\""      
  # 0017 = "public.struct.testStruct"             | "public.struct.testStruct"            
  # 0018 = "....H.as.string."                     | "....H.as.string."                    
  # 0019 = "....g.as.integer"                     | "....g.as.integer"                    
  # 0020 = "....f.as.variant"                     | "....f.as.variant"                    
  # 0021 = "end.struct"                           | "end.struct"                          
  # 0022 = "public.sub.WriteSomething()"          | "public.sub.WriteSomething()"         
  # 0023 = "..print.\"Write.Something\""          | "..print.\"Write.Something\""         
  # 0024 = "end"                                  | "end"                                 
  # 0025 = "public.sub.MoreFunctions().as.string" | "public.sub.MoreFunctions().as.string"
  # 0026 = "print.\"More.stuff.as.string\""       | "print.\"More.stuff.as.string\""      
  # 0027 = "end"                                  | "end"                                 
  # 0028 = "Public.Sub.Main()"                    | "Public.Sub.Main()"                   
  # 0029 = "for.i.as.integer.=.0.to.20"           | "for.i.as.integer.=.0.to.20"          
  # 0030 = "...print.x;;i"                        | "...print.x;;i"                       
  # 0031 = "next"                                 | "next"                                
  # 0032 = "End"                                  | "End"                                 
  ok 1 - Test a Basic App with all supported types
  1..1
ok 7 - TcMMainFunctions:FullReadCompileCheck

Test TcMMainFunctions:ValidFileRebuildCheck
  # Testing If rebuild Of script required check
  ok 1 - Check if rebuild required . not needed
  ok 2 - Check if rebuild required  . Needed
  1..2
ok 8 - TcMMainFunctions:ValidFileRebuildCheck

Test TdMServerPages:RunWebPageGeneratedProgram
  # Run the full web page generation and check we have valid output from the generated program
  # Compare size a=12, b=12
  # 0001 = "Content-type:.text/html;charset=utf-8 | "Content-type:.text/html;charset=utf-8
  # 0002 = "Content-Length:.146\r"                | "Content-Length:.146\r"               
  # 0003 = "\r"                                   | "\r"                                  
  # 0004 = ".#!/bin/env./usr/bin/gbw3"            | ".#!/bin/env./usr/bin/gbw3"           
  # 0005 = "........"                             | "........"                            
  # 0006 = "........<html>"                       | "........<html>"                      
  # 0007 = "........"                             | "........"                            
  # 0008 = "........<h1>Gambas3.Web.Server.</h1>" | "........<h1>Gambas3.Web.Server.</h1>"
  # 0009 = "........"                             | "........"                            
  # 0010 = "........A01.ON\r"                     | "........A01.ON\r"                    
  # 0011 = "........"                             | "........"                            
  # 0012 = "........</html>"                      | "........</html>"                     
  ok 1 - Verify That a correct Basic webpage is generated
  #
  1..1
ok 9 - TdMServerPages:RunWebPageGeneratedProgram

Test TdMServerPages:WebPageProgramCreation
  # Generate the script to be executed from the web page
  # Compare size a=12, b=12
  # 0001 = "USE.\"gb.web\""                       | "USE.\"gb.web\""                      
  # 0002 = "PUBLIC.SUB.Main()"                    | "PUBLIC.SUB.Main()"                   
  # 0003 = "..Response.Buffered.=.TRUE"           | "..Response.Buffered.=.TRUE"          
  # 0004 = "..Response.Begin"                     | "..Response.Begin"                    
  # 0005 = ".._PrintPage()"                       | ".._PrintPage()"                      
  # 0006 = "..Response.End"                       | "..Response.End"                      
  # 0007 = "END"                                  | "END"                                 
  # 0008 = "PRIVATE.SUB._PrintPage()"             | "PRIVATE.SUB._PrintPage()"            
  # 0009 = "PRINT.\".#!/bin/env./usr/bin/gbw3\\n. | "PRINT.\".#!/bin/env./usr/bin/gbw3\\n.
  # 0010 = "print.\"A01.ON\""                     | "print.\"A01.ON\""                    
  # 0011 = "PRINT.\"\\n........\\n........</html> | "PRINT.\"\\n........\\n........</html>
  # 0012 = "END"                                  | "END"                                 
  ok 1 - Compile Web Page, test for correct source code output
  1..1
ok 10 - TdMServerPages:WebPageProgramCreation

# Ausgeführt: 'TbCommandLinePrograms.CreateProgramFromCommandLineArgs;ExecuteACommandLineProgram;Precompile,TcMMainFunctions.CheckLibraryVersions;CheckProce
ssLibrary;Components;FullReadCompileCheck;ValidFileRebuildCheck,TdMServerPages.RunWebPageGeneratedProgram;WebPageProgramCreation'
#
# 3 tests failed:
# 1: TbCommandLinePrograms:CreateProgramFromCommandLineArgs --  Test For bad Component 
# 4: TbCommandLinePrograms:Precompile -- Test for error
# 1: TcMMainFunctions:CheckProcessLibrary -- Verify that error is generated when not found
#
# FAILED
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gambas-basic.org/pipermail/user/attachments/20200719/3b711ae6/attachment-0001.sig>


More information about the User mailing list