[Gambas-user] Test Module, First try
Christof Thalhofer
chrisml at deganius.de
Sun Jul 19 09:51:32 CEST 2020
Am 19.07.20 um 00:58 schrieb Brian G:
> I attach the final...hopefully acceptable style wise version!
On my machine some tests fail, I attach the TAP output. It is because
tests search for language agnostic strings which is not a really good idea:
> # ------------- Expected -------------
> # gbs3: Dim i as integer:Print i:inc i.0: Unknown component: gb.inotifyt
> #
> # ---------------- Got ---------------
> # gbs3: Dim i as integer:Print i:inc i.0: Unbekannte Komponente: gb.inotifyt
> # ------------------------------------
Also you do not test for the existence of the bad component but instead
for a string message. Apart from the language problem your test will
fail if someone decides to change the message later, for instance
'Unknown component' to 'Component not found', which is not what you want!
I did not inspect all of your tests but please ask yourself:
"Do I really test the state of my program that I want to test? Or do I
test something other which is influenced by the state that I really want
to test."
In the latter case you cannot rely on your tests!
> When I delete a test module, the default or any other test suite does
> not recalculate itself, It will crash BailOut , because the test is
> missing, It is required to use shift-F4 to enter the default plan
> then exit it. This corrects the issue.
>
> Is it possible to fix this so that it is automatic and it detects
> that a test is missing and give opportunity to fix or bail?
>
> Its not intuitive from the message what needs to be done to fix the
> issue.
IMO this is not a bug. The test system has to bail out if a test is
called which does not exist. It says to you that sth is wrong and you
have to fix it. This is the real job the test system has to do.
One thing I saw in the TAP output I personally would avoid:
> Test TbCommandLinePrograms:ExecuteACommandLineProgram
> # Application Output
> # We say Hello World
> # 0
> # 1
> # 2
> #
> ok 1 - Check the output from the Executed command line script
> 1..1
> ok 2 - TbCommandLinePrograms:ExecuteACommandLineProgram
These notes are nice if you are debugging your program as well as the
test code. But if it is done and the test reports 'ok', get rid of it:
> Test TbCommandLinePrograms:ExecuteACommandLineProgram
> ok 1 - Check the output from the Executed command line script
> 1..1
> ok 2 - TbCommandLinePrograms:ExecuteACommandLineProgram
Sometimes you want to read the TAP output (if you are looking for
failures) and then you will be focused on searching notes beginning with
"# " which will tell you what failed.
Then notes which tell sth obvious will annoy you:
> Test TbCommandLinePrograms:Precompile
> # Testing PreCompile
> ok 1 - Test precompile short forms I
> ok 2 - Test precompile short forms F
Have a nice sunday! :-)
Alles Gute
Christof Thalhofer
--
Dies ist keine Signatur
-------------- next part --------------
1..10
# Initialize the component collection
#
# When Gambas ide creates a library with no vendor using make executable then an (unknown) directory is created in the users .local library
# This differs from create installation which forces a vendor name before continuing
Test TbCommandLinePrograms:CreateProgramFromCommandLineArgs
not ok 1 - Test For bad Component
#
# ------------- Expected -------------
# gbs3: Dim i as integer:Print i:inc i.0: Unknown component: gb.inotifyt
#
# ---------------- Got ---------------
# gbs3: Dim i as integer:Print i:inc i.0: Unbekannte Komponente: gb.inotifyt
# ------------------------------------
#
# Compare size a=6, b=6
# 0001 = "Public.Sub.Main()" | "Public.Sub.Main()"
# 0002 = "Dim.i.As.Integer" | "Dim.i.As.Integer"
# 0003 = "i=9" | "i=9"
# 0004 = "Print.i" | "Print.i"
# 0005 = "inc.i" | "inc.i"
# 0006 = "End" | "End"
ok 2 - Verify program is formatted Correctly from command Line
1..2
not ok 1 - TbCommandLinePrograms:CreateProgramFromCommandLineArgs
Test TbCommandLinePrograms:ExecuteACommandLineProgram
# Application Output
# We say Hello World
# 0
# 1
# 2
#
ok 1 - Check the output from the Executed command line script
1..1
ok 2 - TbCommandLinePrograms:ExecuteACommandLineProgram
Test TbCommandLinePrograms:Precompile
# Testing PreCompile
ok 1 - Test precompile short forms I
ok 2 - Test precompile short forms F
ok 3 - Test precompile short forms L
not ok 4 - Test for error
#
# ------------- Expected -------------
# gbs3: {l}Acar=45.1: ReDefinition of variable Dim Acar As Long From Dim Acar As Integer Not Allowed
#
# ---------------- Got ---------------
# gbs3: {l}Acar=45.1: Neudefinition der Variablen Dim Acar As Long Von Dim Acar As Integer Nicht erlaubt
# ------------------------------------
#
1..4
not ok 3 - TbCommandLinePrograms:Precompile
Test TcMMainFunctions:CheckLibraryVersions
ok 1 - Veryify we can get the correct library n.n.x
ok 2 - Veryify we can get the correct library n.x.x
ok 3 - Veryify we can get the correct library n.n.n
ok 4 - Veryify we can get the correct library n.x.x
ok 5 - Veryify we can get the correct library x.x.x
ok 6 - Veryify we can get the correct library bad directory name
ok 7 - Veryify we can get the correct library Bad Lib Name
1..7
ok 4 - TcMMainFunctions:CheckLibraryVersions
Test TcMMainFunctions:CheckProcessLibrary
# Verify Library search and versioning
#
not ok 1 - Verify that error is generated when not found
#
# ------------- Expected -------------
# gbs3: ~/projects.1: Unknown Library: DummyLibName
#
# ---------------- Got ---------------
# gbs3: ~/projects.1: Unbekannte Bibliothek: DummyLibName
# ------------------------------------
#
ok 2 - Test for absolute path in library name
ok 3 - Test for auto select from system library n.n.x
ok 4 - Test for auto select from User .local library n.n.n
ok 5 - Test for auto select from User .local library n.n.n
ok 6 - Test for auto select from User .local library n.n.x
ok 7 - Test for auto select from User .local library n.x.x
ok 8 - Test for auto select from User .local unknown library x.x.x
1..8
not ok 5 - TcMMainFunctions:CheckProcessLibrary
Test TcMMainFunctions:Components
# Testing Needed Component
ok 1 - Check if form is correctly set on init
ok 2 - Forms need gui first
1..2
ok 6 - TcMMainFunctions:Components
Test TcMMainFunctions:FullReadCompileCheck
# Compare size a=32, b=32
# 0001 = "Class.Testit.Defined.{" | "Class.Testit.Defined.{"
# 0002 = "...public.a.as.integer" | "...public.a.as.integer"
# 0003 = "public.sub.GetA().as.integer" | "public.sub.GetA().as.integer"
# 0004 = "...return.a" | "...return.a"
# 0005 = "end" | "end"
# 0006 = "}" | "}"
# 0007 = "Project.file.{" | "Project.file.{"
# 0008 = "#.Gambas.Project.File.3.0" | "#.Gambas.Project.File.3.0"
# 0009 = "Title=Gambas.Script" | "Title=Gambas.Script"
# 0010 = "Startup=MMain" | "Startup=MMain"
# 0011 = "Component=gb.eval" | "Component=gb.eval"
# 0012 = "Component=gb.eval.highlight" | "Component=gb.eval.highlight"
# 0013 = "Component=gb.pcre" | "Component=gb.pcre"
# 0014 = "}" | "}"
# 0015 = "'.Gambas.module.file" | "'.Gambas.module.file"
# 0016 = "Public.x.as.string.=.\"hello\"" | "Public.x.as.string.=.\"hello\""
# 0017 = "public.struct.testStruct" | "public.struct.testStruct"
# 0018 = "....H.as.string." | "....H.as.string."
# 0019 = "....g.as.integer" | "....g.as.integer"
# 0020 = "....f.as.variant" | "....f.as.variant"
# 0021 = "end.struct" | "end.struct"
# 0022 = "public.sub.WriteSomething()" | "public.sub.WriteSomething()"
# 0023 = "..print.\"Write.Something\"" | "..print.\"Write.Something\""
# 0024 = "end" | "end"
# 0025 = "public.sub.MoreFunctions().as.string" | "public.sub.MoreFunctions().as.string"
# 0026 = "print.\"More.stuff.as.string\"" | "print.\"More.stuff.as.string\""
# 0027 = "end" | "end"
# 0028 = "Public.Sub.Main()" | "Public.Sub.Main()"
# 0029 = "for.i.as.integer.=.0.to.20" | "for.i.as.integer.=.0.to.20"
# 0030 = "...print.x;;i" | "...print.x;;i"
# 0031 = "next" | "next"
# 0032 = "End" | "End"
ok 1 - Test a Basic App with all supported types
1..1
ok 7 - TcMMainFunctions:FullReadCompileCheck
Test TcMMainFunctions:ValidFileRebuildCheck
# Testing If rebuild Of script required check
ok 1 - Check if rebuild required . not needed
ok 2 - Check if rebuild required . Needed
1..2
ok 8 - TcMMainFunctions:ValidFileRebuildCheck
Test TdMServerPages:RunWebPageGeneratedProgram
# Run the full web page generation and check we have valid output from the generated program
# Compare size a=12, b=12
# 0001 = "Content-type:.text/html;charset=utf-8 | "Content-type:.text/html;charset=utf-8
# 0002 = "Content-Length:.146\r" | "Content-Length:.146\r"
# 0003 = "\r" | "\r"
# 0004 = ".#!/bin/env./usr/bin/gbw3" | ".#!/bin/env./usr/bin/gbw3"
# 0005 = "........" | "........"
# 0006 = "........<html>" | "........<html>"
# 0007 = "........" | "........"
# 0008 = "........<h1>Gambas3.Web.Server.</h1>" | "........<h1>Gambas3.Web.Server.</h1>"
# 0009 = "........" | "........"
# 0010 = "........A01.ON\r" | "........A01.ON\r"
# 0011 = "........" | "........"
# 0012 = "........</html>" | "........</html>"
ok 1 - Verify That a correct Basic webpage is generated
#
1..1
ok 9 - TdMServerPages:RunWebPageGeneratedProgram
Test TdMServerPages:WebPageProgramCreation
# Generate the script to be executed from the web page
# Compare size a=12, b=12
# 0001 = "USE.\"gb.web\"" | "USE.\"gb.web\""
# 0002 = "PUBLIC.SUB.Main()" | "PUBLIC.SUB.Main()"
# 0003 = "..Response.Buffered.=.TRUE" | "..Response.Buffered.=.TRUE"
# 0004 = "..Response.Begin" | "..Response.Begin"
# 0005 = ".._PrintPage()" | ".._PrintPage()"
# 0006 = "..Response.End" | "..Response.End"
# 0007 = "END" | "END"
# 0008 = "PRIVATE.SUB._PrintPage()" | "PRIVATE.SUB._PrintPage()"
# 0009 = "PRINT.\".#!/bin/env./usr/bin/gbw3\\n. | "PRINT.\".#!/bin/env./usr/bin/gbw3\\n.
# 0010 = "print.\"A01.ON\"" | "print.\"A01.ON\""
# 0011 = "PRINT.\"\\n........\\n........</html> | "PRINT.\"\\n........\\n........</html>
# 0012 = "END" | "END"
ok 1 - Compile Web Page, test for correct source code output
1..1
ok 10 - TdMServerPages:WebPageProgramCreation
# Ausgeführt: 'TbCommandLinePrograms.CreateProgramFromCommandLineArgs;ExecuteACommandLineProgram;Precompile,TcMMainFunctions.CheckLibraryVersions;CheckProce
ssLibrary;Components;FullReadCompileCheck;ValidFileRebuildCheck,TdMServerPages.RunWebPageGeneratedProgram;WebPageProgramCreation'
#
# 3 tests failed:
# 1: TbCommandLinePrograms:CreateProgramFromCommandLineArgs -- Test For bad Component
# 4: TbCommandLinePrograms:Precompile -- Test for error
# 1: TcMMainFunctions:CheckProcessLibrary -- Verify that error is generated when not found
#
# FAILED
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.gambas-basic.org/pipermail/user/attachments/20200719/3b711ae6/attachment-0001.sig>
More information about the User
mailing list