#42 CfUnit Logger shows count of only one Test Suite's Tests

v1.1.0.0
closed
nobody
None
2020-05-31
2020-05-20
i-campbell
No

cfUnit version 1.0.9.9
CODESYS IDE version SP16
CODESYS environment CODESYS Control Win x64 SP16
compare strings shows an example of 2 test suites each with 8 tests.
The output from the logger shows:

Failed Tests: 0
Successful Tests: 8
Tests: 8
Test suites: 2

I expected 16, like so:

Failed Tests: 0
Successful Tests: 16
Tests: 16
Test suites: 2

the .\testresults.xml shows all tests, as I expected.

Please see attached log and testresult

2 Attachments

Discussion

  • aliazzz

    aliazzz - 2020-05-21

    Hi Ian,

    Thanks for bringing this to the attention!
    This seems a bit odd since both the xunit xml writer and the logger share the same information source.
    Therefore II would like to suggest you to do the following:

    Be sure to delete any stale CfUnit libraries from the manager (if you installed the package, beware you don't accidentally delete that library.)
    Download a fresh .library from the trunk of CfUnit, it is the latest version (updated just some two weeks ago or so)
    Download the accompanying CfUnit verifier project.

    • Run the verifier project and see if the log matches the generated file.

    • Afterwards, run your own test suite, the logger output should also match the generated xUnit report xml file.

    Would you please be so kind and share the outcome of the above in this ticket?
    This would be helpful if the problem persists.

    Hoping to hear from you,

    Aliazzz

     

    Last edit: aliazzz 2020-05-21
  • i-campbell

    i-campbell - 2020-05-21

    Hi Aliazzz,
    1. Deleted 1.0.9.9 from library
    2. Downloaded and installed from https://forge.codesys.com/prj/cfunit/code/HEAD/tree/trunk/Library/CfUnit.library [R377]
    3. There were two projects in https://forge.codesys.com/prj/cfunit/code/HEAD/tree/trunk/Verifier/ one called CfUnitVerifier.project [R372] and one called Verifier.project [R375]. I ran them both.
    4. Both gave me what I can guess are the same results

    I note the log entries are:

    Failed tests: 112
    Successful tests: 139
    Tests: 251
    Test suites: 23
    

    Whereas the xml output is:
    <testsuites disabled="" failures="112" tests="139">
    In total there are 258 methods in the project, so I would expect around 251 tests, sure.</testsuites>

    When I then run my project,there was no change to my output, it still gave me 8 tests.
    I should correct, that in my original and today's testresult.xml, it did give the same result as the logger of only 8 tests.

    <testsuites disabled="" failures="0" tests="8"></testsuites>

    Failed tests: 0
    Successful tests: 8
    Tests: 8
    Test suites: 2
    

    Should it prove useful, attached are the output logs and output testresults of the two projects.

     

    Last edit: i-campbell 2020-05-21
  • aliazzz

    aliazzz - 2020-05-23

    Solution contributed by I-Campbell

    FB_TestResults

    Replaced;
    57 IF StoringTestSuiteResultNumber = GVL_CfUnit.NumberOfInitializedTestSuites AND
    58 GVL_CfUnit.TestSuiteAddresses[StoringTestSuiteResultNumber]^.AreAllTestsFinished() AND NOT
    59 StoredGeneralTestResults THEN

    With;
    ( If all test suites have finished storing their results, store the general test results )
    57 IF StoredTestSuiteResults AND NOT StoredGeneralTestResults THEN

     
  • aliazzz

    aliazzz - 2020-05-23

    This ticket seems also related to Ticket[#37]

     

    Related

    Tickets: #37

  • aliazzz

    aliazzz - 2020-05-23
    • status: open --> accepted
     
  • aliazzz

    aliazzz - 2020-05-23
    • status: accepted --> closed
     
  • i-campbell

    i-campbell - 2020-05-31

    Information Only: Error is not present in current version of TcUnit. It contains "AND StoredTestSuiteResults" in the comparison in question. That means, no need to raise an issue report with TcUnit.

     

Log in to post a comment.