#37 testresults.xml top level failure count not accurate

v1.1.0.0
closed
aliazzz
None
2020-05-04
2020-04-30
jtebokkel
No

On running multi-cycle tests the top level failure count is not accurate.

I've attached reports. The one I induced a failure on a single cycle test. Both have a failure in a multi-cycle test

<testsuites disabled="" failures="1" tests="65">
<testsuite id="0" name="CODESYS_Control_Win_V3.UnitTest.UnitTests.utThreePhaseExt_InLimits" tests="12" failures="1">
<testcase name="T000_Default_PassAll" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utThreePhaseExt_InLimits" status="FAIL">
<failure message="stTestData.xB <> stTestData.tpd.xB" type="BOOL"/>
</testcase>
...
...
<testsuite id="11" name="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" tests="9" failures="1">
<testcase name="T001_Invalid:0" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T001_Invalid:1" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T001_Invalid:2" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T001_Invalid:3" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T001_Invalid:4" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T001_Invalid:5" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T001_Invalid:6" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
<testcase name="T002_Results" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="FAIL">
<failure message="ix002: 0 res002[ix002].ixData : expected: 1 got: 0" type="INT"/>
</testcase>
<testcase name="T003_QueryLength" classname="CODESYS_Control_Win_V3.UnitTest.UnitTests.utFB_QMachineModel" status="PASS"/>
</testsuite>
</testsuites>

total failures should be 2 in the pasted parts of the report.

2 Attachments
multi-cycle and single-cycle failure-testresults.xml
multi-cycle-failure_testresults.xml

Related

Tickets: #42

Discussion

  • aliazzz

    aliazzz - 2020-05-01

    Hi,

    Thank you for sending this issue in! I'll look into it.

    Gr, Aliazzz

     

  • aliazzz

    aliazzz - 2020-05-01
    • status: open --> pending
    • assigned_to: aliazzz
    • Milestone: xmlwriter --> v1.1.0.0
     

  • aliazzz

    aliazzz - 2020-05-04

    this problem has been resolved in the daily build

     

  • aliazzz

    aliazzz - 2020-05-04
    • status: pending --> closed
     

Log in to post a comment.