#43 See Also Ticket #44 - Testsuite with more than one multi-cycle tests won't finish

v1.1.0.0
closed
aliazzz
None
2020-07-11
2020-06-02
maoravni
No

I've starting to create UT for some refactoring I'm doing in our project.

I have two UT that are multi-cycle as they need to check timed behavior, and several UT that are single cycle.

When a single one of them runs the suite finishes correctly. When both of them are running the suite never finishes.

Tried to place them at different FBs, but same result.

In the archive you'll find them both at different FBs. The two tests are TestMaxActivation and TestOneShotActivation.

Use IPC_UnitTest.

1 Attachments

Discussion

  • maoravni

    maoravni - 2020-06-02

    Forgot to mention I've downloaded the latest trunk. Using CfUnit version 1.0.9.93.

     
  • aliazzz

    aliazzz - 2020-06-02

    Hi, Thank you for this invaluable feedback.

    I am sad to hear that the timed test does not behave as expected, so we will look into it asap.
    You can continue working with the latest version, you can always skip the tests which now fail/hang and wait for the update allthough I expect it this weekend (daytime job, sorry!)
    Otherwise you can revert back to v1.0.0.0 in which timed tests are working.

    Thank you, and we hope you still enjoy using the software while we iron out this problem.

    PS
    I hope you will not mind and share your experiences and what you use the testing framework with us?
    If you do mind, you can omit any private details ;-) We like to hear from you and your experiences so we can improve the framework accordingly. Otherwise PM me, I will not divulge any private details.

    The CfUnit project team

     

    Last edit: aliazzz 2020-06-02
    • aliazzz

      aliazzz - 2020-06-02

      PSS the most recent version in the trunk is 1.0.9.94 RC4

       
  • maoravni

    maoravni - 2020-06-02

    The reason I upgraded to 1.0.9.93 in the first place was because the 2 timed tests didn't work in the first place.

    I checked again with 1.0.0.0, and I got the same behavior: If I comment one of the tests, the suite finishes. If I run the two tests the suite never finishes.

     
    • aliazzz

      aliazzz - 2020-06-02

      Did you declare the timers in the methods themselves?

       
      • maoravni

        maoravni - 2020-06-02

        The timers are declared static in the methods.

        The two methods are in separate FB.

         
        • maoravni

          maoravni - 2020-06-02

          When stepping I can see the two tests finishing correctly. They are not registered as finished by CfUnit.

           
          • aliazzz

            aliazzz - 2020-06-02

            Yes, timers in methods should be declared static (allthough personally I am not a fan of static declarations) or declare them in the main FB body (my personal preference).

            Anyway thank your the extra feedback, as it is useful searching for the issue.

             
  • maoravni

    maoravni - 2020-06-02

    As for the use-case, we have a project that have been running for a long time, and it has many FB that are duplicates of each other with minor differences. Each of them is very simple, but the system composition and relations between these FB is what raises the complexity of the system.

    I'm designing a generic FB that will support a superset of all features these FB require. I need to test all features these FB supports before I start replacing the old code with the new, so I can be sure that the code I'm using is bug-free (at least what has been tested...).

     
    • aliazzz

      aliazzz - 2020-06-02

      Beware of inheritance in this case, choose composition instead, as this leaves you the option to inherit further on.
      Once inherited you cannot inherit something else!

       
      • maoravni

        maoravni - 2020-06-02

        Thanks. It's copied from an old design of mine that was used successfully on an embedded system that I developed in this workplace a few years ago.
        I'm sad to say that I'm using Codesys because of some political decision that I can't avoid. Codesys is an overkill for this specific system.

         
  • maoravni

    maoravni - 2020-06-02

    Any workarounds?

     
  • aliazzz

    aliazzz - 2020-06-02

    Hi,

    Some feedback: I have run the v1.0.9.93 with CfUnitTimedTestExample.
    I runs like a clock and did not hang. It finishes with succes!

    My test environment :
    Win 10 x64 VM, 8GB RAM
    CODESYS IDE x64 v3.5.16.0
    Control Win x64 v3.5.16.0 runtime

    unaltered CfUnitTimedTestExample.project from the trunk;
    with CfUnit v1.0.9.93 (RC3) => Works, verified
    with CfUnit v1.0.9.94 (RC4) => Works, verified

    I can guarantee the testresults and reproduce them.

    So, this probably means the issue you experience resides in your specific setup/code or you have changed some code in the CfUnit library. I don't know so I am just guessing at the issue. Next stop : I will check your project from the attached zip.

    To be continued!

    Aliazzz

     

    Last edit: aliazzz 2020-06-03
    • maoravni

      maoravni - 2020-06-02

      I did not change anything that I know of in the CfUnit library.

      I will try to reproduce the bug with the CfUnitTimesTestExample, but it will have to wait for tomorrow. Meanwhile, if you can, duplicate the test itself, so you have the same test running twice under two different FB. This will replicate my scenario.

       
      • aliazzz

        aliazzz - 2020-06-02

        If you reproduce a bug in CfUnitTimesTestExample, It is either your runtime or your machine setup as I have just verified it’s correct workings in my vanilla (commonly used) setup. What hardware, OS and runtime do you use?

         
        • aliazzz

          aliazzz - 2020-06-03

          It seems you have tested something else then what I have tested...
          I tested single tests only while you have tested two tests after each other.

          From what I now understand the problem you have is as follows:
          As a single timed tests passes, two timed tests don't.

           
          • maoravni

            maoravni - 2020-06-03

            Yes.

            Currently I have timed tests in a single FB so I have some workaround.

            This will be a blocker soon as I add more timed tests in more FB.

            I'll need to implement some synchronization mechanism to block multiple tests running simultaneously. Or modify the library to add the test name to the TEST_FINISHED.

             
  • aliazzz

    aliazzz - 2020-06-02
    • status: open --> accepted
    • assigned_to: aliazzz
    • Milestone: testresult report --> v1.1.0.0
     
  • maoravni

    maoravni - 2020-06-03

    I successfully reproduced the bug with CfUnitTimedTestExample.

    I duplicated the FB_Timer_Tests FB. Changed the name of the test to SomeFooTest2. Add the FB to the VAR list in the main PRG.

    Using CfUnit v1.0.0.0.

    When running the tests I get this in the log file:
    <entry infoid="0" timestamp="03/06/2020 6:00:53" component="CfUnit" severity="information" user="nobody">Test finished: SomeFooTest2</entry>
    <entry infoid="0" timestamp="03/06/2020 6:00:53" component="CfUnit" severity="information" user="nobody">Test finished: SomeFooTest2</entry>
    <entry infoid="0" timestamp="03/06/2020 6:00:50" component="CfUnit" severity="information" user="nobody">Test start: SomeFooTest2</entry>
    <entry infoid="0" timestamp="03/06/2020 6:00:50" component="CfUnit" severity="information" user="nobody">Test start: SomeFooTest</entry>

    Project attached.

     
  • aliazzz

    aliazzz - 2020-06-03
    • summary: suite with more than one multi-cycle tests won't finish --> See Also Ticket #44 - Testsuite with more than one multi-cycle tests won't finish
     
  • aliazzz

    aliazzz - 2020-06-08
    • status: accepted --> pending
     
  • aliazzz

    aliazzz - 2020-06-16

    The latest RC5 in the trunk is tested and fixes the issue.

     
  • aliazzz

    aliazzz - 2020-06-22

    Hi,

    Would you please provide feedback whether the solution has helped?

    Thank you.

     
  • aliazzz

    aliazzz - 2020-07-09

    Hi,

    We integrated your suggestion (RETURN) into RC 5.1 which currently is the latest version.
    please let us us know if it helps you or not.

    If it helped we can close this Ticket.

     
  • aliazzz

    aliazzz - 2020-07-11
    • status: pending --> closed
     

Log in to post a comment.