#51 Multiple timed tests, assertions and access violations

v1.2.0.0
closed
hermsen
None
2021-08-10
2021-07-15
stmos
No

HI all,
I've started looking at the framework for a project for which I wanted to use TDD techniques.

Using the public version 1.1.0.0, after some tests I started having AccessViolations on assertions or static variables that do not get initialized with expected init values...

What I've deduced and what it's not clear to me for timed tests (state-machine based):
If I have too many assertions in a test this causes an access violation in the PLC (using Codesys Control Win x86). If I have a state in the test state machine with an assert that is called repeatedly on each cycle (for example because I want to check that a boolean does not get true for some cycles) at some point I get an AccessViolation exception and the PLC stops. I've found a way around to call asserts only once in case a specific condition happens. Is that true?
For timed tests it seems from the examples that for a test suite we can have only one test case. If I use more than one test case I get strange behaviors, like tests failing for wrong reasons...
I've put static variables on each test case with the unit under test also. Always in the examples I've found that the UUT and other support variables are instead instantiated in the FB of the test suite. Which one is the correct way to do it?
* When I need to use TEST_FINISHED vs TEST_FINISHED_NAME? I get strange behaviors also here. If I name the test in only the first (init) state of the test case test machine I must then used TEST_FINISHED_NAME, but then failed assertions do not appear in the log...

Thanks in advance for the help.

Regards.

Discussion

  • hermsen

    hermsen - 2021-07-16

    First off, thank you for your interest in coUnit.

    On first glance after reading your description it seems to me you are making your tests too "complex". But in no way I can really say anything about it really before seeing your code, so maybe you could post your testproject code for us to study. Some in depth hints and tips;

    "I've found a way around to call asserts only once in case a specific condition happens." To me this seems a good and valid way to assert something since PLC Function Blocks are mostly state dependent. Testing state dependent code is therfore itself also state depending. The clou is not to assert before you are sure that the code is in the 'to be asserted state'. Therefore running asserts continuously in such situations is not recommended. usually such a test is comprised of a CASE statement which guide the FB to the proper test state and then an assert to verify it.

    Be aware that all tests need to be written in such a way that your test can be run in any order all the time and are thus not inter dependable on each other.

    usually this means some stages are present in each test: initialise the testee, bring testee in test state, assert, de-initialise test.

    Hoping to see your code posted here and with kindest regards

    Haico

     
  • hermsen

    hermsen - 2021-07-16

    Ticket moved from /prj/cfunit/questions/18/

     
  • stmos - 2021-07-17

    Hello Haico,
    thanks for the help and sorry for posting in the FAQs instead of in the tickets, I overlooked that and there was no way to edit it.

    I admit that I'm quite a newbie with Codesys even if I did already some projects, so maybe some issues are due to my misunderstandings, and indeed I'm not used to write test cases.

    I can't send you the original code, but I was able to reproduce the first issue in a project based on your examples, attached (Access violations).
    I've put multiple cases inside it - working and not working - just activate the ones you want to test at each run. I'm using Codesys 3.5.16.20 and simulating that on the soft PLC (Codesys Control Win x86).

    I will try to prepare also sample code for the other issue: usage of TEST/TEST_FINISHED/TEST_FINISHED_NAME and asserts (I hope in short time).

    Regarding my third issue (usage of one timed test case for each test suite), it seems to me that the answer is that it is possible to have multiple timed test cases for each test suite, given that the variables are not shared (I don't want the result of a test case to influence the results of the others, for example with shared units under tests).

    Thanks again for the help.

     
  • stmos - 2021-07-17

    Hi Haico,
    maybe I've found the issue, please correct me if I'm wrong.

    See CODESYS Support/POU/GetCurTaskInfo.

    Suggested change:

    **Declaration:**
    
    FUNCTION_BLOCK GetCurTaskInfo
    VAR
        pCurTask : CmpIecTask.RTS_IEC_HANDLE;
    -   pResult : POINTER TO CmpIecTask.RTS_IEC_HANDLE;
    +   pResult : CmpIecTask.RTS_IEC_RESULT;
        pCurTaskInfo : POINTER TO CmpIecTask.Task_Info2;
    END_VAR
    VAR_OUTPUT
        CurTaskInfo : CmpIecTask.Task_Info2;
    END_VAR
    
    -----
    
    **Code:**
    
    -pCurTask := CmpIecTask.IECTaskGetCurrent( pResult := pResult);
    +pCurTask := CmpIecTask.IECTaskGetCurrent( pResult := ADR(pResult));
    pCurTaskInfo := CmpIecTask.IECTaskGetInfo3( hIecTask := pCurTask, pResult := pResult);
    CurTaskInfo := pCurTaskInfo^;
    

    IECTaskGetCurrent needs a pointer to a valid RTS_IEC_RESULT, not RTS_IEC_HANDLE, and the instance of the pointer was not initialized.

    I'll hope this helps.

     
  • hermsen

    hermsen - 2021-07-21

    I'll check this when releasing v1.2.0.0

     
  • hermsen

    hermsen - 2021-08-10
    • status: --> accepted
    • assigned_to: h-hermsen
    • Milestone: --> v1.2.0.0
     
  • hermsen

    hermsen - 2021-08-10
    • status: accepted --> closed
     

Log in to post a comment.