Rpi Runtime stops after about 16 hours with an Exception with CommCycleHook

Nefus
2023-04-13
2023-04-24
  • Nefus - 2023-04-13

    Hello,

    About after 16 hours of well execution, the runtime stops.
    In the codesyscontrol.log I can see the line:

    #### Exception: *EXCEPTION* in CommCycleHook <IP>0x76db7d6c</IP> <BP>0x1000</BP> <SP>0x73f63d38</SP>
    

    And just a few seconds later, the runtime stops.
    Like Edwin Schwellinger adviced in this topic (it seems to be a similar issue?) https://forge.codesys.com/forge/talk/Deutsch/thread/c8bb956c6b/?limit=25#3484 I tried to run Codesys manually via SSH.
    But the problem is that after 2hours it stops because of "Demo mode"... And my issue is only after about 16 hours of run... So I can't see any problems within the 2 hours...

    I also tried the second option (edit the CODESYSControl.cfg and create the /etc/default/codesyscontrol to enable debug logging) but my /tmp/codesyscontrol_debug.log remains empty.

    I have no idea of why I have this CommCycleHook Exception, neither what it means...
    And I cannot have more information in the log files...

    Please, do you have somme advice for me ?

    Thanks.
    Have a nice day.

     
  • eschwellinger

    eschwellinger - 2023-04-13

    but there is no difference on license if you start manually or automatically.. so it should run more then 2h
    and which system i s it - Linux SL oder Raspberry Pi...or?

     

    Last edit: eschwellinger 2023-04-13
  • Nefus - 2023-04-14

    Dear Edwin,

    Thanks for your answer.
    I used Codesys for Raspberry SL.

    You're right, the runtime didn't stopped after 2 hours like I supposed and like indacated in the logs:

    Linux version 4.4.45-v7+ (root@bartosz-VirtualBox) (gcc version 4.8.3 20140106 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.01 - Linaro GCC 2013.11) ) #19 SMP Fri Feb 10 12:54:17 CET 2017
    PlcStartSL[752]: fork child: ppid: 17444 7824
    
    ********* CoDeSysControl DEMO VERSION - runs 2 hours*********
    
    PlcStart[205]: using /etc/CODESYSControl.cfg as config file
    
    machine: armv7l
    timer resolution: 1nsec
    
    2023-04-13T13:08:31Z, 0x0000100d, 4, 0, 0, **** ERROR: License file missing.
    2023-04-13T13:08:31Z, 0x00000126, 2, 0, 0, !!!! Warning: No valid license found for OPC UA IecVarAccess provider. Running in demo mode (2h).
    
    2023-04-13T13:08:46Z, 0x00000072, 4, 0, 2, **** ERROR: No valid license found for the webvisualization
    
    2023-04-13T13:09:01Z, 0x0000100d, 4, 0, 0, **** ERROR: Demo mode for Modbus/TCP Slave started. Will expire and stop!
    

    So effectively, Modbus, OPCUA and Webvisu stopped after 2 hours, but my application continued.πŸ˜‰

    Finally, with this "debug" mode I had much information about my problem: it tells that I have too many files openned.

    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    SysFileOpen_[171]: fopen(PlcLogic/trend/CEMS_Manager_V7.HeatedLine_Unit2_2s.1.sqlite-journal):Too many open files
    SysFileOpen_[171]: fopen(PlcLogic/trend/CEMS_Manager_V7.HeatedLine_Unit4_2s.1.sqlite-journal):Too many open files
    SysFileOpen_[171]: fopen(PlcLogic/CEMS_Manager_V7.ret):Too many open files
    SysFileOpen_[171]: fopen(PlcLogic/trend/CEMS_Manager_V7.HeatedLine_Unit1_2s.1.sqlite-journal):Too many open files
    SysFileOpen_[171]: fopen(PlcLogic/trend/CEMS_Manager_V7.HeatedLine_Unit3_2s.1.sqlite-journal):Too many open files
    SysFileOpen_[171]: fopen(PlcLogic/trend/CEMS_Manager_V7.HeatedLine_Unit5_2s.1.sqlite-journal):Too many open files
    UpdateNetworkAdapterInfo[1719]: getifaddrs:Too many open files
    SysFileOpen_[171]: fopen(/etc/FBoxManager/plcInfo/appList.txt):Too many open files
    executecommand[82]: popen(vcgencmd measure_clock arm):Too many open files
    executecommand[82]: popen(vcgencmd measure_temp):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $2+0}'):Too many open files
    executecommand[82]: popen(df / -BK | tail -n1 | awk '{print $4+0}'):Too many open files
    UpdateNetworkAdapterInfo[1719]: getifaddrs:Too many open files
    SysFileOpen_[171]: fopen(/etc/FBoxManager/plcInfo/appList.txt):Too many open files
    except_signalhandler[246]: SIGSEGV (address:(nil)) in <CommCycleHook>
    except_signalhandler[250]: address not mapped to object
    handleException[75]: getting the context...
    PlcStart[232]: Before Exit
    SysSockRecv[1103]: recv(37,0x7ef79e10,512,0x0):Transport endpoint is not connected
    SysTaskExit[561]: Main: lost cycles: 185
    SysTaskExit[561]: Modbus: lost cycles: 82147
    SysTaskExit[561]: StorageTask: lost cycles: 16
    SysTaskExit[561]: VISU_TASK: lost cycles: 7
    SysFileDelete_[274]: unlink(PlcLogic/FboxApiPlc/FboxApiPlc.ret):No such file or directory
    Socket_CanMiniDriver_Dispose[1127]: CL2_CmdUnRegister(0):10205
    

    So I supposed there 2 options :
    There is a limit on the maximum files simultaneously openned in codesys
    I reached this limit on the raspbian OS directly

    Do you know if there is a limit in the Codesys runtime on the max number of oppenned files? And if there is one, is it possible to override it ?

    Meanwhile, I will scan my code to find if there is somewhere a missing close file that could make this limit reached...

    Thank you for your help.
    Have a nice day.

     
  • Nefus - 2023-04-14

    Hi,

    I suppose I've found what's wrong, and it was in my codesys application...
    I really have a SysFileOpen with no SysFileClose that was called each minutes πŸ˜“...
    So it takes about 16 hours to reach the limit file count by process which is 1024 (command line ulimit -n to see the limit).

    I will run some tests, but I suppose it will be ok now...

    Best regards.

     
  • eschwellinger

    eschwellinger - 2023-04-14

    yes, this could really cause such problems - thanks for the feedback - should help other users
    in such situations too!

     

    Last edit: eschwellinger 2023-04-14
  • Nefus - 2023-04-24

    Hello, so after 10 days my application didn't crashed, and the codesys runtime is still alive.
    So I confirm that the problem was really caused by the openned files limit, due to the missing close I had in my code.

    Problem solved ! πŸ˜„

    Best regards.

     

Log in to post a comment.