I have a Wago PFC200 SL Runtime, that freezes up on me every 2-3 weeks. All it needs is a reboot and it runs again. But after the reboot, the log is "empty", I can only see the boot messages, not any messages from what made it halt.

Is there a way to get a post-mortem log on freezing PLC?

It looses network connection, so there is no OPC-UA connection, no webinterface, no shell access and cant even ping it.

@MadsKaizer,

The PFC200 is nothing more than a device that's running on Linux operating system (OS). This sounds like a kernel panic that's stopping everything. I would recommend you check, if exists, the kern.log file or other files that's OS related. Try to SSH in and look under the /var/log/ path.

All the log files there are only with data from after the latest reboot.

There is a logrotate.status file that contains this:
logrotate state -- version 2
"/var/log/sudo.log" 2021-12-21-14:0:0
"/var/log/runtime.log" 2021-12-21-14:0:0
"/home/log/wagolog.log" 2021-12-21-14:0:0
"/var/log/openvpn.log" 2021-12-21-14:0:0
"/var/log/syslog.log" 2022-1-2-18:1:0
"/var/log/lighttpd/error.log" 2021-12-21-14:0:0
"/var/log/kernel.log" 2021-12-21-14:19:0
"/var/log/lighttpd/access.log" 2021-12-21-14:0:0
"/var/log/earlymsg" 2021-12-21-14:19:0
"/var/log/cron.log" 2022-1-4-7:35:0
"/log/fwupdate.log" 2021-12-21-14:0:0
"/var/wagoearlylog.log" 2021-12-21-14:0:0
"/var/log/messages" 2021-12-21-14:0:0
"/var/log/wago/wagolog.log" 2021-12-21-14:0:0

which seems to suggest that all logs was rotated on the hour of reboot. There is a ".1" of the older logs, but that is just the reboot information.

This does not seem to be the right place for a continues log, that would have information about a crash :(

Hi,
the plclogger is saved on /tmp/codesyscontrol.log
so in case of any runtime crash you should see this in this log.
(attention this is temp file after reboot gone)

In case you have reproducible errors I would recommend to start the PFC manually (via ssh connection) by:
1. login by ssh
2. stop the pfc:
/etc/init.d/codesyscontrol stop
3. start it manually:
LD_LIBRARY_PATH=/opt/codesys/lib/compat LD_PRELOAD=/lib/libwretain.so
/opt/codesys/bin/codesyscontrol.bin -d /etc/CODESYSControl.cfg

open a second ssh client and
tail -f /tmp/codesyscontrol.log

or change:
nano /etc/default/codesyscontrol

(remove '#'DEBUGOUTPUT=1
and '#'DEBUGLOGFILE=/tmp/codesyscontrol_debug.log

@eschwellinger,

Wouldn't this be more than a runtime crash because the device can't be pinged or accessed via SSH. This seems like an OS layer issue.

@MadsKaizer,

If you SSH in and run this command ls -la /var/log/, do you see log files that have been gzipped? If yes, then try zcat the files (e.g. zcat /var/log/kernel.*)... though I'm not sure if zcat is installed. If not, then move the file to your computer using WinSCP or something similar and try to read the file.

maybe, but never seen so far, which Wago firmware version is it? Which CODESYS runtime version?

What environment is the PLC mounted in? Do you use Wagos power filter card? 750-624 og 750-626 for instance.

I experience the same phenomenon with the CODESYS Control for BeagleBone SL.
After rebooting the controller, the softPLC works again.
There is no fixed interval but it is definitely longer than hours, rather about 3 weeks.
The runtime is version 3.5.18.20 but certainly I also had this problem with the two previous versions.

Has any cause been found for this issue?

A frequent entry in the kern.log is:
"VFS: file-max limit 43812 reached"

on BeagleBone for sure solution is to update to 4.7.0.0 version, for this memory leak problem.

Hello eschwellinger,

Thank you for the advice!

The firmware was indeed 4.6.0.0 . I ran updates. I will check logs on regular basis for similar entries after the update.