CODESYS Forge - CODESYS Forge / Talk / Engineering 🇬🇧: Redundancy problem - primary PLC suddenly goes to simulation state

Hello,

I am using two RevPi Connect PLCs with 'CODESYS Control for Raspberry Pi MC SL' license. The PLCs work as redundant. Redundancy license is installed on both of them, and the redundancy works fine.

No and then, the problem occurs and a primary PLC switches to SIMULATION mode. On the secondary PLC an redundancy error occurs - RDCY_NOMESSAGE_ERROR and then it switches state to STANDALONE. After a few seconds, the primary PLC gets back to ACTIVE state and it resets it CODESYS runtime.

In the log I can see thet there was an error on primary PLC:

2023-03-03T18:03:56Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:03:56Z, 0x00000129, 4, 1, 0, **** ERROR: Synchronization terminated after 303ms because of SyncWaitTime
2023-03-03T18:03:56Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:03:56Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:03:56Z, 0x00000129, 1, 1, 0, No sync telegram RSRV_EndCycleReply (8) received after 300ms
2023-03-03T18:03:56Z, 0x00000129, 1, 1, 0, Last tick recv 0xdb2e030 send 0xdb2e160
2023-03-03T18:03:56Z, 0x00000129, 4, 1, 0, **** ERROR: Timeout: No sync message RSRV_EndCycleReply (8) from standby: Standalone
2023-03-03T18:03:56Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:03:56Z, 0x00000129, 4, 1, 0, **** ERROR: RedundancySynchronizeData called state not allowed
2023-03-03T18:03:56Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:03:56Z, 0x00000129, 4, 1, 0, **** ERROR: RedundancySynchronizeData called state not allowed
2023-03-03T18:03:56Z, 0x00000129, 4, 33, 0, **** ERROR: Telegram received (sync): 8
2023-03-03T18:03:56Z, 0x00000129, 4, 33, 0, **** ERROR: -> MsgId: received RSRV_EndCycleReply
2023-03-03T18:03:56Z, 0x00000129, 4, 33, 0, **** ERROR:  expected 1, 3, 11 
2023-03-03T18:03:56Z, 0x00000129, 4, 33, 0, **** ERROR: Received RSRV_EndCycleReply in standalone.
2023-03-03T18:03:56Z, 0x00000129, 1, 0, 0, Connection still established: Go to simulation and try to synchronize

And after a few seconds:

2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, Try to synchronize
2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, RedundancySynchronize called
2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:04:06Z, 0x00000129, 4, 1, 0, **** ERROR: RedundancySynchronizeData called state not allowed
2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, RedundancySynchronizeData called
2023-03-03T18:04:06Z, 0x00000129, 4, 1, 0, **** ERROR: RedundancySynchronizeData called state not allowed
2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, Delete redundant application ...
2023-03-03T18:04:06Z, 0x00001019, 1, 16842752, 0, Reset Driver
2023-03-03T18:04:06Z, 0x00001019, 1, 16973824, 0, Reset Driver
2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, Update boot project files ...
2023-03-03T18:04:06Z, 0x00000129, 1, 0, 0, Update boot project file '$PlcLogic$/Application/Application.app'
2023-03-03T18:04:07Z, 0x00000129, 1, 0, 0, Update boot project file '$PlcLogic$/Application/Application.crc'
2023-03-03T18:04:07Z, 0x00000129, 1, 0, 0, Update boot project file '$PlcLogic$/./Archive.prj'
2023-03-03T18:04:09Z, 0x00000129, 1, 0, 0, Update boot project files finished
2023-03-03T18:04:09Z, 0x00000129, 1, 0, 0, StartRedundantApplication
2023-03-03T18:04:11Z, 0x00008000, 1, 1, 1, IoDrvUpdateConfiguration Success!

I can not get log from secondary PLC in the time the error occurd.

In every cycle I call RedundancySynchronizeData() on the begining and the end of the cycle.

Begining of the cycle:

GetRedundancyState(ADR(sReduState));
bConnectionState := GetConnectionState();
sPlcIdent := GetPlcIdent();

IF bAutoSync THEN
    IF sReduState.eRedundancyState = RDCY.STATE.RS_CYCLE_STANDALONE AND bConnectionState AND xResSync = FALSE AND sPlcIdent = RDCY.PLC_IDENT.PLC_ID_2 THEN
            // Call of Synchronize from redundant application. Note, actual synchronisation is executed from background loop with some delay
            xResSync := RDCY.Synchronize();
    END_IF

    IF sPlcIdent = RDCY.PLC_IDENT.PLC_ID_1 AND sReduState.eRedundancyState = RDCY.STATE.RS_CYCLE_STANDBY THEN
        RDCY.SwitchToActive();
    END_IF
END_IF

IF sPlcIdent = RDCY.PLC_IDENT.PLC_ID_1 THEN
    RedundancySynchronizeData();
END_IF

End of the cycle:

IF sPlcIdent = RDCY.PLC_IDENT.PLC_ID_1 THEN
    RedundancySynchronizeData();
END_IF

In redundancy settings, I have configured the Timeout = 1000ms and Synchronization timeout = 3000ms

I have put the log from primary PLC in the attachment, in the log there can be seen the moment the problem occurs.

Is there something wrong in the configuration? How can I stop this from happening?

codesyscontrol_1.log

Forums

Help

Redundancy problem - primary PLC suddenly goes to simulation state

Forums

Help

Redundancy problem - primary PLC suddenly goes to simulation state document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Redundancy problem - primary PLC suddenly goes to simulation state