[DEBUG] [06/25/2025 18:37:09.379] [main] [EventStream(pekko://test)] logger log1-TestEventListener started
06:37:09,379 PM [test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started
[DEBUG] [06/25/2025 18:37:09.379] [main] [EventStream(pekko://test)] logger log2-Slf4jLogger started
[DEBUG] [06/25/2025 18:37:09.379] [main] [EventStream(pekko://test)] Default Loggers started
[INFO] [06/25/2025 18:37:09.397] [main] [ArteryTransport(pekko://test)] Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.171.159:17355] with UID [7680373565780765384]
06:37:09,397 PM [test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://test@10.30.171.159:17355] with UID [7680373565780765384]
06:37:09,400 PM [test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - Starting up, Pekko version [1.0.3] ...
[INFO] [06/25/2025 18:37:09.397] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - Starting up, Pekko version [1.0.3] ...
06:37:09,403 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - Registered cluster JMX MBean [pekko:type=Cluster]
06:37:09,403 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - Started up successfully
06:37:09,405 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing
06:37:09,405 PM [test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining
[INFO] [06/25/2025 18:37:09.403] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - Registered cluster JMX MBean [pekko:type=Cluster]
[INFO] [06/25/2025 18:37:09.403] [main] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - Started up successfully
[INFO] [06/25/2025 18:37:09.404] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing
[INFO] [06/25/2025 18:37:09.404] [test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining
[WARN] [06/25/2025 18:37:09.420] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower224]: org.opendaylight.controller.cluster.common.actor.Monitor@514cdaaf
06:37:09,420 PM [test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower224] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.420] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/follower224] to Actor[pekko://test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,423 PM [test-pekko.actor.default-dispatcher-10] [INFO] RaftActor - follower224: Starting recovery with journal batch size 1000
06:37:09,423 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower224: started executor
06:37:09,423 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower224: started executor
06:37:09,424 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/follower224/TermInfo.properties does not exist
06:37:09,424 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower224: no eligible files found
06:37:09,424 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower224: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
[WARN] [06/25/2025 18:37:09.426] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader223]: org.opendaylight.controller.cluster.common.actor.Monitor@12b9f3b4
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.426] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,426 PM [test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,427 PM [test-pekko.actor.default-dispatcher-10] [INFO] RaftActor - leader223: Starting recovery with journal batch size 1000
06:37:09,427 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - leader223: started executor
06:37:09,427 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - leader223: started executor
06:37:09,428 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] PropertiesTermInfoStore - follower224: Set currentTerm=0, votedFor=null
06:37:09,428 PM [test-pekko.actor.default-dispatcher-6] [INFO] PekkoRecovery - follower224: Local TermInfo store seeded with TermInfo{term=0}
06:37:09,428 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower224: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@144e4ee2
06:37:09,428 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/TermInfo.properties does not exist
06:37:09,428 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader223: no eligible files found
06:37:09,428 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader223: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:09,430 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader223: Set currentTerm=0, votedFor=null
06:37:09,430 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader223: Local TermInfo store seeded with TermInfo{term=0}
06:37:09,430 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader223: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7a04bdd9
06:37:09,854 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
06:37:09,855 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader223 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.159:17355, Removed)
06:37:09,855 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Follower - leader223 (Follower): Received ElectionTimeout - switching to Candidate
06:37:09,855 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader223 (Candidate): Election: Candidate has following voting peers: [follower225, follower224]
06:37:09,859 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader223: Set currentTerm=1, votedFor=leader223
06:37:09,859 PM [test-pekko.actor.default-dispatcher-5] [INFO] Candidate - leader223 (Candidate): Starting new election term 1
06:37:09,859 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader223 (Candidate): Sending RequestVote{term=1, candidateId=leader223, lastLogIndex=-1, lastLogTerm=-1} to peer follower225
06:37:09,859 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] Candidate - leader223 (Candidate): Sending RequestVote{term=1, candidateId=leader223, lastLogIndex=-1, lastLogTerm=-1} to peer follower224
06:37:09,859 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActorBehavior - leader223 (Follower) :- Switching from behavior Follower to Candidate, election term: 1
06:37:09,860 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower224 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
06:37:09,860 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower224 (Follower): Cluster state: []
06:37:09,860 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower224 (Follower): Candidate in requestVote:RequestVote{term=1, candidateId=leader223, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
06:37:09,860 PM [test-pekko.actor.default-dispatcher-14] [INFO] Follower - follower224 (Follower): Term 1 in "RequestVote{term=1, candidateId=leader223, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
[WARN] [06/25/2025 18:37:09.860] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: RequestVote{term=1, candidateId=leader223, lastLogIndex=-1, lastLogTerm=-1}
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.860] [test-pekko.actor.default-dispatcher-5] [pekko://test/] Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,860 PM [test-pekko.actor.default-dispatcher-10] [INFO] LocalActorRefProvider$$anon$2 -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,864 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - follower224: Set currentTerm=1, votedFor=null
06:37:09,865 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower224 (Follower): In requestVote: RequestVote{term=1, candidateId=leader223, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1
06:37:09,866 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - follower224: Set currentTerm=1, votedFor=leader223
06:37:09,867 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower224 (Follower): requestVote returning: RequestVoteReply{term=1, voteGranted=true}
06:37:09,867 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Candidate - leader223 (Candidate): RaftRPC message received RequestVoteReply{term=1, voteGranted=true}, my term is 1
06:37:09,867 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Candidate - leader223 (Candidate): handleRequestVoteReply: RequestVoteReply{term=1, voteGranted=true}, current voteCount: 1
06:37:09,867 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Election: Leader has following peers: [follower225, follower224]
06:37:09,867 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:09,867 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:09.867] [test-pekko.actor.default-dispatcher-15] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.867] [test-pekko.actor.default-dispatcher-10] [pekko://test/] Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,867 PM [test-pekko.actor.default-dispatcher-15] [INFO] LocalActorRefProvider$$anon$2 -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,868 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower224: Last sync leader does not match current leader leader223, need to catch up to -1
[WARN] [06/25/2025 18:37:09.868] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower224] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower224, initialSyncDone=false]
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.868] [test-pekko.actor.default-dispatcher-10] [pekko://test/user/follower224] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower224] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,868 PM [test-pekko.actor.default-dispatcher-10] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower224] was unhandled. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,868 PM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActorBehavior - leader223 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1
06:37:09,868 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower224 updated: matchIndex: -1, nextIndex: 0
06:37:09,869 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:09,885 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader223: Persist data index=0 term=1 command=zero
06:37:09,885 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader223: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:09,885 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
06:37:09,885 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:09.885] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.885] [test-pekko.actor.default-dispatcher-15] [pekko://test/] Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,885 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:09,885 PM [test-pekko.actor.default-dispatcher-15] [INFO] LocalActorRefProvider$$anon$2 -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,885 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 0 is present for follower follower224
06:37:09,886 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Sending AppendEntries to follower follower224: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Number of entries to be appended = 1
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Append entry to log zero
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower224: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Log size is now 1
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower224, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:09,886 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached -1 (of expected -1)
06:37:09,886 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower224 updated: matchIndex: 0, nextIndex: 1
06:37:09,886 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader223: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
06:37:09,888 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - follower224: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader223 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader223 (Leader): Setting last applied to 0
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader223: Moving last applied index from -1 to 0
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader223: Applying state for log index 0 data zero
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader223: applyState called: zero
[WARN] [06/25/2025 18:37:09.888] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower224] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower224, initialSyncDone=true]
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.888] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower224] Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower224] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,888 PM [test-pekko.actor.default-dispatcher-14] [INFO] TestActorRef -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.base.messages.FollowerInitialSyncUpStatus] to TestActor[pekko://test/user/follower224] was unhandled. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,888 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader223: Persisting ApplyJournalEntries with index=0
[WARN] [06/25/2025 18:37:09.985] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.985] [test-pekko.actor.default-dispatcher-15] [pekko://test/] Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,985 PM [test-pekko.actor.default-dispatcher-15] [INFO] LocalActorRefProvider$$anon$2 -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [8] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader223: Persist data index=1 term=1 command=one
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader223: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:09.986] [test-pekko.actor.default-dispatcher-15] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
[INFO] [pekkoDeadLetter][06/25/2025 18:37:09.986] [test-pekko.actor.default-dispatcher-10] [pekko://test/] Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [INFO] LocalActorRefProvider$$anon$2 -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 1 is present for follower follower224
06:37:09,986 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Sending AppendEntries to follower follower224: AppendEntries{term=1, leaderId=leader223, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader223, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Number of entries to be appended = 1
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Append entry to log one
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower224: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Log size is now 2
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): Commit index set to 0
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower224, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached 0 (of expected -1)
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower224 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1
06:37:09,986 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower224 (Follower): Setting last applied to 0
06:37:09,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower224: Moving last applied index from -1 to 0
06:37:09,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower224: Applying state for log index 0 data zero
06:37:09,987 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower224: applyState called: zero
06:37:09,987 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower224: Persisting ApplyJournalEntries with index=0
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower224 updated: matchIndex: 1, nextIndex: 2
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader223: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader223 (Leader): Setting last applied to 1
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader223: Moving last applied index from 0 to 1
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader223: Applying state for log index 1 data one
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader223: applyState called: one
06:37:09,987 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader223: Persisting ApplyJournalEntries with index=1
06:37:09,988 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower224: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:10,87 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader223: Persist data index=2 term=1 command=two
06:37:10,87 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader223: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=true} 
06:37:10,87 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
06:37:10,87 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 2 is present for follower follower224
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Sending AppendEntries to follower follower224: AppendEntries{term=1, leaderId=leader223, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=true}]}
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader223: Persist data index=3 term=1 command=three
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader223: Append log entry and persist JournaledLogEntry{index=3, term=1, command=three, pending=true} 
[WARN] [06/25/2025 18:37:10.088] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader223, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=true}]}
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): Number of entries to be appended = 1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): Append entry to log two
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower224: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=true} 
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 2 is present for follower follower224
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): Log size is now 3
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): Commit index set to 1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower224, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached 1 (of expected -1)
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower224 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 2
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower224 (Follower): Setting last applied to 1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower224: Moving last applied index from 0 to 1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower224: Applying state for log index 1 data one
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower224: applyState called: one
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower224 updated: matchIndex: 2, nextIndex: 3
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 3 is present for follower follower224
06:37:10,88 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Sending AppendEntries to follower follower224: AppendEntries{term=1, leaderId=leader223, prevLogIndex=2, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=1, command=three, pending=true}]}
06:37:10,88 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower224: Persisting ApplyJournalEntries with index=1
[WARN] [06/25/2025 18:37:10.088] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader223: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader223 (Leader): Setting last applied to 2
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader223: Moving last applied index from 1 to 2
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader223: Applying state for log index 2 data two
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader223: applyState called: two
06:37:10,89 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader223: Persisting ApplyJournalEntries with index=2
[INFO] [pekkoDeadLetter][06/25/2025 18:37:10.089] [test-pekko.actor.default-dispatcher-15] [pekko://test/] Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:10,89 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower224: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:10,89 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader223, prevLogIndex=2, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=3, term=1, command=three, pending=true}]}
06:37:10,89 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): Number of entries to be appended = 1
06:37:10,89 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): After cleanup, lastIndex: 2, entries to be added from: 0
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): Append entry to log three
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower224: Append log entry and persist JournaledLogEntry{index=3, term=1, command=three, pending=true} 
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): Log size is now 4
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower224, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached 1 (of expected -1)
06:37:10,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower224 updated: matchIndex: 3, nextIndex: 4
06:37:10,90 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [INFO] LocalActorRefProvider$$anon$2 -  pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from TestActor[pekko://test/user/leader223] to Actor[pekko://test/] was unhandled. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'.
06:37:10,90 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower224: persist complete JournaledLogEntry{index=3, term=1, command=three, pending=false}
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): Commit index set to 2
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached 2 (of expected -1)
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 3
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower224 (Follower): Setting last applied to 2
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower224: Moving last applied index from 1 to 2
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower224: Applying state for log index 2 data two
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower224: applyState called: two
06:37:10,91 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower224: Persisting ApplyJournalEntries with index=2
06:37:10,91 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader223 (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2
06:37:10,91 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader223 (Leader): Setting last applied to 3
06:37:10,91 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader223: Moving last applied index from 2 to 3
06:37:10,91 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader223: Applying state for log index 3 data three
06:37:10,91 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader223: applyState called: three
06:37:10,91 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader223: Persisting ApplyJournalEntries with index=3
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): Commit index set to 3
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached 3 (of expected -1)
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower224 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower224 (Follower): Setting last applied to 3
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower224: Moving last applied index from 2 to 3
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower224: Applying state for log index 3 data three
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower224: applyState called: three
06:37:10,92 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower224: Persisting ApplyJournalEntries with index=3
06:37:10,92 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader223: persist complete JournaledLogEntry{index=3, term=1, command=three, pending=false}
06:37:10,92 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader223: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=3, lastAppliedTerm=1, lastIndex=3, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false]
06:37:10,92 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SnapshotManager - leader223: captured snapshot at lastSequenceNumber: 6
06:37:10,92 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader223: Persising snapshot at EntryInfo[index=3, term=1]/EntryInfo[index=3, term=1]
06:37:10,92 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader223: support() called
06:37:10,93 PM [leader223-%d515367] [DEBUG] RaftStorage - leader223: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/snapshot-00000000685C41D6-058CE109.tmp
06:37:10,94 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SnapshotManager - leader223: log size 4 exceeds the snapshot batch count 4 - doing snapshotPreCommit with index 3
06:37:10,94 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader223: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 3 and term: 1
06:37:10,95 PM [leader223-%d515367] [DEBUG] RaftStorage - leader223: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/snapshot-00000000685C41D6-058CE109.v1
06:37:10,95 PM [leader223-%d515367] [DEBUG] RaftStorage - leader223: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/snapshot-00000000685C41D6-058CE109.v1
06:37:10,96 PM [leader223-%d515367] [DEBUG] RaftStorage - leader223: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/TermInfo.properties
06:37:10,96 PM [leader223-%d515367] [DEBUG] RaftStorage - leader223: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=3, term=1], timestamp=2025-06-25T18:37:10.093118729Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/snapshot-00000000685C41D6-058CE109.v1, size=392, entryCompress=NONE, stateCompress=NONE}
06:37:10,96 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader223: snapshot is durable as of 2025-06-25T18:37:10.093118729Z
06:37:10,96 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SnapshotManager - leader223: Snapshot success -  sequence number: 6
06:37:10,96 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorSnapshotMessageSupport - leader223: SnapshotComplete received
06:37:10,96 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader223: handled a journal response
[WARN] [06/25/2025 18:37:10.104] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:10,188 PM [awaitility-thread] [DEBUG] RaftStorage - leader223: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/snapshot-00000000685C41D6-058CE109.v1
06:37:10,188 PM [awaitility-thread] [DEBUG] RaftStorage - leader223: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/TermInfo.properties
06:37:10,188 PM [awaitility-thread] [DEBUG] RaftStorage - leader223: picked SnapshotFileV1{lastIncluded=EntryInfo[index=3, term=1], timestamp=2025-06-25T18:37:10.093118729Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest1023663939130888594/leader223/snapshot-00000000685C41D6-058CE109.v1, size=392, entryCompress=NONE, stateCompress=NONE} as the latest file
06:37:10,225 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower224: Lagging 0 entries behind leader leader223 and reached 3 (of expected -1)
[WARN] [06/25/2025 18:37:10.225] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:10,301 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower224: stopped executor with 0 remaining tasks
06:37:10,301 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - follower224: stopped executor with 0 remaining tasks
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader223: Persist data index=4 term=1 command=four
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader223: Append log entry and persist JournaledLogEntry{index=4, term=1, command=four, pending=true} 
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: 3
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 3
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 4 is present for follower follower224
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Sending AppendEntries to follower follower224: AppendEntries{term=1, leaderId=leader223, prevLogIndex=3, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=1, command=four, pending=true}]}
[WARN] [06/25/2025 18:37:10.302] [test-pekko.actor.default-dispatcher-16] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
[WARN] [06/25/2025 18:37:10.302] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/follower224] received dead letter from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=3, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=4, term=1, command=four, pending=true}]}
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader223: Persist data index=5 term=1 command=five
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader223: Append log entry and persist JournaledLogEntry{index=5, term=1, command=five, pending=true} 
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Replicate message: identifier: MockIdentifier{value=five}, logIndex: 5, isSendImmediate: true
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower225: active: false, followerNextIndex: -1, leaderLastIndex: 5, leaderSnapShotIndex: 3
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): Checking sendAppendEntries for follower follower224: active: true, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 3
06:37:10,302 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader223 (Leader): sendAppendEntries: 4 is present for follower follower224
[WARN] [06/25/2025 18:37:10.302] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader223]: AppendEntries{term=1, leaderId=leader223, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:10,304 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader223: persist complete JournaledLogEntry{index=4, term=1, command=four, pending=false}
06:37:16,363 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower224-collector226#-236897885]
06:37:16,374 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower224]
06:37:16,375 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader223-collector227#1724264618]
06:37:16,375 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader223]
06:37:16,376 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader223: stopped executor with 0 remaining tasks
06:37:16,376 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader223: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:16.378] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/leader223] received dead letter: ExecuteInSelfMessage[runnable=org.opendaylight.controller.cluster.raft.ReplicatedLogImpl$$Lambda/0x00000007c0809c28@2dd9114c]
[WARN] [06/25/2025 18:37:16.379] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$a]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.379] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/leader223] received dead letter from TestActor[pekko://test/user/leader223]: org.opendaylight.controller.cluster.raft.behaviors.AbstractLeader$SendHeartBeat@4920ed92
[WARN] [06/25/2025 18:37:16.380] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$b]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.380] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$c]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.380] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$d]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.380] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$e]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.380] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$f]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.381] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$g]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.381] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$h]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.381] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$i]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.381] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$j]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.381] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/leader223] received dead letter from Actor[pekko://test/temp/$k]: org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState@f59c57a
[WARN] [06/25/2025 18:37:16.383] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower229]: org.opendaylight.controller.cluster.common.actor.Monitor@183af0c
06:37:16,384 PM [test-pekko.actor.default-dispatcher-14] [INFO] RaftActor - follower229: Starting recovery with journal batch size 1000
06:37:16,384 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower229: started executor
06:37:16,384 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - follower229: started executor
[WARN] [06/25/2025 18:37:16.386] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader228]: org.opendaylight.controller.cluster.common.actor.Monitor@702fa97d
06:37:16,386 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - leader228: Starting recovery with journal batch size 1000
06:37:16,386 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/follower229/TermInfo.properties does not exist
06:37:16,386 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: started executor
06:37:16,386 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: started executor
06:37:16,387 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower229: no eligible files found
06:37:16,387 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower229: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:16,387 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/TermInfo.properties does not exist
06:37:16,388 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftStorage - leader228: no eligible files found
06:37:16,388 PM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader228: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:16,390 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - follower229: Set currentTerm=0, votedFor=null
06:37:16,390 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - follower229: Local TermInfo store seeded with TermInfo{term=0}
06:37:16,390 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower229: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1f7e10cb
06:37:16,391 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - leader228: Set currentTerm=0, votedFor=null
06:37:16,391 PM [test-pekko.actor.default-dispatcher-14] [INFO] PekkoRecovery - leader228: Local TermInfo store seeded with TermInfo{term=0}
06:37:16,391 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1672aaf9
06:37:16,855 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
06:37:16,855 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader228 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.159:17355, Removed)
06:37:16,855 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader228 (Follower): Received ElectionTimeout - switching to Candidate
06:37:16,855 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader228 (Candidate): Election: Candidate has following voting peers: [follower229, follower230]
06:37:16,858 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader228: Set currentTerm=1, votedFor=leader228
06:37:16,858 PM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader228 (Candidate): Starting new election term 1
06:37:16,858 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader228 (Candidate): Sending RequestVote{term=1, candidateId=leader228, lastLogIndex=-1, lastLogTerm=-1} to peer follower229
06:37:16,858 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader228 (Candidate): Sending RequestVote{term=1, candidateId=leader228, lastLogIndex=-1, lastLogTerm=-1} to peer follower230
06:37:16,858 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader228 (Follower) :- Switching from behavior Follower to Candidate, election term: 1
06:37:16,858 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower229 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
[WARN] [06/25/2025 18:37:16.859] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: RequestVote{term=1, candidateId=leader228, lastLogIndex=-1, lastLogTerm=-1}
06:37:16,859 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower229 (Follower): Cluster state: []
06:37:16,859 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower229 (Follower): Candidate in requestVote:RequestVote{term=1, candidateId=leader228, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
06:37:16,859 PM [test-pekko.actor.default-dispatcher-14] [INFO] Follower - follower229 (Follower): Term 1 in "RequestVote{term=1, candidateId=leader228, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
06:37:16,861 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - follower229: Set currentTerm=1, votedFor=null
06:37:16,861 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower229 (Follower): In requestVote: RequestVote{term=1, candidateId=leader228, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1
06:37:16,863 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] PropertiesTermInfoStore - follower229: Set currentTerm=1, votedFor=leader228
06:37:16,863 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - follower229 (Follower): requestVote returning: RequestVoteReply{term=1, voteGranted=true}
06:37:16,864 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Candidate - leader228 (Candidate): RaftRPC message received RequestVoteReply{term=1, voteGranted=true}, my term is 1
06:37:16,864 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Candidate - leader228 (Candidate): handleRequestVoteReply: RequestVoteReply{term=1, voteGranted=true}, current voteCount: 1
06:37:16,864 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader228 (Leader): Election: Leader has following peers: [follower229, follower230]
06:37:16,864 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:16,864 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower230: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:16,864 PM [test-pekko.actor.default-dispatcher-10] [INFO] RaftActorBehavior - leader228 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1
[WARN] [06/25/2025 18:37:16.864] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:16,864 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower229: Last sync leader does not match current leader leader228, need to catch up to -1
06:37:16,864 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower229 updated: matchIndex: -1, nextIndex: 0
06:37:16,865 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:16.865] [test-pekko.actor.default-dispatcher-14] [pekko://test/user/follower229] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower229, initialSyncDone=false]
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader228: Persist data index=0 term=1 command=zero
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader228: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 0 is present for follower follower229
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): Sending AppendEntries to follower follower229: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:16,899 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower230: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:16,899 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Follower - follower229 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Follower - follower229 (Follower): Number of entries to be appended = 1
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Follower - follower229 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Follower - follower229 (Follower): Append entry to log zero
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - follower229: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Follower - follower229 (Follower): Log size is now 1
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] Follower - follower229 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower229, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:16,900 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader228 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower229 updated: matchIndex: 0, nextIndex: 1
06:37:16,900 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:16,900 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower229: Lagging 0 entries behind leader leader228 and reached -1 (of expected -1)
[WARN] [06/25/2025 18:37:16.900] [test-pekko.actor.default-dispatcher-16] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader228: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader228 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader228 (Leader): Setting last applied to 0
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader228: Moving last applied index from -1 to 0
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader228: Applying state for log index 0 data zero
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader228: applyState called: zero
06:37:16,900 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader228: Persisting ApplyJournalEntries with index=0
06:37:16,901 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower229: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
[WARN] [06/25/2025 18:37:16.901] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower229] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower229, initialSyncDone=true]
[WARN] [06/25/2025 18:37:16.975] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,0 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Persist data index=1 term=1 command=one
06:37:17,0 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:17,0 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
06:37:17,0 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:17,0 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 1 is present for follower follower229
06:37:17,1 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Sending AppendEntries to follower follower229: AppendEntries{term=1, leaderId=leader228, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:17,1 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower230: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader228, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Number of entries to be appended = 1
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Append entry to log one
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower229: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Log size is now 2
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Commit index set to 0
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower229, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower229: Lagging 0 entries behind leader leader228 and reached 0 (of expected -1)
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower229 (Follower): Setting last applied to 0
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower229: Moving last applied index from -1 to 0
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower229: Applying state for log index 0 data zero
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower229: applyState called: zero
06:37:17,1 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower229: Persisting ApplyJournalEntries with index=0
[WARN] [06/25/2025 18:37:17.002] [test-pekko.actor.default-dispatcher-15] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,2 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower229 updated: matchIndex: 1, nextIndex: 2
06:37:17,2 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower229: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:17,2 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader228 (Leader): Setting last applied to 1
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader228: Moving last applied index from 0 to 1
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Applying state for log index 1 data one
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader228: applyState called: one
06:37:17,3 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Persisting ApplyJournalEntries with index=1
[WARN] [06/25/2025 18:37:17.096] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,101 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Persist data index=2 term=1 command=two
06:37:17,101 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=true} 
06:37:17,102 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
06:37:17,102 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:17,102 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 2 is present for follower follower229
06:37:17,102 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Sending AppendEntries to follower follower229: AppendEntries{term=1, leaderId=leader228, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=true}]}
06:37:17,102 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower230: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:17.102] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Persist data index=3 term=1 command=three
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: Append log entry and persist JournaledLogEntry{index=3, term=1, command=three, pending=true} 
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 2 is present for follower follower229
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower230: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Persist data index=4 term=1 command=four
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: Append log entry and persist JournaledLogEntry{index=4, term=1, command=four, pending=true} 
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true
[WARN] [06/25/2025 18:37:17.103] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: -1
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 2 is present for follower follower229
06:37:17,103 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower230: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:17.103] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,104 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:17,104 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: persist complete JournaledLogEntry{index=3, term=1, command=three, pending=false}
06:37:17,104 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader228: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=1, lastIndex=3, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=3, mandatoryTrim=false]
06:37:17,104 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SnapshotManager - leader228: captured snapshot at lastSequenceNumber: 6
06:37:17,104 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader228: Persising snapshot at EntryInfo[index=1, term=1]/EntryInfo[index=3, term=1]
06:37:17,104 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader228: support() called
06:37:17,105 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SnapshotManager - leader228: log size 5 exceeds the snapshot batch count 4 - doing snapshotPreCommit with index 1
06:37:17,105 PM [test-pekko.actor.default-dispatcher-14] [INFO] SnapshotManager - leader228: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 1
06:37:17,105 PM [leader228-%d844295] [DEBUG] RaftStorage - leader228: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.tmp
06:37:17,105 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader228: persist complete JournaledLogEntry{index=4, term=1, command=four, pending=false}
06:37:17,108 PM [leader228-%d844295] [DEBUG] RaftStorage - leader228: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1
06:37:17,108 PM [leader228-%d844295] [DEBUG] RaftStorage - leader228: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1
06:37:17,108 PM [leader228-%d844295] [DEBUG] RaftStorage - leader228: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/TermInfo.properties
06:37:17,108 PM [leader228-%d844295] [DEBUG] RaftStorage - leader228: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=1, term=1], timestamp=2025-06-25T18:37:17.104990743Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1, size=547, entryCompress=NONE, stateCompress=NONE}
06:37:17,108 PM [test-pekko.actor.default-dispatcher-10] [INFO] SnapshotManager - leader228: snapshot is durable as of 2025-06-25T18:37:17.104990743Z
06:37:17,108 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SnapshotManager - leader228: Snapshot success -  sequence number: 6
06:37:17,108 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorSnapshotMessageSupport - leader228: SnapshotComplete received
06:37:17,108 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - leader228: handled a journal response
06:37:17,214 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 2 is present for follower follower229
06:37:17,215 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader228 (Leader): Sending AppendEntries to follower follower229: AppendEntries{term=1, leaderId=leader228, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}, JournaledLogEntry{index=4, term=1, command=four, pending=false}]}
[WARN] [06/25/2025 18:37:17.215] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,335 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 2 is present for follower follower229
06:37:17,335 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Sending AppendEntries to follower follower229: AppendEntries{term=1, leaderId=leader228, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}, JournaledLogEntry{index=4, term=1, command=four, pending=false}]}
[WARN] [06/25/2025 18:37:17.335] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,454 PM [awaitility-thread] [DEBUG] RaftStorage - leader228: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1
06:37:17,454 PM [awaitility-thread] [DEBUG] RaftStorage - leader228: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/TermInfo.properties
[WARN] [06/25/2025 18:37:17.455] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,455 PM [awaitility-thread] [DEBUG] RaftStorage - leader228: picked SnapshotFileV1{lastIncluded=EntryInfo[index=1, term=1], timestamp=2025-06-25T18:37:17.104990743Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1, size=547, entryCompress=NONE, stateCompress=NONE} as the latest file
06:37:17,575 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower229: Lagging -1 entries behind leader leader228 and reached 0 (of expected -1)
[WARN] [06/25/2025 18:37:17.575] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader228]: AppendEntries{term=1, leaderId=leader228, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:17,575 PM [test-pekko.actor.default-dispatcher-14] [WARN] AbstractLeader - leader228 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=1, success=true, followerId=follower229, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 572, lastApplied : 1, commitIndex : 1
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): sendAppendEntries: 2 is present for follower follower229
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Sending AppendEntries to follower follower229: AppendEntries{term=1, leaderId=leader228, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}, JournaledLogEntry{index=4, term=1, command=four, pending=false}]}
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader228, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}, JournaledLogEntry{index=4, term=1, command=four, pending=false}]}
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Number of entries to be appended = 3
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Append entry to log two
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower229: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=false} 
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Append entry to log three
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower229: Append log entry and persist JournaledLogEntry{index=3, term=1, command=three, pending=false} 
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Append entry to log four
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower229: Append log entry and persist JournaledLogEntry{index=4, term=1, command=four, pending=false} 
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Log size is now 5
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): Commit index set to 1
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower229, logLastIndex=4, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower229: Lagging 0 entries behind leader leader228 and reached 1 (of expected -1)
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower229 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 4
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower229 (Follower): Setting last applied to 1
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower229: Moving last applied index from 0 to 1
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower229: Applying state for log index 1 data one
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower229: applyState called: one
06:37:17,576 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower229: Persisting ApplyJournalEntries with index=1
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower229 updated: matchIndex: 4, nextIndex: 5
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 1
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader228 (Leader): Setting last applied to 2
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader228: Moving last applied index from 1 to 2
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Applying state for log index 2 data two
06:37:17,576 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader228: applyState called: two
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader228 (Leader): Setting last applied to 3
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader228: Moving last applied index from 2 to 3
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Applying state for log index 3 data three
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader228: applyState called: three
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader228 (Leader): Setting last applied to 4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader228: Moving last applied index from 3 to 4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Applying state for log index 4 data four
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader228: applyState called: four
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader228: Persisting ApplyJournalEntries with index=4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader228 (Leader): Checking sendAppendEntries for follower follower229: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 1
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower229: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower229: persist complete JournaledLogEntry{index=3, term=1, command=three, pending=false}
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower229: persist complete JournaledLogEntry{index=4, term=1, command=four, pending=false}
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower229 (Follower): Commit index set to 4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower229: Lagging 0 entries behind leader leader228 and reached 4 (of expected -1)
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower229 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 1, lastIndex(): 4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower229 (Follower): Setting last applied to 2
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower229: Moving last applied index from 1 to 2
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower229: Applying state for log index 2 data two
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower229: applyState called: two
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower229 (Follower): Setting last applied to 3
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower229: Moving last applied index from 2 to 3
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower229: Applying state for log index 3 data three
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower229: applyState called: three
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower229 (Follower): Setting last applied to 4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower229: Moving last applied index from 3 to 4
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower229: Applying state for log index 4 data four
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower229: applyState called: four
06:37:17,577 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower229: Persisting ApplyJournalEntries with index=4
06:37:17,667 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader228: stopped executor with 0 remaining tasks
06:37:17,667 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader228: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:17.669] [test-pekko.actor.default-dispatcher-16] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader228]: org.opendaylight.controller.cluster.common.actor.Monitor@4ea91777
06:37:17,670 PM [test-pekko.actor.default-dispatcher-5] [INFO] RaftActor - leader228: Starting recovery with journal batch size 1000
06:37:17,670 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader228: started executor
06:37:17,670 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader228: started executor
06:37:17,672 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader228: Set currentTerm=1, votedFor=leader228
06:37:17,673 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1
06:37:17,673 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/TermInfo.properties
06:37:17,673 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: picked SnapshotFileV1{lastIncluded=EntryInfo[index=1, term=1], timestamp=2025-06-25T18:37:17.104990743Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest7572988727538897487/leader228/snapshot-00000000685C41DD-06420817.v1, size=547, entryCompress=NONE, stateCompress=NONE} as the latest file
06:37:17,673 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader228: support() called
06:37:17,674 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader228: initializing from snapshot taken at 2025-06-25T18:37:17.104990743Z
06:37:17,674 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader228: Set currentTerm=1, votedFor=leader228
06:37:17,674 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - Recovery snapshot applied for leader228 in 104.9 μs: snapshotIndex=1, snapshotTerm=1, journal-size=3
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader228: Received ReplicatedLogEntry for recovery: index: 4, size: 4
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [WARN] AbstractReplicatedLog - leader228: Cannot append new entry - new index 4 is not greater than the last index 4
java.lang.Exception: stack trace
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.AbstractReplicatedLog.appendImpl(AbstractReplicatedLog.java:195)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.AbstractReplicatedLog.append(AbstractReplicatedLog.java:182)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRecovery.appendRecoveredEntry(PekkoRecovery.java:247)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRecovery.onRecoveredJournalLogEntry(PekkoRecovery.java:242)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.PekkoRecovery.handleRecoveryMessage(PekkoRecovery.java:129)
	at org.opendaylight.controller.cluster.raft@11.0.0-SNAPSHOT/org.opendaylight.controller.cluster.raft.RaftActor.handleRecover(RaftActor.java:207)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29)
	at scala.library@2.13.16/scala.PartialFunction.applyOrElse(PartialFunction.scala:214)
	at scala.library@2.13.16/scala.PartialFunction.applyOrElse$(PartialFunction.scala:213)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29)
	at scala.library@2.13.16/scala.PartialFunction$OrElse.apply(PartialFunction.scala:266)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$2$$anonfun$1.applyOrElse(Eventsourced.scala:651)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced$$anon$3.stateReceive(Eventsourced.scala:769)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
	at org.opendaylight.controller.repackaged.pekko@11.0.0-SNAPSHOT/org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader228: Received apply journal entries for recovery, applying to state: 2 to 4
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader228: Moving last applied index from 1 to 3
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - leader228: Recovery completed in 994.6 μs - Switching actor to Follower - last log index = 4, last log term = 1, snapshot index = 1, snapshot term = 1, journal size = 3
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - leader228: restoring local TermInfo{term=1, votedFor=leader228}
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader228: Set currentTerm=1, votedFor=leader228
06:37:17,675 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader228: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1072762e
06:37:17,676 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower229-collector231#-1470650137]
06:37:17,676 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader228 (Follower): Received TimeoutNow - switching to Candidate
06:37:17,676 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader228 (Candidate): Election: Candidate has following voting peers: []
06:37:17,679 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader228: Set currentTerm=2, votedFor=leader228
06:37:17,679 PM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader228 (Candidate): Starting new election term 2
06:37:17,679 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader228 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
06:37:17,679 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader228 (Candidate): Received ElectionTimeout
06:37:17,679 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader228 (Leader): Election: Leader has following peers: []
06:37:17,679 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader228 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
06:37:17,687 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower229]
06:37:17,687 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower229: stopped executor with 0 remaining tasks
06:37:17,687 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower229: stopped executor with 0 remaining tasks
06:37:17,687 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader228-collector232#298005852]
06:37:17,687 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader228]
06:37:17,688 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader228-collector233#170894278]
06:37:17,688 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader228]
06:37:17,688 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: stopped executor with 0 remaining tasks
06:37:17,688 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader228: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:17.694] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower235]: org.opendaylight.controller.cluster.common.actor.Monitor@761c4137
06:37:17,695 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - follower235: Starting recovery with journal batch size 1000
06:37:17,695 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower235: started executor
06:37:17,695 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower235: started executor
[WARN] [06/25/2025 18:37:17.696] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader234]: org.opendaylight.controller.cluster.common.actor.Monitor@2b8236dd
06:37:17,696 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower235/TermInfo.properties does not exist
06:37:17,697 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower235: no eligible files found
06:37:17,697 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower235: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:17,697 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - leader234: Starting recovery with journal batch size 1000
06:37:17,697 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader234: started executor
06:37:17,697 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - leader234: started executor
06:37:17,698 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/leader234/TermInfo.properties does not exist
06:37:17,698 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftStorage - leader234: no eligible files found
06:37:17,699 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader234: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:17,700 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower235: Set currentTerm=0, votedFor=null
06:37:17,700 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower235: Local TermInfo store seeded with TermInfo{term=0}
06:37:17,700 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower235: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@2f69f4ad
06:37:17,701 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] PropertiesTermInfoStore - leader234: Set currentTerm=0, votedFor=null
06:37:17,701 PM [test-pekko.actor.default-dispatcher-5] [INFO] PekkoRecovery - leader234: Local TermInfo store seeded with TermInfo{term=0}
06:37:17,701 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] RaftActor - leader234: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3f03cf63
06:37:18,155 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
06:37:18,155 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader234 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.159:17355, Removed)
06:37:18,155 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - leader234 (Follower): Received ElectionTimeout - switching to Candidate
06:37:18,155 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader234 (Candidate): Election: Candidate has following voting peers: [follower236, follower235]
06:37:18,158 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - leader234: Set currentTerm=1, votedFor=leader234
06:37:18,158 PM [test-pekko.actor.default-dispatcher-16] [INFO] Candidate - leader234 (Candidate): Starting new election term 1
06:37:18,158 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader234 (Candidate): Sending RequestVote{term=1, candidateId=leader234, lastLogIndex=-1, lastLogTerm=-1} to peer follower236
06:37:18,158 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Candidate - leader234 (Candidate): Sending RequestVote{term=1, candidateId=leader234, lastLogIndex=-1, lastLogTerm=-1} to peer follower235
[WARN] [06/25/2025 18:37:18.158] [test-pekko.actor.default-dispatcher-5] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader234]: RequestVote{term=1, candidateId=leader234, lastLogIndex=-1, lastLogTerm=-1}
06:37:18,159 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActorBehavior - leader234 (Follower) :- Switching from behavior Follower to Candidate, election term: 1
06:37:18,159 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower235 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
06:37:18,159 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower235 (Follower): Cluster state: []
06:37:18,159 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower235 (Follower): Candidate in requestVote:RequestVote{term=1, candidateId=leader234, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
06:37:18,159 PM [test-pekko.actor.default-dispatcher-10] [INFO] Follower - follower235 (Follower): Term 1 in "RequestVote{term=1, candidateId=leader234, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
06:37:18,164 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower235: Set currentTerm=1, votedFor=null
06:37:18,164 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower235 (Follower): In requestVote: RequestVote{term=1, candidateId=leader234, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower235: Set currentTerm=1, votedFor=leader234
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower235 (Follower): requestVote returning: RequestVoteReply{term=1, voteGranted=true}
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Candidate - leader234 (Candidate): RaftRPC message received RequestVoteReply{term=1, voteGranted=true}, my term is 1
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Candidate - leader234 (Candidate): handleRequestVoteReply: RequestVoteReply{term=1, voteGranted=true}, current voteCount: 1
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Election: Leader has following peers: [follower236, follower235]
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:18,167 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:18.167] [test-pekko.actor.default-dispatcher-5] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader234]: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:18,168 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower235: Last sync leader does not match current leader leader234, need to catch up to -1
[WARN] [06/25/2025 18:37:18.168] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower235] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower235, initialSyncDone=false]
06:37:18,168 PM [test-pekko.actor.default-dispatcher-10] [INFO] RaftActorBehavior - leader234 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1
06:37:18,168 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower235 updated: matchIndex: -1, nextIndex: 0
06:37:18,168 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:18,204 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - leader234: Persist data index=0 term=1 command=zero
06:37:18,205 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - leader234: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:18,205 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
06:37:18,205 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:18.205] [test-pekko.actor.default-dispatcher-15] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader234]: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:18,205 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:18,205 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): sendAppendEntries: 0 is present for follower follower235
06:37:18,205 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Sending AppendEntries to follower follower235: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Number of entries to be appended = 1
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Append entry to log zero
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower235: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Log size is now 1
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower235, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:18,205 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower235: Lagging 0 entries behind leader leader234 and reached -1 (of expected -1)
06:37:18,206 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower235 updated: matchIndex: 0, nextIndex: 1
06:37:18,206 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:18,206 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower235: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
[WARN] [06/25/2025 18:37:18.206] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower235] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower235, initialSyncDone=true]
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader234: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader234 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader234 (Leader): Setting last applied to 0
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader234: Moving last applied index from -1 to 0
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader234: Applying state for log index 0 data zero
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader234: applyState called: zero
06:37:18,207 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader234: Persisting ApplyJournalEntries with index=0
[WARN] [06/25/2025 18:37:18.285] [test-pekko.actor.default-dispatcher-16] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader234]: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:18,305 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader234: Persist data index=1 term=1 command=one
06:37:18,305 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - leader234: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:18,306 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
06:37:18,306 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:18,306 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:18,306 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): sendAppendEntries: 1 is present for follower follower235
06:37:18,306 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): Sending AppendEntries to follower follower235: AppendEntries{term=1, leaderId=leader234, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader234, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Number of entries to be appended = 1
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Append entry to log one
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower235: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Log size is now 2
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): Commit index set to 0
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower235, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower235: Lagging 0 entries behind leader leader234 and reached 0 (of expected -1)
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - follower235 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - follower235 (Follower): Setting last applied to 0
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - follower235: Moving last applied index from -1 to 0
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower235: Applying state for log index 0 data zero
06:37:18,306 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - follower235: applyState called: zero
06:37:18,306 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower235 updated: matchIndex: 1, nextIndex: 2
06:37:18,306 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:18,307 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - follower235: Persisting ApplyJournalEntries with index=0
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader234: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader234 (Leader): Setting last applied to 1
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader234: Moving last applied index from 0 to 1
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader234: Applying state for log index 1 data one
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader234: applyState called: one
[WARN] [06/25/2025 18:37:18.307] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader234]: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:18,307 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader234: Persisting ApplyJournalEntries with index=1
06:37:18,307 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower235: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
[WARN] [06/25/2025 18:37:18.405] [test-pekko.actor.default-dispatcher-16] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader234]: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:18,410 PM [test-pekko.actor.default-dispatcher-10] [INFO] RaftActor - follower236: Starting recovery with journal batch size 1000
06:37:18,410 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower236: started executor
06:37:18,410 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower236: started executor
[WARN] [06/25/2025 18:37:18.410] [test-pekko.actor.default-dispatcher-10] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower236]: org.opendaylight.controller.cluster.common.actor.Monitor@43b44891
06:37:18,410 PM [test-pekko.actor.default-dispatcher-14] [INFO] PeerInfos - leader234: Peer address for peer follower236 set to pekko://test/user/follower236
06:37:18,410 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader234: Persist data index=2 term=1 command=two
06:37:18,410 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader234: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=true} 
06:37:18,411 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
06:37:18,411 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:18,411 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:18,411 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): sendAppendEntries: 2 is present for follower follower235
06:37:18,411 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Sending AppendEntries to follower follower235: AppendEntries{term=1, leaderId=leader234, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=true}]}
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader234, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=true}]}
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): Number of entries to be appended = 1
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): Append entry to log two
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower235: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=true} 
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): Log size is now 3
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): Commit index set to 1
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower235, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower235: Lagging 0 entries behind leader leader234 and reached 1 (of expected -1)
06:37:18,411 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 2
06:37:18,412 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower235 (Follower): Setting last applied to 1
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower235 updated: matchIndex: 2, nextIndex: 3
06:37:18,412 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower235: Moving last applied index from 0 to 1
06:37:18,412 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower235: Applying state for log index 1 data one
06:37:18,412 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower235: applyState called: one
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower235: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] ReplicatedLogImpl - leader234: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader234 (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1
06:37:18,412 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/TermInfo.properties does not exist
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActorBehavior - leader234 (Leader): Setting last applied to 2
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractReplicatedLog - leader234: Moving last applied index from 1 to 2
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader234: Applying state for log index 2 data two
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [INFO] MockRaftActor - leader234: applyState called: two
06:37:18,412 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower236: no eligible files found
06:37:18,412 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower236: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:18,412 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] RaftActor - leader234: Persisting ApplyJournalEntries with index=2
06:37:18,413 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower235: Persisting ApplyJournalEntries with index=1
06:37:18,414 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] ReplicatedLogImpl - follower235: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:18,417 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=0, votedFor=null
06:37:18,417 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower236: Local TermInfo store seeded with TermInfo{term=0}
06:37:18,417 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower236: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@afdec9b
06:37:18,418 PM [test-pekko.actor.default-dispatcher-10] [INFO] Follower - follower236 (Follower): Term 1 in "AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term
06:37:18,419 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=1, votedFor=null
06:37:18,419 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower236 updated: matchIndex: -1, nextIndex: 0
06:37:18,420 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:18,420 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader234 (Leader): sendAppendEntries: 0 is present for follower follower236
06:37:18,420 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader234 (Leader): Sending AppendEntries to follower follower236: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=false}, JournaledLogEntry{index=1, term=1, command=one, pending=false}, JournaledLogEntry{index=2, term=1, command=two, pending=false}]}
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower236: Last sync leader does not match current leader leader234, need to catch up to -1
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader234, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=false}, JournaledLogEntry{index=1, term=1, command=one, pending=false}, JournaledLogEntry{index=2, term=1, command=two, pending=false}]}
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): Number of entries to be appended = 3
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): Append entry to log zero
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower236: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=false} 
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): Append entry to log one
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower236: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=false} 
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): Append entry to log two
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower236: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=false} 
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): Log size is now 3
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): Commit index set to 2
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower236, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower236: Lagging 0 entries behind leader leader234 and reached 2 (of expected -1)
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower236 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower236 (Follower): Setting last applied to 0
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower236: Moving last applied index from -1 to 0
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower236: Applying state for log index 0 data zero
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower236: applyState called: zero
06:37:18,420 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower236 updated: matchIndex: 2, nextIndex: 3
06:37:18,420 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - leader234: fakeSnapshot purging log to 1 for term 1
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower236 (Follower): Setting last applied to 1
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower236: Moving last applied index from 0 to 1
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower236: Applying state for log index 1 data one
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower236: applyState called: one
06:37:18,420 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower236 (Follower): Setting last applied to 2
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower236: Moving last applied index from 1 to 2
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower236: Applying state for log index 2 data two
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower236: applyState called: two
06:37:18,420 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower236: Persisting ApplyJournalEntries with index=2
06:37:18,421 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower236: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
06:37:18,421 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower236: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:18,421 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] ReplicatedLogImpl - follower236: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
[WARN] [06/25/2025 18:37:18.421] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower236] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower236, initialSyncDone=false]
[WARN] [06/25/2025 18:37:18.421] [test-pekko.actor.default-dispatcher-5] [pekko://test/user/follower236] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower236, initialSyncDone=true]
06:37:18,525 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower236: Lagging 0 entries behind leader leader234 and reached 2 (of expected -1)
06:37:18,525 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): Commit index set to 2
06:37:18,525 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - follower236: fakeSnapshot purging log to 1 for term 1
06:37:18,525 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower235: Lagging 0 entries behind leader leader234 and reached 2 (of expected -1)
06:37:18,525 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] Follower - follower235 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2
06:37:18,525 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - follower235 (Follower): Setting last applied to 2
06:37:18,525 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - follower235: Moving last applied index from 1 to 2
06:37:18,525 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower235: Applying state for log index 2 data two
06:37:18,526 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - follower235: applyState called: two
06:37:18,526 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower235: Persisting ApplyJournalEntries with index=2
06:37:18,526 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SnapshotManager - follower235: fakeSnapshot purging log to 1 for term 1
06:37:18,613 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower236: stopped executor with 0 remaining tasks
06:37:18,613 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower236: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:18.615] [test-pekko.actor.default-dispatcher-10] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower236]: org.opendaylight.controller.cluster.common.actor.Monitor@5fe5bef8
06:37:18,616 PM [test-pekko.actor.default-dispatcher-16] [INFO] RaftActor - follower236: Starting recovery with journal batch size 1000
06:37:18,616 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower236: started executor
06:37:18,616 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower236: started executor
06:37:18,617 PM [test-pekko.actor.default-dispatcher-10] [INFO] PeerInfos - leader234: Peer address for peer follower236 set to pekko://test/user/follower236
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=1, votedFor=null
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower236: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/TermInfo.properties
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftStorage - follower236: no eligible files found
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [INFO] PekkoRecovery - follower236: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PekkoRecovery - follower236: restoring local TermInfo{term=1}
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=1, votedFor=null
06:37:18,618 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - follower236: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@5dbdf170
06:37:18,644 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower236 (Follower): The followers log is empty and the senders prevLogIndex is 2
06:37:18,644 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower235: Lagging 0 entries behind leader leader234 and reached 2 (of expected -1)
06:37:18,644 PM [test-pekko.actor.default-dispatcher-16] [INFO] Follower - follower236 (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=1, success=false, followerId=follower236, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:18,645 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] SyncStatusTracker - follower236: Last sync leader does not match current leader leader234, need to catch up to 2
[WARN] [06/25/2025 18:37:18.645] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower236] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower236, initialSyncDone=false]
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [INFO] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=1, success=false, followerId=follower236, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: 1, snapshotTerm: 1, replicatedToAllIndex: 1
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower236 updated: matchIndex: -1, nextIndex: 0
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [INFO] AbstractLeader - leader234 (Leader): follower follower236 appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader234 (Leader): Checking sendAppendEntries for follower follower236: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: 1
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [INFO] AbstractLeader - leader234 (Leader): Initiating install snapshot to follower follower236: follower nextIndex: 0, leader snapshotIndex: 1, leader lastIndex: 2, leader log size: 1
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [INFO] SnapshotManager - leader234: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=1, lastIndex=2, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on follower236
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SnapshotManager - leader234: lastSequenceNumber prior to capture: 6
06:37:18,645 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - leader234: support() called
06:37:18,646 PM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower236 (Follower): Cannot append entries because the replicatedToAllIndex 1 does not appear to be in the in-memory journal - lastIndex: -1, snapshotIndex: -1, snapshotTerm: -1
06:37:18,646 PM [test-pekko.actor.default-dispatcher-15] [INFO] Follower - follower236 (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=1, success=false, followerId=follower236, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:18,646 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractLeader - leader234 (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=1, success=false, followerId=follower236, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: 1, snapshotTerm: 1, replicatedToAllIndex: 1
06:37:18,646 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractLeader - leader234 (Leader): follower follower236 appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader234: Persising snapshot at EntryInfo[index=2, term=1]/EntryInfo[index=2, term=1]
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader234: support() called
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader234: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 1
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader234 (Leader): sendInstallSnapshot
06:37:18,647 PM [leader234-%d265602] [DEBUG] RaftStorage - leader234: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/leader234/snapshot-00000000685C41DE-26985FF0.tmp
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] LeaderInstallSnapshotState - leader234 (Leader): Snapshot 310 bytes, total chunks to send: 1
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] LeaderInstallSnapshotState - leader234 (Leader): Next chunk: total length=310, offset=0, size=310, hashCode=-477251589
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader234 (Leader): next snapshot chunk size for follower follower236: 310
06:37:18,647 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader234 (Leader): InstallSnapshot sent to follower Vector(user, follower236), Chunk: 1/1
06:37:18,647 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower236 (Follower): handleInstallSnapshot: InstallSnapshot{term=1, leaderId=leader234, lastIncludedIndex=2, lastIncludedTerm=1, datasize=310, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}
06:37:18,647 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotTracker - follower236 (Follower): addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1
06:37:18,647 PM [test-pekko.actor.default-dispatcher-6] [INFO] Follower - follower236 (Follower): Snapshot received from leader: leader234
06:37:18,648 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - follower236: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=310}}
06:37:18,648 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower236: support() called
06:37:18,648 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower236: Converted InstallSnapshot from leader: leader234 to state
06:37:18,648 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower236: lastSequenceNumber prior to persisting applied snapshot: 0
06:37:18,648 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower236: support() called
06:37:18,649 PM [follower236-%d981632] [DEBUG] RaftStorage - follower236: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.tmp
06:37:18,650 PM [leader234-%d265602] [DEBUG] RaftStorage - leader234: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/leader234/snapshot-00000000685C41DE-26985FF0.v1
06:37:18,650 PM [leader234-%d265602] [DEBUG] RaftStorage - leader234: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/leader234/snapshot-00000000685C41DE-26985FF0.v1
06:37:18,651 PM [leader234-%d265602] [DEBUG] RaftStorage - leader234: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/leader234/TermInfo.properties
06:37:18,651 PM [leader234-%d265602] [DEBUG] RaftStorage - leader234: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=1], timestamp=2025-06-25T18:37:18.647520240Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/leader234/snapshot-00000000685C41DE-26985FF0.v1, size=374, entryCompress=NONE, stateCompress=NONE}
06:37:18,651 PM [test-pekko.actor.default-dispatcher-15] [INFO] SnapshotManager - leader234: snapshot is durable as of 2025-06-25T18:37:18.647520240Z
06:37:18,651 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SnapshotManager - leader234: Snapshot success -  sequence number: 6
06:37:18,651 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorSnapshotMessageSupport - leader234: SnapshotComplete received
06:37:18,651 PM [follower236-%d981632] [DEBUG] RaftStorage - follower236: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1
06:37:18,651 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader234: handled a journal response
06:37:18,651 PM [follower236-%d981632] [DEBUG] RaftStorage - follower236: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1
06:37:18,651 PM [follower236-%d981632] [DEBUG] RaftStorage - follower236: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/TermInfo.properties
06:37:18,651 PM [follower236-%d981632] [DEBUG] RaftStorage - follower236: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=1], timestamp=2025-06-25T18:37:18.649233134Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1, size=374, entryCompress=NONE, stateCompress=NONE}
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - follower236: snapshot is durable as of 2025-06-25T18:37:18.649233134Z
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - follower236: Snapshot success -  sequence number: 0
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower236: support() called
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower236: applySnapshot called
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower236 (Follower): handleInstallSnapshot returning: InstallSnapshotReply{term=1, followerId=follower236, chunkIndex=1, success=true}
06:37:18,652 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader234 (Leader): handleInstallSnapshotReply: InstallSnapshotReply{term=1, followerId=follower236, chunkIndex=1, success=true}
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorSnapshotMessageSupport - follower236: SnapshotComplete received
06:37:18,652 PM [test-pekko.actor.default-dispatcher-15] [INFO] AbstractLeader - leader234 (Leader): Snapshot successfully installed on follower follower236 (last chunk 1) - matchIndex set to 2, nextIndex set to 3
06:37:18,652 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower236: handled a journal response
06:37:18,765 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] SyncStatusTracker - follower236: Lagging 0 entries behind leader leader234 and reached 2 (of expected 2)
06:37:18,765 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower235: Lagging 0 entries behind leader leader234 and reached 2 (of expected -1)
[WARN] [06/25/2025 18:37:18.765] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/follower236] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower236, initialSyncDone=true]
06:37:18,768 PM [awaitility-thread] [DEBUG] RaftStorage - follower236: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1
06:37:18,768 PM [awaitility-thread] [DEBUG] RaftStorage - follower236: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/TermInfo.properties
06:37:18,768 PM [awaitility-thread] [DEBUG] RaftStorage - follower236: picked SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=1], timestamp=2025-06-25T18:37:18.649233134Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1, size=374, entryCompress=NONE, stateCompress=NONE} as the latest file
06:37:18,779 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower236: stopped executor with 0 remaining tasks
06:37:18,780 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower236: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:18.781] [test-pekko.actor.default-dispatcher-10] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower236]: org.opendaylight.controller.cluster.common.actor.Monitor@eedd6ff
06:37:18,783 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower236: Starting recovery with journal batch size 1000
06:37:18,783 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower236: started executor
06:37:18,783 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower236: started executor
06:37:18,784 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=1, votedFor=null
06:37:18,784 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower236: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1
06:37:18,785 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower236: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/TermInfo.properties
06:37:18,785 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower236: picked SnapshotFileV1{lastIncluded=EntryInfo[index=2, term=1], timestamp=2025-06-25T18:37:18.649233134Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest10506808917672749377/follower236/snapshot-00000000685C41DE-26B282EE.v1, size=374, entryCompress=NONE, stateCompress=NONE} as the latest file
06:37:18,785 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower236: support() called
06:37:18,786 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PekkoRecovery - follower236: initializing from snapshot taken at 2025-06-25T18:37:18.649233134Z
06:37:18,786 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=1, votedFor=null
06:37:18,786 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - Recovery snapshot applied for follower236 in 105.7 μs: snapshotIndex=2, snapshotTerm=1, journal-size=0
06:37:18,787 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower236: Recovery completed in 413.0 μs - Switching actor to Follower - last log index = 2, last log term = 1, snapshot index = 2, snapshot term = 1, journal size = 0
06:37:18,787 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PekkoRecovery - follower236: restoring local TermInfo{term=1}
06:37:18,787 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower236: Set currentTerm=1, votedFor=null
06:37:18,787 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower236: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@7c0daba4
06:37:18,787 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower235-collector237#-1755541749]
06:37:18,798 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower235]
06:37:18,798 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower235: stopped executor with 0 remaining tasks
06:37:18,798 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower235: stopped executor with 0 remaining tasks
06:37:18,798 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader234-collector238#-260204937]
06:37:18,798 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader234]
06:37:18,798 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader234: stopped executor with 0 remaining tasks
06:37:18,799 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader234: stopped executor with 0 remaining tasks
06:37:18,799 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower236-collector239#969819547]
06:37:18,799 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower236]
06:37:18,799 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower236-collector240#1403952098]
06:37:18,799 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower236]
06:37:18,799 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower236-collector241#-995575797]
06:37:18,799 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower236]
06:37:18,799 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower236: stopped executor with 0 remaining tasks
06:37:18,799 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower236: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:18.803] [test-pekko.actor.default-dispatcher-10] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/follower243]: org.opendaylight.controller.cluster.common.actor.Monitor@1a4ef250
06:37:18,804 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActor - follower243: Starting recovery with journal batch size 1000
06:37:18,804 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower243: started executor
06:37:18,804 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower243: started executor
[WARN] [06/25/2025 18:37:18.806] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader242]: org.opendaylight.controller.cluster.common.actor.Monitor@54171093
06:37:18,806 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/follower243/TermInfo.properties does not exist
06:37:18,807 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - follower243: no eligible files found
06:37:18,807 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower243: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:18,807 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActor - leader242: Starting recovery with journal batch size 1000
06:37:18,807 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader242: started executor
06:37:18,807 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftStorage - leader242: started executor
06:37:18,808 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/TermInfo.properties does not exist
06:37:18,808 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader242: no eligible files found
06:37:18,809 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader242: Recovery completed  - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0
06:37:18,810 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower243: Set currentTerm=0, votedFor=null
06:37:18,810 PM [test-pekko.actor.default-dispatcher-10] [INFO] PekkoRecovery - follower243: Local TermInfo store seeded with TermInfo{term=0}
06:37:18,810 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower243: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@3b82269c
06:37:18,812 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader242: Set currentTerm=0, votedFor=null
06:37:18,812 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader242: Local TermInfo store seeded with TermInfo{term=0}
06:37:18,812 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@67b8e11a
06:37:19,234 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultConfigParamsImpl - No custom RaftPolicy specified. Using DefaultRaftPolicy
06:37:19,234 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader242 (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(pekko://test@10.30.171.159:17355, Removed)
06:37:19,234 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader242 (Follower): Received ElectionTimeout - switching to Candidate
06:37:19,234 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader242 (Candidate): Election: Candidate has following voting peers: [follower244, follower243]
06:37:19,238 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader242: Set currentTerm=1, votedFor=leader242
06:37:19,238 PM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader242 (Candidate): Starting new election term 1
06:37:19,238 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader242 (Candidate): Sending RequestVote{term=1, candidateId=leader242, lastLogIndex=-1, lastLogTerm=-1} to peer follower244
06:37:19,238 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader242 (Candidate): Sending RequestVote{term=1, candidateId=leader242, lastLogIndex=-1, lastLogTerm=-1} to peer follower243
06:37:19,238 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader242 (Follower) :- Switching from behavior Follower to Candidate, election term: 1
[WARN] [06/25/2025 18:37:19.238] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: RequestVote{term=1, candidateId=leader242, lastLogIndex=-1, lastLogTerm=-1}
06:37:19,239 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower243 (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term.
06:37:19,239 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower243 (Follower): Cluster state: []
06:37:19,239 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower243 (Follower): Candidate in requestVote:RequestVote{term=1, candidateId=leader242, lastLogIndex=-1, lastLogTerm=-1} with higher term appears reachable, updating term.
06:37:19,239 PM [test-pekko.actor.default-dispatcher-10] [INFO] Follower - follower243 (Follower): Term 1 in "RequestVote{term=1, candidateId=leader242, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term
06:37:19,241 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower243: Set currentTerm=1, votedFor=null
06:37:19,241 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower243 (Follower): In requestVote: RequestVote{term=1, candidateId=leader242, lastLogIndex=-1, lastLogTerm=-1} - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1
06:37:19,243 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] PropertiesTermInfoStore - follower243: Set currentTerm=1, votedFor=leader242
06:37:19,243 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower243 (Follower): requestVote returning: RequestVoteReply{term=1, voteGranted=true}
06:37:19,243 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader242 (Candidate): RaftRPC message received RequestVoteReply{term=1, voteGranted=true}, my term is 1
06:37:19,243 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Candidate - leader242 (Candidate): handleRequestVoteReply: RequestVoteReply{term=1, voteGranted=true}, current voteCount: 1
06:37:19,243 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Election: Leader has following peers: [follower244, follower243]
06:37:19,243 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower244: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:19,244 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1
06:37:19,244 PM [test-pekko.actor.default-dispatcher-6] [INFO] RaftActorBehavior - leader242 (Candidate) :- Switching from behavior Candidate to Leader, election term: 1
[WARN] [06/25/2025 18:37:19.244] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,244 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower243: Last sync leader does not match current leader leader242, need to catch up to -1
06:37:19,244 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader242 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower243 updated: matchIndex: -1, nextIndex: 0
06:37:19,244 PM [test-pekko.actor.default-dispatcher-5] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1
[WARN] [06/25/2025 18:37:19.244] [test-pekko.actor.default-dispatcher-10] [pekko://test/user/follower243] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower243, initialSyncDone=false]
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: Persist data index=0 term=1 command=zero
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader242: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Replicate message: identifier: MockIdentifier{value=zero}, logIndex: 0, isSendImmediate: true
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower244: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 0 is present for follower follower243
06:37:19,266 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Sending AppendEntries to follower follower243: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
[WARN] [06/25/2025 18:37:19.266] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=0, term=1, command=zero, pending=true}]}
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Number of entries to be appended = 1
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): After cleanup, lastIndex: -1, entries to be added from: 0
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Append entry to log zero
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower243: Append log entry and persist JournaledLogEntry{index=0, term=1, command=zero, pending=true} 
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Log size is now 1
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower243, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:19,267 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower243: Lagging 0 entries behind leader leader242 and reached -1 (of expected -1)
06:37:19,268 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower243: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
[WARN] [06/25/2025 18:37:19.268] [test-pekko.actor.default-dispatcher-6] [pekko://test/user/follower243] unhandled message from Actor[pekko://test/deadLetters]: FollowerInitialSyncUpStatus[memberId=follower243, initialSyncDone=true]
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower243 updated: matchIndex: 0, nextIndex: 1
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader242: persist complete JournaledLogEntry{index=0, term=1, command=zero, pending=false}
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActorBehavior - leader242 (Leader): Setting last applied to 0
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader242: Moving last applied index from -1 to 0
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: Applying state for log index 0 data zero
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader242: applyState called: zero
06:37:19,268 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: Persisting ApplyJournalEntries with index=0
[WARN] [06/25/2025 18:37:19.354] [test-pekko.actor.default-dispatcher-15] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: Persist data index=1 term=1 command=one
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader242: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Replicate message: identifier: MockIdentifier{value=one}, logIndex: 1, isSendImmediate: true
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower244: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 1 is present for follower follower243
06:37:19,368 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Sending AppendEntries to follower follower243: AppendEntries{term=1, leaderId=leader242, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader242, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=1, term=1, command=one, pending=true}]}
[WARN] [06/25/2025 18:37:19.368] [test-pekko.actor.default-dispatcher-14] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Number of entries to be appended = 1
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): After cleanup, lastIndex: 0, entries to be added from: 0
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Append entry to log one
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower243: Append log entry and persist JournaledLogEntry{index=1, term=1, command=one, pending=true} 
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Log size is now 2
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Commit index set to 0
06:37:19,368 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower243, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower243: Lagging 0 entries behind leader leader242 and reached 0 (of expected -1)
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower243 (Follower): Setting last applied to 0
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower243: Moving last applied index from -1 to 0
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower243: Applying state for log index 0 data zero
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower243: applyState called: zero
06:37:19,369 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader242 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower243 updated: matchIndex: 1, nextIndex: 2
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower243: Persisting ApplyJournalEntries with index=0
06:37:19,369 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader242: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - leader242 (Leader): Setting last applied to 1
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - leader242: Moving last applied index from 0 to 1
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader242: Applying state for log index 1 data one
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader242: applyState called: one
06:37:19,369 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader242: Persisting ApplyJournalEntries with index=1
06:37:19,369 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - follower243: persist complete JournaledLogEntry{index=1, term=1, command=one, pending=false}
06:37:19,469 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader242: Persist data index=2 term=1 command=two
06:37:19,469 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader242: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=true} 
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Replicate message: identifier: MockIdentifier{value=two}, logIndex: 2, isSendImmediate: true
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower244: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Sending AppendEntries to follower follower243: AppendEntries{term=1, leaderId=leader242, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=true}]}
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - leader242: Persist data index=3 term=1 command=three
[WARN] [06/25/2025 18:37:19.470] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader242: Append log entry and persist JournaledLogEntry{index=3, term=1, command=three, pending=true} 
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Replicate message: identifier: MockIdentifier{value=three}, logIndex: 3, isSendImmediate: true
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower244: active: false, followerNextIndex: -1, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1
06:37:19,470 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
[WARN] [06/25/2025 18:37:19.470] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,471 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader242: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:19,471 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - leader242: persist complete JournaledLogEntry{index=3, term=1, command=three, pending=false}
06:37:19,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader242: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=1, lastIndex=3, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=2, mandatoryTrim=false]
06:37:19,471 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader242: captured snapshot at lastSequenceNumber: 6
06:37:19,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader242: Persising snapshot at EntryInfo[index=1, term=1]/EntryInfo[index=3, term=1]
06:37:19,471 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - leader242: support() called
06:37:19,472 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SnapshotManager - leader242: log size 4 exceeds the snapshot batch count 4 - doing snapshotPreCommit with index 1
06:37:19,472 PM [test-pekko.actor.default-dispatcher-6] [INFO] SnapshotManager - leader242: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 1
06:37:19,472 PM [leader242-%d102090] [DEBUG] RaftStorage - leader242: starting snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/snapshot-00000000685C41DF-1C270B54.tmp
06:37:19,475 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
06:37:19,475 PM [leader242-%d102090] [DEBUG] RaftStorage - leader242: finished snapshot writeout to /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/snapshot-00000000685C41DF-1C270B54.v1
[WARN] [06/25/2025 18:37:19.475] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,475 PM [leader242-%d102090] [DEBUG] RaftStorage - leader242: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/snapshot-00000000685C41DF-1C270B54.v1
06:37:19,475 PM [leader242-%d102090] [DEBUG] RaftStorage - leader242: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/TermInfo.properties
06:37:19,475 PM [leader242-%d102090] [DEBUG] RaftStorage - leader242: retaining snapshot SnapshotFileV1{lastIncluded=EntryInfo[index=1, term=1], timestamp=2025-06-25T18:37:19.472320852Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/snapshot-00000000685C41DF-1C270B54.v1, size=520, entryCompress=NONE, stateCompress=NONE}
06:37:19,595 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
06:37:19,595 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Sending AppendEntries to follower follower243: AppendEntries{term=1, leaderId=leader242, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}]}
[WARN] [06/25/2025 18:37:19.595] [test-pekko.actor.default-dispatcher-15] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,714 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
06:37:19,714 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractLeader - leader242 (Leader): Sending AppendEntries to follower follower243: AppendEntries{term=1, leaderId=leader242, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}]}
[WARN] [06/25/2025 18:37:19.714] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,722 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - leader242: Persist data index=4 term=1 command=four
06:37:19,722 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - leader242: Append log entry and persist JournaledLogEntry{index=4, term=1, command=four, pending=true} 
06:37:19,722 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader242 (Leader): Replicate message: identifier: MockIdentifier{value=four}, logIndex: 4, isSendImmediate: true
06:37:19,722 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower244: active: false, followerNextIndex: -1, leaderLastIndex: 4, leaderSnapShotIndex: 1
06:37:19,722 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 1
06:37:19,722 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
[WARN] [06/25/2025 18:37:19.722] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,728 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] ReplicatedLogImpl - leader242: persist complete JournaledLogEntry{index=4, term=1, command=four, pending=false}
06:37:19,835 PM [test-pekko.actor.default-dispatcher-14] [DEBUG] SyncStatusTracker - follower243: Lagging -1 entries behind leader leader242 and reached 0 (of expected -1)
[WARN] [06/25/2025 18:37:19.835] [test-pekko.actor.default-dispatcher-6] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [WARN] AbstractLeader - leader242 (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=1, success=true, followerId=follower243, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 466, lastApplied : 1, commitIndex : 1
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader242 (Leader): sendAppendEntries: 2 is present for follower follower243
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractLeader - leader242 (Leader): Sending AppendEntries to follower follower243: AppendEntries{term=1, leaderId=leader242, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}, JournaledLogEntry{index=4, term=1, command=four, pending=false}]}
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): handleAppendEntries: AppendEntries{term=1, leaderId=leader242, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=5, leaderRaftVersion=5, entries==[JournaledLogEntry{index=2, term=1, command=two, pending=false}, JournaledLogEntry{index=3, term=1, command=three, pending=false}, JournaledLogEntry{index=4, term=1, command=four, pending=false}]}
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): Number of entries to be appended = 3
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): After cleanup, lastIndex: 1, entries to be added from: 0
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): Append entry to log two
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower243: Append log entry and persist JournaledLogEntry{index=2, term=1, command=two, pending=false} 
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): Append entry to log three
06:37:19,835 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower243: Append log entry and persist JournaledLogEntry{index=3, term=1, command=three, pending=false} 
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): Append entry to log four
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] ReplicatedLogImpl - follower243: Append log entry and persist JournaledLogEntry{index=4, term=1, command=four, pending=false} 
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): Log size is now 5
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): Commit index set to 1
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): handleAppendEntries returning : AppendEntriesReply{term=1, success=true, followerId=follower243, logLastIndex=4, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=5, raftVersion=5, recipientRaftVersion=5}
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SyncStatusTracker - follower243: Lagging 0 entries behind leader leader242 and reached 1 (of expected -1)
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] Follower - follower243 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 4
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorBehavior - follower243 (Follower): Setting last applied to 1
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] AbstractReplicatedLog - follower243: Moving last applied index from 0 to 1
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower243: Applying state for log index 1 data one
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [INFO] MockRaftActor - follower243: applyState called: one
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader242 (Leader): handleAppendEntriesReply - FollowerLogInformation for follower243 updated: matchIndex: 4, nextIndex: 5
06:37:19,836 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActor - follower243: Persisting ApplyJournalEntries with index=1
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader242 (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 1
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader242 (Leader): Setting last applied to 2
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader242: Moving last applied index from 1 to 2
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader242: Applying state for log index 2 data two
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader242: applyState called: two
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader242 (Leader): Setting last applied to 3
06:37:19,836 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader242: Moving last applied index from 2 to 3
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader242: Applying state for log index 3 data three
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader242: applyState called: three
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActorBehavior - leader242 (Leader): Setting last applied to 4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractReplicatedLog - leader242: Moving last applied index from 3 to 4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader242: Applying state for log index 4 data four
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [INFO] MockRaftActor - leader242: applyState called: four
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] RaftActor - leader242: Persisting ApplyJournalEntries with index=4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower243: persist complete JournaledLogEntry{index=2, term=1, command=two, pending=false}
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower243: persist complete JournaledLogEntry{index=3, term=1, command=three, pending=false}
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] ReplicatedLogImpl - follower243: persist complete JournaledLogEntry{index=4, term=1, command=four, pending=false}
06:37:19,837 PM [test-pekko.actor.default-dispatcher-16] [DEBUG] AbstractLeader - leader242 (Leader): Checking sendAppendEntries for follower follower243: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 1
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): Commit index set to 4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower243: Lagging 0 entries behind leader leader242 and reached 4 (of expected -1)
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] Follower - follower243 (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 1, lastIndex(): 4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower243 (Follower): Setting last applied to 2
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower243: Moving last applied index from 1 to 2
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower243: Applying state for log index 2 data two
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower243: applyState called: two
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower243 (Follower): Setting last applied to 3
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower243: Moving last applied index from 2 to 3
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower243: Applying state for log index 3 data three
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower243: applyState called: three
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActorBehavior - follower243 (Follower): Setting last applied to 4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] AbstractReplicatedLog - follower243: Moving last applied index from 3 to 4
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower243: Applying state for log index 4 data four
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [INFO] MockRaftActor - follower243: applyState called: four
06:37:19,837 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] RaftActor - follower243: Persisting ApplyJournalEntries with index=4
06:37:19,955 PM [test-pekko.actor.default-dispatcher-6] [DEBUG] SyncStatusTracker - follower243: Lagging 0 entries behind leader leader242 and reached 4 (of expected -1)
[WARN] [06/25/2025 18:37:19.955] [test-pekko.actor.default-dispatcher-10] [pekko://test/] unhandled message from TestActor[pekko://test/user/leader242]: AppendEntries{term=1, leaderId=leader242, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=5, recipientRaftVersion=4, leaderRaftVersion=5, entries==[]}
06:37:20,25 PM [test-pekko.actor.default-dispatcher-10] [INFO] SnapshotManager - leader242: snapshot is durable as of 2025-06-25T18:37:19.472320852Z
06:37:20,25 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] SnapshotManager - leader242: Snapshot success -  sequence number: 6
06:37:20,25 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftActorSnapshotMessageSupport - leader242: SnapshotComplete received
06:37:20,26 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: handled a journal response
06:37:20,36 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - leader242: stopped executor with 0 remaining tasks
06:37:20,36 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - leader242: stopped executor with 0 remaining tasks
[WARN] [06/25/2025 18:37:20.038] [test-pekko.actor.default-dispatcher-15] [pekko://test/user/termination-monitor] received dead letter from TestActor[pekko://test/user/leader242]: org.opendaylight.controller.cluster.common.actor.Monitor@6a3d3ff8
06:37:20,39 PM [test-pekko.actor.default-dispatcher-10] [INFO] RaftActor - leader242: Starting recovery with journal batch size 1000
06:37:20,39 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - leader242: started executor
06:37:20,39 PM [test-pekko.actor.default-dispatcher-10] [DEBUG] RaftStorage - leader242: started executor
06:37:20,41 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader242: Set currentTerm=1, votedFor=leader242
06:37:20,41 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader242: selected SNAPSHOT_V1 to handle file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/snapshot-00000000685C41DF-1C270B54.v1
06:37:20,41 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader242: skipping unrecognized file /tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/TermInfo.properties
06:37:20,41 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader242: picked SnapshotFileV1{lastIncluded=EntryInfo[index=1, term=1], timestamp=2025-06-25T18:37:19.472320852Z, file=/tmp/org.opendaylight.controller.cluster.raft.RecoveryIntegrationTest9873792917405333164/leader242/snapshot-00000000685C41DF-1C270B54.v1, size=520, entryCompress=NONE, stateCompress=NONE} as the latest file
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [INFO] MockRaftActor - leader242: support() called
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader242: initializing from snapshot taken at 2025-06-25T18:37:19.472320852Z
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader242: Set currentTerm=1, votedFor=leader242
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - Recovery snapshot applied for leader242 in 58.55 μs: snapshotIndex=1, snapshotTerm=1, journal-size=2
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader242: Received ReplicatedLogEntry for recovery: index: 4, size: 4
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader242: Received apply journal entries for recovery, applying to state: 2 to 4
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractReplicatedLog - leader242: Moving last applied index from 1 to 3
06:37:20,42 PM [test-pekko.actor.default-dispatcher-15] [INFO] PekkoRecovery - leader242: Recovery completed in 401.2 μs - Switching actor to Follower - last log index = 4, last log term = 1, snapshot index = 1, snapshot term = 1, journal size = 3
06:37:20,43 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PekkoRecovery - leader242: restoring local TermInfo{term=1, votedFor=leader242}
06:37:20,43 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader242: Set currentTerm=1, votedFor=leader242
06:37:20,43 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftActor - leader242: Pekko recovery completed with org.opendaylight.controller.cluster.raft.RecoveryLog@1e6370da
06:37:20,43 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Follower - leader242 (Follower): Received TimeoutNow - switching to Candidate
06:37:20,43 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader242 (Candidate): Election: Candidate has following voting peers: []
06:37:20,45 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] PropertiesTermInfoStore - leader242: Set currentTerm=2, votedFor=leader242
06:37:20,45 PM [test-pekko.actor.default-dispatcher-15] [INFO] Candidate - leader242 (Candidate): Starting new election term 2
06:37:20,45 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader242 (Follower) :- Switching from behavior Follower to Candidate, election term: 2
06:37:20,45 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] Candidate - leader242 (Candidate): Received ElectionTimeout
06:37:20,45 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] AbstractLeader - leader242 (Leader): Election: Leader has following peers: []
06:37:20,45 PM [test-pekko.actor.default-dispatcher-15] [INFO] RaftActorBehavior - leader242 (Candidate) :- Switching from behavior Candidate to Leader, election term: 2
06:37:20,138 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/follower243-collector245#2091646757]
06:37:20,149 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/follower243]
06:37:20,149 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower243: stopped executor with 0 remaining tasks
06:37:20,149 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - follower243: stopped executor with 0 remaining tasks
06:37:20,149 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader242-collector246#1838642107]
06:37:20,150 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader242]
06:37:20,150 PM [main] [INFO] TestActorFactory - Killing actor Actor[pekko://test/user/leader242-collector247#1948074282]
06:37:20,150 PM [main] [INFO] TestActorFactory - Killing actor TestActor[pekko://test/user/leader242]
06:37:20,150 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader242: stopped executor with 0 remaining tasks
06:37:20,150 PM [test-pekko.actor.default-dispatcher-15] [DEBUG] RaftStorage - leader242: stopped executor with 0 remaining tasks
[INFO] [06/25/2025 18:37:20.152] [main] [CoordinatedShutdown(pekko://test)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
06:37:20,153 PM [test-pekko.actor.default-dispatcher-15] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
06:37:20,154 PM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - Exiting completed
06:37:20,155 PM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - Shutting down...
06:37:20,155 PM [test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://test@10.30.171.159:17355] - Successfully shut down
[WARN] [06/25/2025 18:37:20.153] [test-pekko.actor.default-dispatcher-10] [pekko://test/system/cluster/core/daemon] unhandled message from Actor[pekko://test/deadLetters]: Leave(pekko://test@10.30.171.159:17355)
[INFO] [06/25/2025 18:37:20.154] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - Exiting completed
[INFO] [06/25/2025 18:37:20.154] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - Shutting down...
[INFO] [06/25/2025 18:37:20.154] [test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://test)] Cluster Node [pekko://test@10.30.171.159:17355] - Successfully shut down
[INFO] [06/25/2025 18:37:20.156] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.171.159:17355/system/remoting-terminator] Shutting down remote daemon.
06:37:20,156 PM [test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.
[INFO] [06/25/2025 18:37:20.156] [test-pekko.remote.default-remote-dispatcher-8] [pekko://test@10.30.171.159:17355/system/remoting-terminator] Remote daemon shut down; proceeding with flushing remote transports.
06:37:20,156 PM [test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports.